Compare commits
33
Commits
ext-v1.0.6
..
v1.1.1
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
33f8b94bb0 | ||
|
|
0ed57b6d54 | ||
|
|
c29d48f38c | ||
|
|
f1e82ce4e5 | ||
|
|
2d412e3bba | ||
|
|
46d78c6e15 | ||
|
|
81a14f9b38 | ||
|
|
88dc6d5a54 | ||
|
|
4a2d89406b | ||
|
|
d8d435fead | ||
|
|
1f8e61f3bf | ||
|
|
a9ee858f53 | ||
|
|
8489d90926 | ||
|
|
5272acaea0 | ||
|
|
cba51238b6 | ||
|
|
5fffb800f9 | ||
|
|
1a729eb9ec | ||
|
|
00c763300e | ||
|
|
224b3d4a66 | ||
|
|
81d26a265a | ||
|
|
5b599125d1 | ||
|
|
18d89d6cf2 | ||
|
|
199b5c532f | ||
|
|
db5f0e830b | ||
|
|
7a449c40b1 | ||
|
|
97fda7aa10 | ||
|
|
a49592a3ba | ||
|
|
ed9ac03c0c | ||
|
|
cb46ad7abe | ||
|
|
586975090b | ||
|
|
544805266b | ||
|
|
f3051fb536 | ||
|
|
86b183e2d7 |
No files matched your search
@@ -150,3 +150,18 @@ When editing a file full of `json.set`:
|
||||
- Tables: `doc/design/stdlib-table.md` · `doc/design/table-cell-expressions.md`
|
||||
- Browser effects: `doc/roadmap/browser-wasm-e.md` · `doc/roadmap/browser-wasm-f.md`
|
||||
- Release: [marqdo-release](../marqdo-release/SKILL.md)
|
||||
|
||||
## 渐进式契约 + MLSP(Phase 2/3 落地后的开发戒律)
|
||||
|
||||
- **契约实现**:`src/contract.rs`(类型词汇 / 三形状契约表解析 / 合并 / 四边界校验诊断);
|
||||
`src/parse/mod.rs::extract_function_contract`(位置 + 形状双重判定,**在升参推断之前**执行;
|
||||
绑定表与普通文档表绝不吃 ⇒ 无契约文件零回归)。契约表从 body 移除——元数据永不执行。
|
||||
- **静态互查**:`src/check.rs`(`marqdo check [--json]`)——契约行 vs 升参推断形参双向漂移、
|
||||
未知类型名、错位契约;Error ⇒ 非零退出。
|
||||
- **AI 工具面**:`src/mlsp.rs`(`marqdo mlsp`,行分隔 JSON)——`locate` / `syntax` / `validate` /
|
||||
`repair_targets` / `schema`。语法唯一事实来源 = `parse::CORE_CONSTRUCTS`(CI 守卫与
|
||||
`doc/design/core-surface.md` 同步);MLSP 只加检索别名,**不定义语法**。
|
||||
- **准入戒律**:新特性先过 `doc/design/layers.md` 三层归属(Language 层改动须 ADR + core-surface);
|
||||
新契约形状/类型词汇 = 文档层扩展,进 `src/contract.rs`,不进词法/语法。
|
||||
- **错误出口唯一**:`src/load.rs::label_error` 已示范——装载层/任何新代码都**不许**把
|
||||
`Diagnostic` 拍平成字符串;错误链里必须能 `Diagnostic::find` 找回结构化诊断(AI/MLSP 面)。
|
||||
@@ -85,7 +85,7 @@ Checks: clean tree; `main` ff-only; smoke tests; CHANGELOG Unreleased; no secret
|
||||
|------|------|-----|-----|
|
||||
| `Cargo.toml` + `crates/marqdo-wasm/Cargo.toml` | bump | bump | — |
|
||||
| `ext/VERSION` | bump | — | bump |
|
||||
| `CHANGELOG.md` | `## vVER` | `## vVER` | `## Ext pack vVER` |
|
||||
| `CHANGELOG.md` | `## vVER`(含**核心表面 Δ**,见 `doc/design/layers.md`) | same | `## Ext pack vVER` |
|
||||
| `README.md` / `public/**` install + download table | update | update | update ext notes |
|
||||
| `.cursor/skills/marqdo/SKILL.md` | if surface changed | same | rare |
|
||||
| `.github/workflows/release.yml` Highlights stub | yes | yes | — |
|
||||
|
||||
@@ -36,6 +36,62 @@ Canonical design (repo): [`markdown-mapping-v0.3.md`](../../doc/design/markdown-
|
||||
11. **Code-as-documentation / no bag glue:** Prefer **GFM tables** for maps, lists, wire, commands. Prefer **`table.put` / named helpers** over `json.set` / `json.append` chains. `lib/json` is for parse/stringify/quote only — not a dict builder. Unreadable json pipelines are a style bug.
|
||||
12. **Bracket-marked calls:** Function name may be marked with `[函数名]`; bare words before `[` are boolean modifiers (`礼貌 [问候] x` → `礼貌=True`). `]` immediately followed by `(` is **index** `[key](coll)`, not a call. Prefer `[k](c)` over footnote index in new code. See [bracket-call-modifiers.md](../../doc/design/bracket-call-modifiers.md).
|
||||
|
||||
## 记不住语法细节?查 MLSP(AI 必读)
|
||||
|
||||
**不必背 Marqdo 语法细节**——语法可查、程序可验、修复有界。写码前后走**查询式**工作流:
|
||||
|
||||
```bash
|
||||
printf '%s\n' '{"id":1,"method":"syntax","params":{"query":"返回"}}' | marqdo mlsp
|
||||
printf '%s\n' '{"id":2,"method":"locate","params":{"path":"a.mq.md","symbol":"加一"}}' | marqdo mlsp
|
||||
printf '%s\n' '{"id":3,"method":"validate","params":{"path":"a.mq.md"}}' | marqdo mlsp
|
||||
printf '%s\n' '{"id":4,"method":"schema","params":{"path":"a.mq.md","unit":"加一"}}' | marqdo mlsp
|
||||
printf '%s\n' '{"id":5,"method":"repair_targets","params":{"code":"contract.arg_mismatch","doc_anchor":"a.mq.md#L9-L15"}}' | marqdo mlsp
|
||||
```
|
||||
|
||||
| method | 何时用 |
|
||||
|--------|--------|
|
||||
| `syntax` | 不确定某个语法怎么写(关键字/别名 → 构造卡片:「返回」「表」「调用」「循环」…) |
|
||||
| `locate` | 找符号定义位置 + 契约;或反查构造卡片 |
|
||||
| `validate` | 提交前校验(结构化诊断数组;`ok:false` 就修) |
|
||||
| `schema` | 查单元契约(形参/返回/字段) |
|
||||
| `repair_targets` | 拿错误的**有界**修复靶点:只改锚点范围内行,越界必弃权(abstain) |
|
||||
|
||||
行分隔 JSON:每行请求 `{"id","method","params"}` → 每行响应 `{"id","ok","result"|"error"}`。
|
||||
语法唯一事实来源 = `doc/design/core-surface.md`(CI 守卫同步);卡片带 `manual` 指回文档行。
|
||||
所有错误诊断都是结构化 JSON:`code` / `severity` / `span` / `suggestion` / `doc_anchor` / `doc_quote` / `contract_ref`。
|
||||
|
||||
### 渐进式契约(可选标注;schema 就长在文档表里)
|
||||
|
||||
契约是**未绑定文档表**(绑定表是数据,永远不是契约)。三种形状、严格位置:
|
||||
|
||||
| 形状(须含 `类型` 列) | 允许位置 | 附着 |
|
||||
|--------------------------|----------|------|
|
||||
| `参数`/`类型`/`说明` 表 + `返回`/`类型`/`说明` 表 | 函数体首个可执行行之前 | 函数 |
|
||||
| `字段`/`类型`/`可空`/`说明` 表 | 紧邻 `` `名` = `` 绑定之前 | 集合(变量) |
|
||||
| `字段`/`类型`/`可空`/`说明` 表 | 对象(`#`)体首成员之前 | 对象类型 |
|
||||
|
||||
````text
|
||||
## 加一
|
||||
|
||||
对于输入变量`n`,期望是数字。
|
||||
|
||||
| 参数 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| n | number | 输入值 |
|
||||
|
||||
| 返回 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| | number | 加一结果 |
|
||||
|
||||
*n+1*
|
||||
````
|
||||
|
||||
- 类型词汇:`text` / `number` / `bool` / `list` / `map` / `any` / `#` 对象名(拼错 ⇒ `contract.unknown_type` + 最近建议)。
|
||||
- 契约表**永不执行**、不污染升参推断;无契约文件行为逐字节不变(动态默认,零摩擦)。
|
||||
- 运行时只在四个边界校验:调用实参 / 字段访问 `[键](集合)` / 表绑定 / 返回。
|
||||
- `marqdo check [FILE] [--json]` 静态互查:契约行 vs 升参推断形参**双向**漂移(错行=Error;新升参未覆盖=Warning)、未知类型、错位契约;有 Error 非零退出。
|
||||
- 契约诊断带 `doc_anchor` + `doc_quote`(表原文)——修复只照锚点局部改。
|
||||
|
||||
## Markup → meaning (v0.3)
|
||||
|
||||
| Marker | Meaning |
|
||||
@@ -145,7 +201,7 @@ import clock:lib/time.mq.md
|
||||
- Instance methods stay `` > `obj`.method `` (backticks on the receiver only).
|
||||
- `lib/…` resolves via `MARQDO_LIB`, cwd `lib/`, or `lib/` next to the `marqdo` binary.
|
||||
- Design: [module-namespace.md](../../doc/design/module-namespace.md).
|
||||
- Official optional extensions (`ext/`, not stdlib): install with `marqdo ext add llm|agent|web|quantum|linalg` (see `doc/design/ext-cli.md`). `ext/llm` — chat; **`ext/agent`** — document-driven agents (see **ext/agent** below); **`ext/web`** — dynamic sites (see below); **`ext/quantum`** — circuits + Q7/Q8 (see below); **`ext/linalg`** — formula-first linear algebra (see below).
|
||||
- Official optional extensions (`ext/`, not stdlib): install with `marqdo ext add llm|agent|web|quantum|linalg` (see `doc/design/ext-cli.md`). **`ext/llm`** — intelligence primitive (`ask` / `stream` / `collect`; `complete` is compat); **`ext/agent`** — document-driven agents (see **ext/agent** below); **`ext/web`** — dynamic sites (see below); **`ext/quantum`** — circuits + Q7/Q8 (see below); **`ext/linalg`** — formula-first linear algebra (see below).
|
||||
- Native plugins: `lib/plugin` — `## load` / `unload` / `list`. C ABI: `include/marqdo_abi.h`.
|
||||
- Prefer `table.put` / `表.改` for list/map element updates; keep `json` for parse/stringify/quote (see `doc/design/stdlib-table.md`).
|
||||
- Builtins (no import): `print`/`打印`, `input`/`输入`, `len`/`长度`, `str`/`文本`, `int`/`整数`; literals `True`/`真`, `False`/`假`, `None`/`空`; logic `and`/`且`, `or`/`或`, `not`/`非`.
|
||||
@@ -161,10 +217,12 @@ Install: `marqdo ext add agent` (ZH: `智能体`). Native: `cargo build --releas
|
||||
1. **Code is documentation** — the runbook `.mq.md` is the model’s ground truth, not a hidden system prompt bag.
|
||||
2. **Call site is first-class** — `build_step_context` injects path/function/line so the model knows where it is in the program.
|
||||
3. **Document is the knowledge base (OKF)** — writeback → promote/solidify → reuse / `llm_free`; MCP/corpus are evidence only.
|
||||
4. **Skill Compilation (v2)** — successful runs become `.mq.md` episodes; after enough evidence, `maybe_learn` compiles an executable skill (`llm_free`). Prefer `## run` over bare `## plan` when you want learning.
|
||||
5. **Adaptive Routing (P4)** — router is abstract (`marqdo` / `small-llm` / `llm` / `auto`); `jev` is a compat alias of `small-llm`. Decision via `router_model` (any llm handle). Policy Compilation writes `policies/router.mq.md`.
|
||||
|
||||
APIs: `# agent` / `# 智能体` → `## step` / `## plan`. Context: standing + task + **call site** + tools + budgeted source/skill + act protocol (`CALL:` / `READ:`). Inspect without LLM: `dump_step_context`. Inside `writeback=` / `写回=` branches use `host_writeback_record` (param name shadows the writeback import).
|
||||
APIs: `# agent` / `# 智能体` → `## run` / `## route` / `## step` / `## plan` (+ `resolve` / `record_episode` / `maybe_learn` / `compile` / `compile_policy` / `metrics`). Context: standing + task + **call site** + tools + budgeted source/skill + act protocol (`CALL:` / `READ:`). Inspect without LLM: `dump_step_context`. Inside `writeback=` / `写回=` branches use `host_writeback_record` (param name shadows the writeback import).
|
||||
|
||||
Examples: [agent-pong](../../examples/agent-pong/) · [agent-okf-flywheel](../../examples/agent-okf-flywheel/). Harness: `scripts/agent-harness.sh`. Design: [ext-agent.md](../../doc/design/ext-agent.md) · Wave B: [agent-framework-2026-09.md](../../doc/research/agent-framework-2026-09.md). Dev LLM: copy [.env.example](../../.env.example) → `.env` (gitignored).
|
||||
Examples: [agent-pong](../../examples/agent-pong/) · [agent-okf-flywheel](../../examples/agent-okf-flywheel/). Harness: `scripts/agent-harness.sh`. Design: [ext-agent.md](../../doc/design/ext-agent.md) · Wave B: [agent-framework-2026-09.md](../../doc/research/agent-framework-2026-09.md) · v2: [doc/next/003.md](../../doc/next/003.md) · P4: [doc/next/004.md](../../doc/next/004.md). Dev LLM: copy [.env.example](../../.env.example) → `.env` (gitignored).
|
||||
|
||||
## Official extension: `ext/web` (dynamic sites)
|
||||
|
||||
@@ -244,11 +302,14 @@ Examples: [linalg-svd](../../examples/linalg-svd/) · [linalg-least-squares](../
|
||||
|
||||
## AI authoring workflow
|
||||
|
||||
1. Decide language surface (English builtins + `lib/text.mq.md`, or Chinese + `lib/文本.mq.md`).
|
||||
2. Write `.mq.md` with `# main`, blank lines around prose, correct markers.
|
||||
3. Run: `marqdo run path/to/file.mq.md` (cwd = project root so `lib/` resolves).
|
||||
4. On `path:line:col: message`, fix that span; re-run until exit 0.
|
||||
5. Optional: `marqdo view .` / `marqdo debug .` for structure and breakpoints.
|
||||
1. **查再写**:不确定语法就 `marqdo mlsp`(`syntax` / `locate`)——不必背语法细节。
|
||||
2. Decide language surface (English builtins + `lib/text.mq.md`, or Chinese + `lib/文本.mq.md`).
|
||||
3. Write `.mq.md` with `# main`, blank lines around prose, correct markers.
|
||||
4. 可选:给函数/集合写**渐进式契约**(上节三种表格形状;写全或不写,别写一半)。
|
||||
5. Run: `marqdo run path/to/file.mq.md` (cwd = project root so `lib/` resolves).
|
||||
6. On errors:结构化诊断(含 `doc_anchor` / `suggestion`)→ `repair_targets` 取有界靶点,**只改锚点范围内**的行。
|
||||
7. 提交前:`marqdo check path/to/file.mq.md`(或 `mlsp validate`)清到零 Error。
|
||||
8. Optional: `marqdo view .` / `marqdo debug .` for structure and breakpoints.
|
||||
|
||||
## Anti-patterns (common model mistakes)
|
||||
|
||||
@@ -287,3 +348,5 @@ Examples: [linalg-svd](../../examples/linalg-svd/) · [linalg-least-squares](../
|
||||
- [ ] Browser client: `lib/browser` + tables when returning effects
|
||||
- [ ] Quantum draw: quote `kind=` / `theme=`; rebuild plugin → `ext add quantum` before view
|
||||
- [ ] `marqdo run` succeeds (or errors addressed)
|
||||
- [ ] 语法拿不准的地方已查 MLSP(`syntax`/`locate`),而不是凭记忆发明
|
||||
- [ ] 契约(如有)与升参一致:`marqdo check` 零 Error
|
||||
@@ -0,0 +1,15 @@
|
||||
## 层归属(必填,见 doc/design/layers.md)
|
||||
|
||||
- [ ] Language(`src/lex` `src/parse` `src/ast` `src/interp` `src/diagnostics` / 契约)
|
||||
- [ ] Runtime(`lib/` `plugins/` `crates/` catalog / view / debug / builtin)
|
||||
- [ ] Document(`*.mq.md` `skills/` `ext/` `public/` `examples/`)
|
||||
|
||||
## 核心表面(见 doc/design/core-surface.md)
|
||||
|
||||
- [ ] 不涉及核心标记
|
||||
- [ ] 涉及核心标记:ADR 编号 ______;core-surface.md 与 `CORE_CONSTRUCTS` 已同步(CI 守卫)
|
||||
|
||||
## 验收
|
||||
|
||||
- [ ] 相关验收(机器可验证)全绿
|
||||
- [ ] 全量回归绿(`cargo test` / `go test ./...` 视触及面)
|
||||
@@ -38,7 +38,7 @@ jobs:
|
||||
run: |
|
||||
export PATH="/c/ProgramData/mingw64/mingw64/bin:/c/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/bin:$PATH"
|
||||
cargo build --release
|
||||
cargo build --release -p marqdo_plugin_agent -p marqdo_plugin_quantum -p marqdo_plugin_linalg
|
||||
cargo build --release -p marqdo_plugin_agent -p marqdo_plugin_quantum -p marqdo_plugin_linalg -p marqdo_plugin_llm
|
||||
bash ./scripts/build-web-plugin.sh
|
||||
|
||||
- name: Checkout extension sources (branch vscode-extension only)
|
||||
@@ -87,6 +87,7 @@ jobs:
|
||||
Copy-Item target/release/agent.dll "$stageNative/native/agent.dll"
|
||||
Copy-Item target/release/quantum.dll "$stageNative/native/quantum.dll"
|
||||
Copy-Item target/release/linalg.dll "$stageNative/native/linalg.dll"
|
||||
Copy-Item target/release/llm.dll "$stageNative/native/llm.dll"
|
||||
if (Test-Path "dist/$nativeZip") { Remove-Item "dist/$nativeZip" -Force }
|
||||
Compress-Archive -Path "$stageNative/*" -DestinationPath "dist/$nativeZip" -Force
|
||||
$stage = "dist/stage-bundle"
|
||||
@@ -100,6 +101,7 @@ jobs:
|
||||
Copy-Item target/release/agent.dll "$stage/ext/native/agent.dll"
|
||||
Copy-Item target/release/quantum.dll "$stage/ext/native/quantum.dll"
|
||||
Copy-Item target/release/linalg.dll "$stage/ext/native/linalg.dll"
|
||||
Copy-Item target/release/llm.dll "$stage/ext/native/llm.dll"
|
||||
if (Test-Path "dist/$zip") { Remove-Item "dist/$zip" -Force }
|
||||
Compress-Archive -Path "$stage/*" -DestinationPath "dist/$zip" -Force
|
||||
$stageLib = "dist/stage-stdlib"
|
||||
@@ -124,9 +126,12 @@ jobs:
|
||||
## Marqdo ${{ github.ref_name }}
|
||||
|
||||
### 亮点 / Highlights
|
||||
- **v1.0.3**:Ubuntu PPA + 扩展 CDN(ext.marqdo.com);CLI/扩展可分发;noble 捆绑 Rust 工具链
|
||||
- **v1.0.3**: Ubuntu PPA + extension CDN; independent CLI/ext packs; bundled Rust for noble
|
||||
- **v1.0.1**:修复 Go `libweb` 动态路由 `{param}` 查询代入、详情 Markdown、CSS 数字误报
|
||||
- **v1.1.1**:AI 栈收口——llm `ask`/`stream`/`collect` · Metadata Binding · Skill Compilation + Adaptive Routing · MCP 客户端 + resume · 可选 `plugins/llm`
|
||||
- **v1.1.1**: AI stack — llm ask/stream/collect, Metadata Binding, Skill Compilation + Adaptive Routing, MCP client + resume, optional plugins/llm
|
||||
- **v1.1.0**:AI 原生闭环三件套——核心表面守卫 · 渐进式契约 + `marqdo check` · **MLSP for AI** · `run --json`
|
||||
- **v1.1.0**: AI-native loop — core-surface guard, progressive contracts + `marqdo check`, **MLSP for AI**, `run --json`
|
||||
- **v1.0.7**:WS/网络边界加固;`url_encode` `%20`
|
||||
- **v1.0.3**:Ubuntu PPA + 扩展 CDN(ext.marqdo.com)
|
||||
- **Linux 包必出**:`*-native-x86_64-unknown-linux-gnu.zip` + Linux CLI bundle
|
||||
- Details: [CHANGELOG](https://github.com/cflmy/marqdo/blob/main/CHANGELOG.md) · [mapping v0.3](https://github.com/cflmy/marqdo/blob/main/doc/design/markdown-mapping-v0.3.md)
|
||||
|
||||
@@ -136,7 +141,7 @@ jobs:
|
||||
|-------|----------|
|
||||
| `marqdo-*-windows-msvc.exe` | CLI with **embedded** official stdlib |
|
||||
| `marqdo-*-windows-msvc.zip` | `marqdo.exe` + `lib/` + `ext/`(含 native DLL) |
|
||||
| `marqdo-*-native-*.zip` | 预编译 `web`/`agent`/`quantum`/`linalg`(Windows DLL + **Linux .so 必出**) |
|
||||
| `marqdo-*-native-*.zip` | 预编译 `web`/`agent`/`quantum`/`linalg`/`llm`(Windows DLL + **Linux .so 必出**) |
|
||||
| `marqdo-*-unknown-linux-gnu.zip` | Linux CLI + `lib/` + `ext/` + native `.so` |
|
||||
| `marqdo-*-stdlib.zip` | `lib/` only |
|
||||
| `marqdo-*-ext.zip` | Official `ext/` L1 sources |
|
||||
@@ -181,7 +186,7 @@ jobs:
|
||||
run: |
|
||||
set -euo pipefail
|
||||
cargo build --release
|
||||
cargo build --release -p marqdo_plugin_agent -p marqdo_plugin_quantum -p marqdo_plugin_linalg
|
||||
cargo build --release -p marqdo_plugin_agent -p marqdo_plugin_quantum -p marqdo_plugin_linalg -p marqdo_plugin_llm
|
||||
bash ./scripts/build-web-plugin.sh
|
||||
|
||||
- name: Package Linux native zip + CLI bundle
|
||||
@@ -190,7 +195,7 @@ jobs:
|
||||
VER="${GITHUB_REF_NAME#v}"
|
||||
TARGET="x86_64-unknown-linux-gnu"
|
||||
mkdir -p dist/stage-native/native
|
||||
for lib in libweb.so libagent.so libquantum.so liblinalg.so; do
|
||||
for lib in libweb.so libagent.so libquantum.so liblinalg.so libllm.so; do
|
||||
src="target/release/${lib}"
|
||||
if [[ ! -f "$src" && "$lib" == libweb.so ]]; then
|
||||
src="plugins/web/build/libweb.so"
|
||||
|
||||
@@ -84,3 +84,6 @@ plugins/*/tmp/
|
||||
|
||||
# ephemeral sqlite / test artifacts
|
||||
tests/ext/:memory:
|
||||
|
||||
# local secrets
|
||||
.env
|
||||
+115
@@ -8,6 +8,121 @@
|
||||
|
||||
### Changed
|
||||
|
||||
## v1.1.1 — 2026-09-24
|
||||
|
||||
### Highlights
|
||||
|
||||
**AI 栈收口(agent + llm)**:语义 `ask`/`stream`/`collect`、Artifact Metadata Binding、Skill Compilation + Adaptive Routing、MCP 客户端与 resume 检查点、可选 `plugins/llm` Transport ABI。
|
||||
本版 CLI 与扩展包 **同步为 1.1.1**。
|
||||
|
||||
- **LLM 原语**:`ext/ai/llm` 从 OpenAI HTTP 客户端提升为 `ask`/`stream`/`collect` 智能原语(`complete` 保留兼容);`LLMResult.tool_calls`;可选原生 `llm_chat_completions`(无 `.so` 时回退 `lib/net`)。
|
||||
- **Metadata Binding**:frontmatter `${env|arg|sys|secret}` 加载期解析;Phase 2 将绑定键注入入口 env(正文禁止 `${…}`);`sys.meta` / `--bind`;llm ctor 优先 meta。
|
||||
- **Agent**:Skill Compilation(episode → llm_free skills)+ Adaptive Routing(`marqdo` / `small-llm` / `llm` / `auto`;`jev` 为 `small-llm` 兼容别名);MCP **stdio 客户端**;resume 检查点;零 LLM 金样路径。
|
||||
- **度量与后端**:`model_turn` usage、named handles(fast/reasoning)、`type:prompt` 加载、openai/ollama 分流。
|
||||
|
||||
```bash
|
||||
# Ubuntu PPA / 源码
|
||||
sudo add-apt-repository ppa:cflmy/marqdo && sudo apt update && sudo apt install marqdo
|
||||
git checkout v1.1.1 && cargo build --release
|
||||
marqdo ext add llm && marqdo ext add agent
|
||||
```
|
||||
|
||||
**核心表面 Δ(Core surface Δ)**:**无增删**——19 个核心构造不变。本版为 **Runtime**(Metadata Binding、LLM/agent 插件与扩展)变更,不触碰语言标记。
|
||||
|
||||
### Added
|
||||
- **`ext/ai/llm` 语义表面**:`ask` / `stream` / `collect`(`complete` 兼容);`LLMResult.tool_calls`(OpenAI-shaped `choices[0].message.tool_calls`);`sys.module_source` + 入口文档 `prompt_body` 离线路径。
|
||||
- **`plugins/llm` Transport ABI**(`llm_chat_completions`):`ext/ai/llm/openai` 优先原生插件,否则 `lib/net` HTTP/SSE。
|
||||
- **Artifact Metadata Binding**:frontmatter `${env|arg|sys|secret}` → `Module.metadata`;Secret 掩码;`sys.meta` / `--bind`;Phase 2 入口 env 注入(正文拒绝 `${…}`;字节码 prologue 同源)。
|
||||
- **Agent Skill Compilation + Adaptive Routing**:episode → `maybe_learn` → llm_free skills;路由后端 `marqdo` / `small-llm` / `llm` / `auto`(`jev` = `small-llm` 兼容别名,不再探测 `MARQDO_JEV`)。
|
||||
- **Agent MCP stdio 客户端**(`agent_mcp_client` / `mcp_connect` / `mcp_client_list` / `mcp_client_call`)+ Python mock 离线金样。
|
||||
- **Agent resume 检查点**(`.marqdo/agent-resume/<id>.json` — save/load/clear/list;`run` 可设 `resume_id=`)。
|
||||
- **AI 栈度量与后端**:`model_turn` usage、named handles(fast/reasoning)、`type:prompt` 加载、openai/ollama 分流、agent-zero-llm 金样。
|
||||
|
||||
### Fixed
|
||||
- **plan force 不再冲掉已有 `resources` workbook**(保留 llm_free 资源;金样 `agent-plan-preserve`)。
|
||||
- **compiled-skill Marqdo 语法**:llm_free await 不再误落到 plan+HTTP。
|
||||
|
||||
### Changed
|
||||
- Adaptive Routing:决策面为 `router_model`(任意 llm handle);`jev` 降为 `small-llm` 文档/兼容别名。
|
||||
|
||||
## v1.1.0 — 2026-09-23
|
||||
|
||||
### Highlights
|
||||
|
||||
**AI 原生闭环三件套落地**:核心表面守卫(问题一)· 渐进式契约(问题二)· MLSP for AI(问题三)。
|
||||
`marqdo check` 静态契约防漂移、`marqdo mlsp` 让 AI **按需查询**语法而非背诵、`run --json` 统一结构化诊断(错误即数据,供 AI 直接消费)。
|
||||
本版 CLI 与扩展包 **同步为 1.1.0**。
|
||||
|
||||
- **核心要小(问题一)**:`doc/design/core-surface.md` 列出 19 个核心构造 + `CORE_CONSTRUCTS` 守卫测试——任一侧(清单/文档/金样)单边改动 CI 必红;三层架构(Core / Runtime / Document)入 `doc/design/layers.md`。
|
||||
- **语义校验(问题二)**:渐进式契约表(`参数`/`返回`/`字段`/集合)嵌入文档叙述;四边界校验(调用实参 / 返回值 / 表绑定 / 键访问);`marqdo check` 静态防漂移(类型 typo / 参数名漂移 / 重复契约必报)。
|
||||
- **AI 可靠反馈 + 不必学语法(问题三)**:`marqdo mlsp`(MLSP,行分隔 JSON over stdio)提供 `locate` / `syntax` / `validate` / `repair_targets` / `repair_apply` / `schema`;`syntax` 卡片机械携带戒律原文 / 样例 / 文档锚点,AI 无需预学语法。修复护栏:有界行编辑、越界必拒(abstain)。
|
||||
- **验证(P4)**:锚点消融 anchor 一轮修复率 **100%**(双模型,plain 85%/90%);契约消融拦截 100% / 误报 0%;防漂移必报 9/9;核心守卫演练 3/3 必红。详见 [perf-validation-report-2026-09-23.md](doc/roadmap/perf-validation-report-2026-09-23.md)。
|
||||
|
||||
```bash
|
||||
# Ubuntu PPA / 源码
|
||||
sudo add-apt-repository ppa:cflmy/marqdo && sudo apt update && sudo apt install marqdo
|
||||
git checkout v1.1.0 && cargo build --release
|
||||
marqdo ext add web
|
||||
```
|
||||
|
||||
**核心表面 Δ(Core surface Δ)**:**无增删**——19 个核心构造(`heading` / `bold-code` / `italic-return` / `param-list` / `call-quote` / `branch-list` / `loop-list` / `table-bind` / `footnote-index` / `link-index` / …)与 v1.0.7 完全一致,未新增标记。本版为 **Runtime 层(契约校验 / MLSP / 诊断 JSON)** 与 **Document 层(core-surface / layers)** 变更,不触碰语言核心。
|
||||
|
||||
### Added
|
||||
- **MLSP(`marqdo mlsp`)**:面向 AI 的语言服务协议(行分隔 JSON over stdio)——`locate` / `syntax` / `validate` / `repair_targets` / `repair_apply` / `schema`;`syntax` 卡片含 `semantics` / `rule` / `sample` / `doc_quote`(自持,机器可读戒律原文)。
|
||||
- **`marqdo check`**:静态契约漂移检查(未知类型词 / 参数名漂移 / 返回越约 / 重复契约表 / 错位契约表)。
|
||||
- **渐进式契约(Progressive contracts)**:文档叙述内嵌契约表(`参数`/`返回`/`字段` + 集合 `字段`),四边界校验(调用实参 / 返回值 / 表绑定 / 键访问);被绑定的 `字段|类型` 表是**数据**、永不误判为契约。
|
||||
- **`marqdo run --json`**:统一结构化诊断(`code` / `severity` / `suggestion` / `doc_anchor` / `doc_quote` / `contract_ref`),错误即数据。
|
||||
- **修复护栏**:`mlsp repair_apply` 有界行编辑——只允许落在 `doc_anchor` 指定范围内的编辑,越界整体拒绝(all-or-nothing / abstain)。
|
||||
- **守卫与文档**:`doc/design/core-surface.md`(19 构造)、`doc/design/layers.md`(三层架构)、`tests/core_surface.rs`(守卫生效验证)。
|
||||
|
||||
### Fixed
|
||||
- **错误链不再扁平化**:`src/load.rs` 以 `label_error` 保留结构化 `Diagnostic`("Error Exit Only" 原则),修复契约类错误被降级为字符串的回归。
|
||||
- **契约 parse 边界**:`interp.rs` 键访问优先给出 `index_diag`(索引/键诊断先于取值),空字段 vs 缺字段语义消歧。
|
||||
|
||||
### Changed
|
||||
- **`.cursor/skills/marqdo`**:由「背诵式压缩操作手册」改为**查询式工作流**——写码前 `mlsp syntax` / `locate`,提交前 `mlsp validate` + `marqdo check`,修复照 `doc_anchor` 局部改。
|
||||
- **契约表一经出现即为完整声明**(新推断的升参未覆盖 ⇒ `contract.param_drift` 警告,不静默)。
|
||||
- 取元**主推** `[键](集合)`(`link-index`);`` `集合`[^键] ``(`footnote-index`)保留兼容。
|
||||
|
||||
### 已知未决(诚实标注)
|
||||
- **P4 实验 (b) 盲测未达标**:「查询式 vs 全文档背诵」四判据未全立(`deepseek-flash`:docs 4/6 vs query 0/3;端点 infra 削样本)。查询机制本身已验证无缺陷,瓶颈在**模型自查纪律**(不知道自己不知道 ⇒ 不查询)——协议改进(写码前强制自检 / 陌生构造强制查询)列入下一阶段。见 [perf-validation-report-2026-09-23.md](doc/roadmap/perf-validation-report-2026-09-23.md) §3。
|
||||
|
||||
## Unreleased
|
||||
|
||||
## v1.0.7 — 2026-09-22
|
||||
|
||||
### Highlights
|
||||
|
||||
**WS / 网络边界加固**:WebSocket 握手同源校验、broadcast/room 默认需登录(`require_auth`/`需要登录` 可显式关闭)、消息 1 MiB × 256 条上限、空 room 回收;HTTP 响应 8 MiB 在读流时截断;`url_encode` 空格 `%20`。本版 CLI 与扩展包 **同步为 1.0.7**。
|
||||
|
||||
- **安全默认**:浏览器 WS 握手仅同源;有状态 fan-out 端点未登录 401 / 公开直播流显式 `require_auth=False`
|
||||
- **资源上限**:WS 单条 1 MiB、单连接 256 条;HTTP/SSE 8 MiB 读流截断——对端无法借大响应耗尽内存
|
||||
- **测试**:P3 金样对齐 quiet-admin 门禁语义;wasm smoke 以 ESM 加载官方 bridge
|
||||
|
||||
```bash
|
||||
# Ubuntu PPA / 源码
|
||||
sudo add-apt-repository ppa:cflmy/marqdo && sudo apt update && sudo apt install marqdo
|
||||
git checkout v1.0.7 && cargo build --release
|
||||
marqdo ext add web
|
||||
```
|
||||
|
||||
### Fixed
|
||||
- **WS 安全默认**:握手改为同源校验(不再放行任意 Origin);`route_ws` 新增 `require_auth`/`需要登录`,broadcast/room 默认要求登录会话,公开直播流需显式设假,未认证握手 401。
|
||||
- **WS 资源上限**:单条消息 1 MiB、单连接最多收集 256 条回复;传输错误/超时返回 `{ok:false, error}`,不再伪装空成功;空 room 自动回收(防 URL 派生 room 名耗尽内存)。
|
||||
- **HTTP 响应上限**:8 MiB 上限在读流时截断(含错误路径与 SSE 错误路径),上游无法借超大响应耗尽内存。
|
||||
|
||||
### Changed
|
||||
- `net.url_encode`:空格编码为 `%20`(原 `+`),URL 段语义下与字面 `+` 消歧义。
|
||||
- `web`/`网页` 文档同步:`route_ws` 的 `require_auth`/`需要登录` 与 `ws.connect` 的收集上限语义。
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Added
|
||||
|
||||
### Fixed
|
||||
|
||||
### Changed
|
||||
|
||||
## Ext pack v1.0.6 — 2026-09-18
|
||||
|
||||
### Highlights
|
||||
|
||||
Generated
+12
-2
@@ -572,7 +572,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
|
||||
|
||||
[[package]]
|
||||
name = "marqdo"
|
||||
version = "1.0.3"
|
||||
version = "1.1.1"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"assert_cmd",
|
||||
@@ -603,7 +603,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "marqdo-wasm"
|
||||
version = "1.0.3"
|
||||
version = "1.1.1"
|
||||
dependencies = [
|
||||
"marqdo",
|
||||
"serde_json",
|
||||
@@ -632,6 +632,16 @@ dependencies = [
|
||||
"serde_json",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "marqdo_plugin_llm"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"idna_adapter",
|
||||
"serde_json",
|
||||
"ureq",
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "marqdo_plugin_quantum"
|
||||
version = "0.1.0"
|
||||
|
||||
+2
-1
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "marqdo"
|
||||
version = "1.0.3"
|
||||
version = "1.1.1"
|
||||
edition = "2021"
|
||||
license = "Apache-2.0"
|
||||
description = "Marqdo — Markdown markers as an interpreted programming language"
|
||||
@@ -58,6 +58,7 @@ getrandom = { version = "=0.2.17", default-features = false, features = ["custom
|
||||
members = [
|
||||
"plugins/demo",
|
||||
"plugins/agent",
|
||||
"plugins/llm",
|
||||
"plugins/quantum",
|
||||
"plugins/linalg",
|
||||
"crates/marqdo-wasm",
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# Marqdo
|
||||
|
||||
**代码即文档。文档即代码。文档即知识库。**
|
||||
**代码即文档。文档即知识库。知识即可执行。**
|
||||
|
||||
Marqdo 把 Markdown **标记当作编程语法**:同一个 `.mq.md` 文件,既是给人读的文稿,也是给机器跑的程序。写文档的过程就是在写可执行逻辑——大型项目里,说明、示例与实现不再分叉成三份真相。
|
||||
Marqdo 把 Markdown **标记当作编程语法**:同一个 `.mq.md` 文件,既是给人读的文稿,也是给机器跑的程序。写文档的过程就是在写可执行逻辑——大型项目里,说明、示例与实现不再分叉成三份真相。它不是「又一个 Markdown 编程语言」,而是**文档原生的编程运行时(Document-native runtime)**——一门关于**可执行知识(Executable Knowledge)**的语言。
|
||||
|
||||
欢迎访问 [marqdo 官方网站](https://www.marqdo.com/) 阅读更多特性与可执行文档。
|
||||
|
||||
@@ -10,6 +10,47 @@ Marqdo 把 Markdown **标记当作编程语法**:同一个 `.mq.md` 文件,
|
||||
|
||||
---
|
||||
|
||||
## 核心优势:可执行知识(Executable Knowledge)
|
||||
|
||||
> **Knowledge That Can Run —— 知识即可执行。**
|
||||
> Marqdo 不是「用 Markdown 写程序」这么简单:它让一个 `.mq.md` 文档**同时**是文稿、知识与程序——同一份结构,同时服务于人、解释器、工具与 AI。
|
||||
|
||||
```text
|
||||
.mq.md(同一份文件)
|
||||
┌─────────┼─────────┐
|
||||
文稿 知识 程序
|
||||
└─────────┼─────────┘
|
||||
▼
|
||||
Marqdo 运行时
|
||||
▼
|
||||
可执行知识(Executable Knowledge)
|
||||
```
|
||||
|
||||
| 真正的核心优势 | 说明 |
|
||||
|------|------|
|
||||
| **知识-代码一致性**<br>Knowledge-Code Coherence | 说明、数据、示例与实现**同一份源文件**:文档里的表格就是程序的数据,文档里的调用就是真实执行。「文档过期」不再是悄悄漂移,而是可被机器发现的**知识不一致**——知识陈述本身可以被测试。 |
|
||||
| **默认安全,AI 友好** | 无标记文字默认是叙述(注释),只有显式标记才进入执行。AI 自由发挥**不会污染**文档与代码的边界——这比「自然语言编程」安全得多,是 AI 原生语言的重要前提。 |
|
||||
| **multi-view artifact**<br>`.mq.md` 双后缀即开放接口 | `.mq` 告诉解释器「可执行」,`.md` 告诉整个 Markdown 生态「仍是文档」。GitHub 直接读、VS Code 直接改、搜索引擎直接索引、LLM 直接理解、Marqdo 直接运行。**Marqdo 不占有知识,只是给 Markdown 增加执行语义。** |
|
||||
| **表格即数据结构** | 程序的数据结构直接就是文档的数据结构:单列竖表 → `List`;横表(1 数据行)→ `Map`;横表(多数据行)→ `Map` of `List`(列向聚合);首列写 `@` / `行` / `row` → 行向记录 `List` of `Map`。取元 `[键](集合)` 也是 Markdown 链接形。这是 literate programming 的下一步。 |
|
||||
| **语义干净** | **返回是架构,打印只是函数**:`*…*` 改变控制流,`print` 只是副作用函数;分支与循环长在 Markdown 结构上(`1.` `2.` 递增 = 分支,`-` = 循环),不是所有列表都被粗暴当成 if。 |
|
||||
| **AI 的可靠上下文与长期记忆** | 一个 `.mq.md` 同时是 documentation + schema + code + examples + knowledge + executable behavior;AI 不必再拼装 README + OpenAPI + JSON Schema + prompt。整个工程天然就是 AI 的**世界模型**——比 RAG 更接近 Executable RAG。 |
|
||||
| **完整工程,不是玩具** | Rust 正经解释器(词法 / 语法 / 语义 / 树遍历 + 字节码后端)、stdlib、模块与对象、debugger、viewer、catalog、WASM、原生插件 ABI、VS Code 扩展、agent / LLM / MCP。 |
|
||||
|
||||
---
|
||||
|
||||
## 愿景与期许
|
||||
|
||||
不把「AI 能写 Marqdo」当终点,而是 **「AI 能可靠地生成可验证的 Marqdo」**:
|
||||
|
||||
1. **AI 可验证语言**:`LLM → Marqdo AST → 语义检查 → 结构化诊断 → LLM 修复 → 合格程序`。让类型与语义检查成为 AI 编程闭环里的强约束,堵住「语义看似合理、类型实则不对」的生成。
|
||||
2. **Marqdo Schema / Language Server for AI**:机器可读的语言 Schema(`get_syntax` / `validate` / `repair` …),AI 不必读千行文档猜语法,而是 discover → construct → validate → execute → inspect。
|
||||
3. **程序即 AI 长期记忆**:`.mq.md` 工程 = 可执行知识库 = AI 的世界模型(Executable Knowledge Representation)。
|
||||
4. **可执行知识图**:`.mq.md` 之间的 Markdown 链接天然长成文档图 + 知识图 + 程序图(Executable Knowledge Graph)。
|
||||
5. **知识可测试**:文档中的示例表自动执行;知识与执行结果不一致会被机器报告——Knowledge-Code Coherence 从理念变为日常。
|
||||
6. **可靠性基准**:让同一个 AI 分别用 Python / TypeScript / Marqdo 完成真实任务,测量生成长度、语法 / 语义错误率、修复轮次、token 消耗与人类阅读 / 修改成本——用实验数据证明「文档型语法降低 LLM 生成熵」。
|
||||
|
||||
---
|
||||
|
||||
## 为什么重要
|
||||
|
||||
| 痛点 | Marqdo 的答案 |
|
||||
@@ -38,7 +79,7 @@ Marqdo 把 Markdown **标记当作编程语法**:同一个 `.mq.md` 文件,
|
||||
| 叙述 `` `名` `` / `` `名`=默认 `` | 形参由体推断(`` + `名` `` 仍可用) |
|
||||
| `1.` … | 体内 `1.` `2.` … = 分支(`N. *` = else) |
|
||||
| `-` | 循环(`` - `条件` `` 或 `` - [项](集合) ``) |
|
||||
| 表格 | 集合;取元优先 `[键](集合)` |
|
||||
| 表格 | 集合(几何即类型,见下);取元优先 `[键](集合)` |
|
||||
| 单独一行 `---` / `***` | **叙述分隔**(跳过;不是函数收束) |
|
||||
|
||||
```markdown
|
||||
@@ -58,6 +99,8 @@ Marqdo 把 Markdown **标记当作编程语法**:同一个 `.mq.md` 文件,
|
||||
对于输入变量`n`,执行**n=n+1**接着返回*n*。
|
||||
```
|
||||
|
||||
**表格几何即类型**:单列竖表 → `List`;横表(1 数据行)→ `Map`;横表(多数据行)→ `Map` of `List`(列向聚合);首列写 `@` / `行` / `row` → 行向记录 `List` of `Map`。详见 [tables-maps-footnotes.md](doc/roadmap/tables-maps-footnotes.md)。
|
||||
|
||||
完整约定:[markdown-mapping-v0.3.md](doc/design/markdown-mapping-v0.3.md)。
|
||||
|
||||
---
|
||||
@@ -68,7 +111,8 @@ Marqdo 把 Markdown **标记当作编程语法**:同一个 `.mq.md` 文件,
|
||||
2. **返回是架构;打印只是函数。**
|
||||
3. **代码即文档即知识库**:文稿可执行,执行可回看结构,结构可导航与调试。
|
||||
4. **对齐 [OKF](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md)(Open Knowledge Format)方向**:清单由工具从 `.mq.md` **自动生成**,不是手填配置——见下节与 [catalog-cli.md](doc/design/catalog-cli.md)。
|
||||
5. **正经解释器(Rust)**:词法/行分类 → 语法 → 语义 → 树遍历(及字节码);**不用 Flex/Bison**。见 [路线图](doc/roadmap/interpreter.md)。
|
||||
5. **正经解释器(Rust)**:词法/行分类 → 语法 → 语义 → 树遍历(及字节码);**不用 Flex/Bison**。见 [路线图](doc/roadmap/interpreter.md)。
|
||||
6. **语言核心保持小**:Language(标记映射 / AST / 语义 / 类型)、Runtime(stdlib / 插件 ABI / WASM / agent)、Document(知识 + 代码 + UI + AI)三层分明——生态功能不淹没语言身份。
|
||||
|
||||
---
|
||||
|
||||
@@ -98,8 +142,10 @@ marqdo catalog public -o .marqdo
|
||||
|
||||
---
|
||||
|
||||
## 现状(v1.0.3)
|
||||
## 现状(v1.1.1)
|
||||
|
||||
- **AI 栈(v1.1.1)**:`ext/ai/llm` 语义 `ask`/`stream`/`collect` · Artifact Metadata Binding(`${env|arg|sys|secret}`)· Agent Skill Compilation + Adaptive Routing · MCP stdio 客户端 + resume 检查点 · 可选 `plugins/llm` Transport ABI
|
||||
- **AI 原生闭环(v1.1.0)**:核心表面守卫(19 构造,单边改动 CI 必红)· **渐进式契约**(文档内嵌 `参数`/`返回`/`字段` 表 + 四边界校验 + `marqdo check` 防漂移)· **MLSP for AI**(`marqdo mlsp`:`locate`/`syntax`/`validate`/`repair_targets`/`repair_apply`/`schema`,AI 按需查语法而非背诵)· **结构化诊断**(`run --json`,错误即数据);验证见 [perf-validation-report-2026-09-23.md](doc/roadmap/perf-validation-report-2026-09-23.md)
|
||||
- 映射与解释器:**v0.3 语法宪法**(`**` 代码 / `*` 返回 / 叙述声明);Phase I 树遍历 + 字节码后端;金样例在 `tests/` 与 `tests/markup-v03/`
|
||||
- **对象**:`#` = 类型/构造,`##`+ = 函数/方法;见 [objects.md](doc/design/objects.md)
|
||||
- **`marqdo view`**:文档浏览器(Structure + 函数大纲/搜索 + Execution + Variables 浮窗)
|
||||
@@ -111,16 +157,16 @@ marqdo catalog public -o .marqdo
|
||||
- **`web`**:W0–W7 + P3 + **W8**;**定制 C0–C4**;**宿主 H1–H2**(`proxy`/`invoke`,见 [ext-hosting.md](doc/roadmap/ext-hosting.md));**表驱动引言**(`compose_intro` / `引言装配`);原生层默认 **Go `libweb`**([ADR 0004](doc/adr/0004-web-plugin-go.md),`scripts/build-web-plugin.sh`);示例 [web-site](examples/web-site/) · [web-site-zh](examples/web-site-zh/) · [marqdo-blog](examples/marqdo-blog/) · [anlian-mq](examples/anlian-mq/);生产路径见 [web-asgi-servers-and-marqdo.md](doc/design/web-asgi-servers-and-marqdo.md)
|
||||
- **`quantum`**:Q0–Q7 + Q8a/b 主题 SVG;示例 [quantum-entanglement](examples/quantum-entanglement/)
|
||||
- **`linalg`**:L0–L6 + 公式文档面(中缀 / `declare` / 展示化简);示例 [linalg-transpose](examples/linalg-transpose/) · [linalg-svd](examples/linalg-svd/) · [linalg-least-squares](examples/linalg-least-squares/)
|
||||
- **`agent`**:A1–A4 + **MCP Server stdio**(`mcp_server`);宿主缺口见 [ext-hosting.md](doc/roadmap/ext-hosting.md) · [agent-framework-gaps-after-a4.md](doc/research/agent-framework-gaps-after-a4.md)
|
||||
- **`llm`**:OpenAI 兼容对话
|
||||
- **`agent`**:A1–A4 + **MCP Server/Client stdio** + Skill Compilation + Adaptive Routing + resume;宿主缺口见 [ext-hosting.md](doc/roadmap/ext-hosting.md) · [agent-framework-gaps-after-a4.md](doc/research/agent-framework-gaps-after-a4.md)
|
||||
- **`llm`**:语义 `ask`/`stream`/`collect`(OpenAI/Ollama;可选原生 `plugins/llm`)
|
||||
- 安装:`marqdo ext list` / `add …` / `remove`([ext-cli.md](doc/design/ext-cli.md);默认 `~/.marqdo/ext`)。原生插件先 `cargo build -p marqdo_plugin_*` 再 `ext add`
|
||||
- **原生插件 ABI**:[`include/marqdo_abi.h`](include/marqdo_abi.h) · [ext-abi.md](doc/design/ext-abi.md);`plugins/{demo,agent,web,quantum,linalg}`(`web` 为 Go;其余为 Rust)
|
||||
- **原生插件 ABI**:[`include/marqdo_abi.h`](include/marqdo_abi.h) · [ext-abi.md](doc/design/ext-abi.md);`plugins/{demo,agent,web,quantum,linalg,llm}`(`web` 为 Go;其余为 Rust)
|
||||
- **用户静态站**:`public/` → `view output` → CI 发布 [gh-pages](https://cflmy.github.io/marqdo/)
|
||||
- **VS Code 扩展**:分支 **`vscode-extension`**(`main` 不跟踪源码;见 [vscode-extension-commit.md](doc/design/vscode-extension-commit.md));Release 附带 `.vsix`
|
||||
- **浏览器 Marqdo(WASM)**:`marqdo wasm build` + 官方 bridge 自启(作者零业务 JS;桥内可含列表/路由/storage/ws/文件/Canvas/音频/Observer/拖放);`lib/browser` + GFM;`web.client_embed`;示例 [browser-hello](examples/browser-hello/) · [browser-app](examples/browser-app/) · [browser-media](examples/browser-media/) · [web-client-site](examples/web-client-site/)([ADR 0002](doc/adr/0002-browser-marqdo-wasm.md) · [D](doc/roadmap/browser-wasm-d.md) · [E](doc/roadmap/browser-wasm-e.md) · [F](doc/roadmap/browser-wasm-f.md))
|
||||
- 选型:[ADR 0001 — Rust](doc/adr/0001-implementation-language.md) · [ADR 0002 — 浏览器 WASM](doc/adr/0002-browser-marqdo-wasm.md)(C0–C5 完结,见 [roadmap/browser-wasm.md](doc/roadmap/browser-wasm.md))· [ADR 0003 — 异步效应](doc/adr/0003-browser-async-effects.md)
|
||||
|
||||
### 如何使用最新 Marqdo(v1.0.3)
|
||||
### 如何使用最新 Marqdo(v1.1.1)
|
||||
|
||||
**安装解释器(任选其一)**
|
||||
|
||||
@@ -135,7 +181,7 @@ marqdo catalog public -o .marqdo
|
||||
```bash
|
||||
# 1) 源码安装(跟 tag 或 main)
|
||||
git clone https://github.com/cflmy/marqdo.git && cd marqdo
|
||||
git checkout v1.0.3 # 或留在 main
|
||||
git checkout v1.1.1 # 或留在 main
|
||||
cargo build --release
|
||||
export PATH="$PWD/target/release:$PATH"
|
||||
marqdo version
|
||||
@@ -148,13 +194,18 @@ marqdo view public --no-open
|
||||
marqdo debug public --no-open
|
||||
marqdo catalog public -o .marqdo
|
||||
|
||||
# 2b) AI 原生闭环(v1.1.0+):契约检查 / 结构化诊断 / MLSP 查询
|
||||
marqdo check tests/contracts/ok_contract.mq.md # 静态契约防漂移
|
||||
marqdo run tests/structure/hello.mq.md --json # 结构化诊断(错误即数据)
|
||||
printf '{"id":1,"method":"syntax","params":{"query":"返回"}}\n' | marqdo mlsp
|
||||
|
||||
# 3) 安装官方扩展(Release / CDN 用户:无需本机 Rust/Go)
|
||||
marqdo ext add web # 或:网页 — 自动下载 L1 + 预编译 native(Go libweb)
|
||||
marqdo ext add agent # 或:智能体
|
||||
marqdo ext add quantum # 或:量子
|
||||
marqdo ext add linalg # 或:线性代数
|
||||
marqdo ext add llm # 或:大模型(纯 .mq.md)
|
||||
# 开发者:cargo build --release -p marqdo_plugin_{agent,quantum,linalg}
|
||||
marqdo ext add llm # 或:大模型(.mq.md + 可选 native llm)
|
||||
# 开发者:cargo build --release -p marqdo_plugin_{agent,quantum,linalg,llm}
|
||||
# bash ./scripts/build-web-plugin.sh # 需 Go + cgo → libweb
|
||||
# 离线:解压 Windows zip(已含 ext/native)或手动放下 native zip;MARQDO_EXT_NO_DOWNLOAD=1
|
||||
|
||||
@@ -169,7 +220,7 @@ marqdo wasm build
|
||||
# → dist/wasm/ … 见 examples/browser-hello/
|
||||
|
||||
# 6) Releases:Windows exe/zip/vsix;Linux CLI zip + native `.so` zip
|
||||
# https://github.com/cflmy/marqdo/releases/tag/v1.0.3
|
||||
# https://github.com/cflmy/marqdo/releases/tag/v1.1.1
|
||||
```
|
||||
|
||||
开发期也可用 `cargo run -- …` 代替已安装的 `marqdo`:
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "marqdo-wasm"
|
||||
version = "1.0.3"
|
||||
version = "1.1.1"
|
||||
edition = "2021"
|
||||
license = "Apache-2.0"
|
||||
description = "Marqdo interpreter as WebAssembly (browser host bridge)"
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"name": "marqdo-bridge",
|
||||
"private": true,
|
||||
"type": "module"
|
||||
}
|
||||
Vendored
+27
@@ -1,3 +1,30 @@
|
||||
marqdo (1.1.1-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 1.1.1: AI stack closure — llm ask/stream/collect, Metadata Binding,
|
||||
Skill Compilation + Adaptive Routing, MCP stdio client + resume checkpoints,
|
||||
optional plugins/llm Transport ABI (fallback lib/net).
|
||||
|
||||
-- cflmy <pingan@cflmy.cn> Thu, 24 Sep 2026 20:32:00 +0800
|
||||
|
||||
marqdo (1.1.0-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 1.1.0: core surface guard + progressive contracts + MLSP for AI.
|
||||
New `marqdo check` (static contract drift) and `marqdo mlsp` (language
|
||||
server for AI: locate/syntax/validate/repair_targets/schema/repair_apply);
|
||||
`run --json` unified structured diagnostics; contract tables
|
||||
(参数/返回/字段/集合) with four validation boundaries; anchored repair
|
||||
guardrails (bounded line edits, out-of-scope refused).
|
||||
|
||||
-- cflmy <pingan@cflmy.cn> Wed, 23 Sep 2026 13:30:00 +0800
|
||||
|
||||
marqdo (1.0.7-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 1.0.7: WS/net boundary hardening (same-origin handshake,
|
||||
require_auth/需要登录, message caps, empty-room GC); HTTP 8 MiB cap
|
||||
enforced while reading; url_encode space -> %20.
|
||||
|
||||
-- cflmy <pingan@cflmy.cn> Tue, 22 Sep 2026 15:40:00 +0800
|
||||
|
||||
marqdo (1.0.3-1ppa2) noble; urgency=medium
|
||||
|
||||
* Fix Launchpad build: dh_clean -Xvendor/ so cargo vendor checksums keep
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
# ADR 0006:Artifact Metadata Binding(`${…}` 进核心 · Metadata-only)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Status | **Accepted** |
|
||||
| Date | 2026-09-24 |
|
||||
| Related | [binding.md](../design/binding.md) · [core-surface.md](../design/core-surface.md) · [layers.md](../design/layers.md) · [doc/next/006.md](../next/006.md) |
|
||||
|
||||
## Context
|
||||
|
||||
- Executable Document 需要声明「依赖哪些外部运行时资源」(模型名、API 密钥、base URL),而不是在 `ext/*` 里各自 `sys.env_get`。
|
||||
- [doc/next/006.md](../next/006.md) 提出 **Marqdo Binding**(`${namespace.name}`)与 Artifact Metadata 统一解析。
|
||||
- [layers.md](../design/layers.md):Language 层变更须 ADR;核心构造计数只许持平或下降;默认进 ext。
|
||||
- 正文插值会与 `$$…$$` 公式面冲突,且会新增内联构造压力。
|
||||
|
||||
## Decision
|
||||
|
||||
1. 采纳 **Artifact Metadata Binding** 为 Language 层能力;正式规范见 [binding.md](../design/binding.md)。
|
||||
2. **`${…}` 仅**在文件头 `---` Metadata 中生效;正文 / 粗体 / 调用 / 表 / 叙述 **永不**写 Binding Expression(否决 [006 §17](../next/006.md))。
|
||||
3. **不**把 Binding 记入 [core-surface.md](../design/core-surface.md) 的 19 构造守卫表——与 `import` 同属 §4「其它语言面」。
|
||||
4. 核心只做声明与解析;扩展(如 `ext/ai/llm`)消费已绑定的 `Module.metadata` / `sys.meta*` / 入口提升变量。
|
||||
5. 第一版 namespace:`env` · `arg` · `sys` · `secret`。`config` / `file` 推迟。
|
||||
6. 加载期一次 resolve → Bound Artifact;`Value::Secret` 遮掩展示;`sys.env_get` 保留为逃逸口。
|
||||
7. **Phase 2**:入口执行前把 metadata 键提升为普通变量(形参/赋值可覆盖);正文用 `` `model` ``,不再 `${env…}`。
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- 配置成为文档的一部分;LLM / agent / 其它 ext 共用同一套 Binding,避免各写一套 getenv。
|
||||
- 缺失 `!` 在加载期失败,而不是运行到 HTTP 401。
|
||||
- 不膨胀 CORE_CONSTRUCTS;与公式 `$$` 无冲突。
|
||||
- 正文保持普通变量语义,不引入第二套环境变量语法。
|
||||
|
||||
### Negative / Risks
|
||||
|
||||
- Secret 在 `Text + Secret` 拼接时会揭示明文(供 Authorization);须约束日志面。
|
||||
- metadata 键与业务变量同名时依赖「形参/赋值覆盖」规则,作者需注意命名。
|
||||
|
||||
### Neutral
|
||||
|
||||
- `#` / `##` / `**` / `*` 等核心标记不变;frontmatter `import` 句法不变。
|
||||
@@ -0,0 +1,186 @@
|
||||
# Marqdo Binding · Artifact Metadata
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Accepted · Phase 1 + Phase 2(scope lift)** |
|
||||
| 日期 | 2026-09-24 |
|
||||
| ADR | [0006-artifact-metadata-binding.md](../adr/0006-artifact-metadata-binding.md) |
|
||||
| 讨论稿 | [doc/next/006.md](../next/006.md) |
|
||||
| 实现 | `src/binding/` · `Module.metadata` · entry env lift · `sys.meta` / `sys.meta_get` · `marqdo run --bind` |
|
||||
|
||||
> **边界**:核心负责「声明与解析配置」;扩展负责「解释配置的语义」。
|
||||
> **`${…}` 只出现在文件头 Artifact Metadata**——正文 / 粗体 / 调用 / 表 **永不**写 `${env…}`(否决讨论稿 §17)。正文使用已绑定的普通名字。
|
||||
|
||||
---
|
||||
|
||||
## 1. Artifact Metadata
|
||||
|
||||
文件开头成对 `---` 之间为 **Artifact Metadata**(声明式元信息环境),不是「随便贴的 YAML」。
|
||||
|
||||
```text
|
||||
.mq.md
|
||||
├── Metadata ← Identity / Configuration / Dependencies / Bindings
|
||||
└── Executable Body
|
||||
```
|
||||
|
||||
- `import bind:…` / `导入` 行:模块依赖(既有语义不变)。
|
||||
- 其它 `key: value` 行:元信息;值可为字面量或 **Binding Expression**。
|
||||
- Phase 1 仅支持 **flat** 键(同级 `key: value`);块标量 `>` / `|` 收成 Text。
|
||||
|
||||
例:
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: demo
|
||||
type: llm
|
||||
model: ${env.OPENAI_MODEL ?? "gpt-4o-mini"}
|
||||
api_key: ${secret.OPENAI_API_KEY!}
|
||||
base_url: ${env.OPENAI_BASE_URL ?? "https://api.openai.com/v1"}
|
||||
import sys:lib/sys.mq.md
|
||||
---
|
||||
```
|
||||
|
||||
加载后内部为已绑定的 `Module.metadata`(确定性快照),执行期不再对同一键反复 `getenv`。
|
||||
|
||||
---
|
||||
|
||||
## 2. Binding Expression
|
||||
|
||||
外形:
|
||||
|
||||
```text
|
||||
${expression}
|
||||
```
|
||||
|
||||
### 2.1 路径
|
||||
|
||||
```text
|
||||
${namespace.name}
|
||||
```
|
||||
|
||||
| Namespace | 意义 | Phase 1 |
|
||||
|-----------|------|---------|
|
||||
| `env` | 进程环境变量 | ✅ |
|
||||
| `arg` | CLI / Runtime 绑定(`marqdo run --bind KEY=VALUE`) | ✅ |
|
||||
| `sys` | 系统信息:`cwd` · `platform` | ✅ |
|
||||
| `secret` | 敏感环境变量 → `Value::Secret` | ✅ |
|
||||
| `config` | 配置 Artifact | 推迟 |
|
||||
| `file` | 外部文件内容 | 推迟 |
|
||||
|
||||
### 2.2 默认与必需
|
||||
|
||||
```text
|
||||
${env.OPENAI_MODEL ?? "gpt-4o-mini"}
|
||||
${secret.OPENAI_API_KEY!}
|
||||
```
|
||||
|
||||
- `A ?? B`:A 缺失(未设置 / 空)则用 B。
|
||||
- `A!`:A 必须存在,否则加载失败(`config.required_binding_missing`)。
|
||||
|
||||
### 2.3 类型转换
|
||||
|
||||
```text
|
||||
${int(env.MAX_TOKENS ?? "4096")}
|
||||
${float(env.TEMPERATURE ?? "0.2")}
|
||||
${bool(env.STREAM ?? "false")}
|
||||
```
|
||||
|
||||
### 2.4 独占 value vs 插值
|
||||
|
||||
| 形态 | 语义 |
|
||||
|------|------|
|
||||
| `model: ${env.OPENAI_MODEL}` | **Value binding** — 保留解析后的类型(含 Secret) |
|
||||
| `title: "Hello ${env.USER}"` | **Interpolation** — 结果为 Text |
|
||||
| `model: gpt-4o-mini` | **Literal** — 字面量;**不会**被环境变量静默覆盖 |
|
||||
|
||||
---
|
||||
|
||||
## 3. Resolution Policy
|
||||
|
||||
对每个 metadata 键:
|
||||
|
||||
1. 解析该键的字面量 / Binding / 插值(字面量定稿后不受 env 覆盖)。
|
||||
2. 若 CLI 提供了同名 `--bind KEY=VALUE`,**覆盖**该键的最终值(Text)。
|
||||
3. Binding 内部查找顺序(仅当表达式写出时):`arg` / `env` / `secret` / `sys` 按表达式 namespace;`??` 链从左到右。
|
||||
|
||||
推荐作者意图:
|
||||
|
||||
```text
|
||||
CLI --bind
|
||||
→ Artifact 字面量 / 已写死的 Binding 结果
|
||||
→ Binding 源(env / secret / …)
|
||||
→ ?? 默认
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Secret
|
||||
|
||||
`${secret.NAME}` → `Value::Secret`。
|
||||
|
||||
- `print` / `str` / Display → `<secret>`(不泄露)。
|
||||
- `type` → `secret`。
|
||||
- 与 Text 拼接(如 `bearer + api_key`)时**揭示**明文,供 HTTP Authorization。
|
||||
- 不得进入普通 episode / 错误信息明文(扩展作者责任;核心 Display 已遮掩)。
|
||||
|
||||
---
|
||||
|
||||
## 5. Runtime API
|
||||
|
||||
| API | 语义 |
|
||||
|-----|------|
|
||||
| `sys.meta` / `系统.元信息` | 入口模块已绑定 metadata(map) |
|
||||
| `sys.meta_get` / `系统.取元信息` | `key=` → 值或 `None` |
|
||||
| `marqdo run FILE --bind KEY=VALUE` | 可重复;写入 `arg` 并覆盖同名 metadata 键 |
|
||||
| `sys.env_get` | **保留**逃逸口;新代码优先 Metadata Binding |
|
||||
|
||||
---
|
||||
|
||||
## 6. 错误
|
||||
|
||||
| code | 何时 |
|
||||
|------|------|
|
||||
| `config.required_binding_missing` | `${ns.name!}` 且源缺失 |
|
||||
| `config.unknown_binding_ns` | 未知 namespace |
|
||||
| `config.bad_binding` | 表达式非法 / cast 失败 |
|
||||
|
||||
诊断尽量带 frontmatter 行号。
|
||||
|
||||
---
|
||||
|
||||
## 7. 与扩展的分工
|
||||
|
||||
- **核心**:解析 Metadata、resolve Binding、暴露 `Module.metadata` / `sys.meta*`,并在入口把 metadata **提升为普通变量**。
|
||||
- **`ext/ai/llm`**:构造次序为 **显式实参 → 入口作用域同名变量 / `sys.meta_get` → `sys.env_get` 回退 → 默认**。
|
||||
|
||||
---
|
||||
|
||||
## 8. Phase 2 · Metadata scope lift(Accepted)
|
||||
|
||||
加载期绑定完成后,入口执行(`# main` 或唯一无参顶层对象)前,把 `Module.metadata` 的每个键注入 entry 环境,成为**普通变量**:
|
||||
|
||||
```yaml
|
||||
---
|
||||
model: ${env.ModelName ?? "gpt-4o-mini"}
|
||||
---
|
||||
```
|
||||
|
||||
```marqdo
|
||||
# main
|
||||
|
||||
> print text=`model`
|
||||
```
|
||||
|
||||
### 规则
|
||||
|
||||
1. 注入时机:进入入口函数之前。
|
||||
2. 冲突:形参 / 调用实参 **覆盖** 同名 metadata;程序内赋值亦然。metadata **不**覆盖已存在的绑定。
|
||||
3. `sys.meta` / `sys.meta_get` 仍可用(显式通道)。
|
||||
4. **拒绝** 正文 / 粗体 / 表 / 叙述中的 `${…}`(讨论稿 006 §17 否决)。普通语言也不在语句里 `${ENV}`。
|
||||
|
||||
### 仍推迟(P3+)
|
||||
|
||||
- `config` / `file` namespace
|
||||
- 嵌套 YAML map
|
||||
- 显式 `override:` 糖
|
||||
- 计入 `CORE_CONSTRUCTS`
|
||||
@@ -0,0 +1,47 @@
|
||||
# 核心表面清单(core-surface)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Accepted · 核心语法封闭集合(CI 守卫锁定)** |
|
||||
| 日期 | 2026-09-22 |
|
||||
| 守卫 | `tests/core_surface.rs` ⇄ `src/parse/mod.rs` `CORE_CONSTRUCTS`(改动任一侧不同步 → CI 红) |
|
||||
| 相关 | [markdown-mapping-v0.3.md](markdown-mapping-v0.3.md)(语义宪法)· [layers.md](layers.md)(准入戒律)· [three-problems.md](../roadmap/three-problems.md) 问题一 · [three-problems-plan.md](../roadmap/three-problems-plan.md) T1.1/T1.3 |
|
||||
|
||||
> **戒律总纲**:核心标记是**封闭集合**(约 19 个构造)。新增任何一行必须走 ADR(见 [layers.md](layers.md) 准入戒律)并回答「为什么现有标记不够」;生态需求的默认答案是**进 ext**。本表 id 列与守卫测试强一致:改表不同步 `CORE_CONSTRUCTS`(或反之)、样例文件缺失,一律 CI 红。
|
||||
|
||||
## 1. 行构造
|
||||
|
||||
| id | 外形 | 语义 | 戒律 | 样例 |
|
||||
|----|------|------|------|------|
|
||||
| `narrative` | 无标记文字 | 叙述(注释) | 段落内延续到空行;**默认安全**——永远不存在「叙述被执行」的路径 | `tests/markup-v03/soft-emphasis.mq.md` |
|
||||
| `blank` | 空行 | 段落边界 | 叙述分段的唯一手段;不得被赋予其它语义 | `tests/markup-v03/inc-prose.mq.md` |
|
||||
| `heading-unit` | `#` / `##` | 对象 / 函数单元 | 标题即单元边界,按深度嵌套;正文禁止用 `##` 做版式(会变成函数) | `tests/markup-v03/decl-ident-only.mq.md` |
|
||||
| `call-line` | `>` | 调用行 | 输出等副作用 = **普通函数**,不是架构标记 | `tests/markup-v03/dual-call.mq.md` |
|
||||
| `branch-list` | 体内 `1.` 递增 | 分支 | 自 `1.` 起序号递增构成一个判断;重出 `1.` 开新判断;`N. *` = else;**不是所有有序列表都是 if** | `tests/markup-v03/greet-default.mq.md` |
|
||||
| `loop-list` | `-` | 循环 | 条件形 `` - `条件` `` 或遍历形 `- [项](集合)` | `tests/structure/loop.mq.md` |
|
||||
| `param` | `` + `名` `` | 形参(兼容形) | 首选叙述 `` `名` `` 升参(宪法 §8);`+` 形仅兼容保留 | `tests/markup-v03/greet-default.mq.md` |
|
||||
| `table` | GFM 表 | 集合 | 几何即类型([tables-maps-footnotes.md](../roadmap/tables-maps-footnotes.md));**绑定 = 数据,未绑定 = 文档表** | `tests/structure/collection.mq.md` |
|
||||
| `fence` | ` ```lang ` | 外联 | 语言块原样外联,不进 Marqdo 语义 | `tests/lib/foreign-python.mq.md` |
|
||||
| `hr` | 单行 `---` 或 `***` | 叙述分隔 | **不收束函数**(返回才收束) | `tests/markup-v03/module-hr.mq.md` |
|
||||
| `writeback` | `<!-- marqdo-out … -->` | 持久化输出 | 只读回放,不执行 | `tests/lib/writeback-get.mq.md` |
|
||||
|
||||
## 2. 内联标记
|
||||
|
||||
| id | 外形 | 语义 | 戒律 | 样例 |
|
||||
|----|------|------|------|------|
|
||||
| `ident` | `` `名` `` | 声明 / 引用 | 首次未赋值 = 声明;叙述与程序共用同一视觉习惯 | `tests/markup-v03/decl-ident-only.mq.md` |
|
||||
| `bold-code` | `**…**` | 代码(赋值 / 调用 / 表达式) | 只有粗体进入执行;判别序:赋值 → 调用 → 表达式 | `tests/markup-v03/multi-bold.mq.md` |
|
||||
| `italic-return` | `*…*` | 返回 | 改变控制流(**返回是架构**);返回即结束函数体 | `tests/markup-v03/inc-prose.mq.md` |
|
||||
| `empty-return` | `*None*` / `****` | 空返回 | 返回 `None` 并结束函数体 | `tests/markup-v03/empty-return-none.mq.md` |
|
||||
| `link-index` | `[键](集合)` | 取元 / foreach | **主取元语法**;与 bracket-call 靠 `]` 后是否紧邻 `(` 消歧 | `tests/lib/table-collections.mq.md` |
|
||||
| `bracket-call` | `修饰… [函数] 实参…` | 方括号标记调用 | 前置修饰 → `名=True`;与 link-index 消歧同上 | `tests/structure/bracket-call.mq.md` |
|
||||
| `footnote-index` | `集合[^键]` | 过渡取元 | **仅兼容**;新代码一律写 link-index | `tests/structure/footnote-index.mq.md` |
|
||||
| `soft-emphasis` | `*仅*` / `**说明**` | 忽略 | 非代码形强调**不执行**——默认安全的关键消歧规则 | `tests/markup-v03/soft-emphasis.mq.md` |
|
||||
|
||||
## 3. 词法大类(LineKind,`src/lex/mod.rs`)
|
||||
|
||||
行级入口只有四个大类:`Blank` / `Comment`(叙述段落,含软强调与行内 `` `名` ``)/ `Writeback` / `Code`(`# * > + - | ~ $` 或数字开头)。本清单第 1 节的行构造是在 `Code`(及 `Comment`/`Writeback`)内部的**语义细分**;第 2 节内联标记由 `src/parse/prose.rs` 行内扫描产出。两层合起来 = 核心表面全集。
|
||||
|
||||
## 4. 范围外(进 Language 层,但不在本守卫)
|
||||
|
||||
frontmatter / Artifact Metadata(元信息)、`import` 行(模块装载)、以及 Metadata 内的 **Binding Expression**(`${ns.name}`,见 [binding.md](binding.md) · [ADR 0006](../adr/0006-artifact-metadata-binding.md))属于 Language 层的其它语言面,随 [layers.md](layers.md) 准入流程登记,**暂不进本清单守卫**(不增 CORE_CONSTRUCTS 计数)。Phase 2 将已绑定 metadata **提升为入口普通变量**(仍非新标记)。如需一并守卫,走 ADR 扩表。
|
||||
@@ -180,11 +180,26 @@ ext/
|
||||
| 中文 | 英文 | 角色 |
|
||||
|------|------|------|
|
||||
| `# 智能体` | `# agent` | 构造:`大模型`/`model`,`工具`/`tools`,可选 `站立提示`/`standing` |
|
||||
| `## 运行` | `## run` | **v2 主入口**:Adaptive `route` → execute → record episode → maybe_learn(技能 + 策略) |
|
||||
| `## 单步` | `## step` | 原子执行;注入源码 + 位置;可选工具;返回结构化 map;**默认自动写回**(`写回=` / `writeback=` 可关) |
|
||||
| `## 多步` | `## plan` | 建/续工作簿 → subtask 跑文件 → 修订循环 → 汇总(见 [ext-agent-plan.md](ext-agent-plan.md)) |
|
||||
| `## 解析` | `## resolve` | 技能解析:exact → alias → canonical → near |
|
||||
| `## 路由` | `## route` | **P4 Adaptive Routing**:`marqdo` \| `small-llm` \| `llm` \| `auto`(`jev` 为 `small-llm` 兼容别名) |
|
||||
| `## 应用路由` | `## route_apply` | 将小模型/大模型路由回复(`SKILL:<slug>` / `EXPLORE`)落到 kb 技能 |
|
||||
| `## 编译策略` | `## compile_policy` | Policy Compilation → `policies/router.mq.md` |
|
||||
| `## 记录情景` | `## record_episode` | 每次执行沉淀为 `.mq.md` episode(`.marqdo/agent-memory/`) |
|
||||
| `## 尝试学习` | `## maybe_learn` | 情景足够时归纳 → 合成 → 编译技能,并尝试编译策略 |
|
||||
| `## 编译` | `## compile` | 候选技能 → 版本化 `skills/<slug>/vN.mq.md` + 可选 OKF promote |
|
||||
| `## 指标` | `## metrics` | Reasoning amortization(`avg_llm_calls` / `compiled_runs`) |
|
||||
| `## 分解` | `## decompose` |(可公开或内部)目标 → 子任务表 |
|
||||
| `## 清空历史` | `## clear_history` | 若仍保留会话式辅助状态则清空;**不以隐藏 chat 袋为真相源**——真相在文档写回 |
|
||||
|
||||
**Skill Compilation(v2)**:Agent 的记忆不是向量袋,而是可执行 `.mq.md`。路径:`episodes/` → `experiences/` → `candidates/` → `skills/vN.mq.md` → OKF `agent-kb`(`llm_free`)。编译技能正文为合法 Marqdo(`**result = …**` + 斜体返回);金样 `tests/ext/agent-zero-llm.mq.md` 验收 policy+skill 全路径 `execution.llm_calls=0`。`plan` 对已存在的 `resources/<slug>.mq.md` **不覆盖**(`agent-plan-preserve`)。
|
||||
|
||||
**LLM 成本**:`model_turn` 走 `model.ask` / `stream`+`collect`,把 `LLMResult.tokens` 写入 episode / `execution`。
|
||||
|
||||
**Adaptive Routing(P4)**:Router 是抽象能力,backend = `marqdo`(已编译 `policies/router.mq.md`)/ `small-llm`(`router_model` 任意 llm 句柄)/ `llm` / `auto` 级联。`jev` 仅为 `small-llm` 兼容别名,**不进入框架专属设计**。热路径在 **`plugins/agent`**(`route.rs`);**禁止**向 `src/host/` 加领域原语。设计:[doc/next/003.md](../next/003.md) · [doc/next/004.md](../next/004.md)。
|
||||
|
||||
构造参数(v1):
|
||||
|
||||
| 参数 | 说明 |
|
||||
|
||||
+74
-51
@@ -2,13 +2,21 @@
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Status | Accepted (v0.1.x) |
|
||||
| Date | 2026-08-06 |
|
||||
| Related | [objects.md](objects.md) · [ext-abi.md](ext-abi.md) · [ext-agent.md](ext-agent.md) · [ext-cli.md](ext-cli.md) |
|
||||
| Status | **Accepted · v0.3 semantic**(005:Intelligence Primitive) |
|
||||
| Date | 2026-09-24 |
|
||||
| Related | [objects.md](objects.md) · [ext-agent.md](ext-agent.md) · [ext-cli.md](ext-cli.md) · [ai-stack.md](../roadmap/ai-stack.md) · [doc/next/005.md](../next/005.md) |
|
||||
|
||||
## Scope
|
||||
|
||||
`ext/ai/` holds **official LLM + agent** extensions. Import paths:
|
||||
`ext/ai/llm` is Marqdo’s **intelligence primitive**, not an OpenAI SDK wrapper.
|
||||
|
||||
```text
|
||||
Author surface: ask / stream / collect / create / fast / reasoning / prompt_load
|
||||
Transport: ext/ai/llm/openai.mq.md (+ ollama defaults)
|
||||
Compat: complete / chat
|
||||
```
|
||||
|
||||
Import:
|
||||
|
||||
```markdown
|
||||
---
|
||||
@@ -16,70 +24,85 @@ import llm:ext/ai/llm.mq.md
|
||||
---
|
||||
```
|
||||
|
||||
Resolution: `MARQDO_EXT/ai/…`, repo `ext/ai/…`, cwd `ext/ai/…` (see [ext-cli.md](ext-cli.md)).
|
||||
|
||||
Other official ext: [ext-agent.md](ext-agent.md) / [ext-agent-plan.md](ext-agent-plan.md) (**document-driven agent** — step with default writeback / plan workbook; thin `TOOL:` loop superseded). Installer: [ext-cli.md](ext-cli.md). Native plugins: [ext-abi.md](ext-abi.md).
|
||||
|
||||
Streaming: `complete stream=True` returns an event list (`reasoning` / `delta` / `done` / `error`); `echo=True` prints thinking (`reasoning_content`) and answer tokens as they arrive. See [agent-streaming.md](../roadmap/agent-streaming.md). Default (`stream=False`) is unchanged (answer string).
|
||||
|
||||
**Note:** Prefer `marqdo ext add llm`. Import `import llm:ext/ai/llm.mq.md` from repo or install root.
|
||||
|
||||
## Platform prerequisites
|
||||
|
||||
| Capability | Surface |
|
||||
|------------|---------|
|
||||
| HTTPS + headers | `lib/net` → `http_post` / `headers=` map |
|
||||
| Dotenv | `## load_env` in ext (or `load_dotenv` / `lib/sys`) |
|
||||
| JSON quote | `lib/json` → `quote` |
|
||||
|
||||
## `ext/ai/llm` / `ext/ai/大模型` (object handles)
|
||||
## Public API
|
||||
|
||||
| English | Chinese | Role |
|
||||
|---------|---------|------|
|
||||
| `## load_env` | `## 加载环境` | Free function: load `.env` |
|
||||
| `## stream_result` | `## 流式结果` | Collapse event list → final text |
|
||||
| `# llm` | `# 大模型` | Object ctor → handle map |
|
||||
| `## complete` / `## chat` | `## 运行` / `## 聊天` | Methods; optional `stream` / `流式`, `echo` / `打印增量` |
|
||||
| `## load_env` | `## 加载环境` | Load `.env` |
|
||||
| `## create` | `## 创建` | Factory → LLM handle |
|
||||
| `## fast` / `## reasoning` | `## 快速` / `## 推理` | Named handles |
|
||||
| `## prompt_load` / `## prompt` / `## prompt_body` | `## 加载提示` / `## 提示` / `## 提示正文` | Prompt artifacts (`type: prompt` → body) |
|
||||
| `## ask` (module) | `## 提问` | Convenience → **text** (`prompt=` or `path=`) |
|
||||
| `## collect` | `## 收集` | Events → text |
|
||||
| `## stream_result` | `## 流式结果` | Alias of `collect` |
|
||||
| `# llm` | `# 大模型` | Handle ctor (`backend=` openai-compatible \| ollama) |
|
||||
| `## ask` (method) | `## 提问` | → **LLMResult** map |
|
||||
| `## stream` | `## 流式` | → event list |
|
||||
| `## complete` / `## chat` | `## 运行` / `## 聊天` | Compat / transport primitive |
|
||||
|
||||
### Preferred
|
||||
|
||||
```markdown
|
||||
---
|
||||
import llm:ext/ai/llm.mq.md
|
||||
---
|
||||
**answer = > llm.ask prompt="What is Marqdo?"**
|
||||
*answer*
|
||||
|
||||
# main
|
||||
**model = > llm.create**
|
||||
**result = > `model`.ask prompt=`prompt`**
|
||||
*[text](result)*
|
||||
|
||||
> load_env
|
||||
**fast = > llm.fast**
|
||||
**events = > `fast`.stream prompt=`prompt`**
|
||||
**text = > llm.collect events=`events`**
|
||||
|
||||
*`model` = > llm *
|
||||
*`reply` = > `model`.complete prompt=Say hi in one word *
|
||||
> print text=`reply`
|
||||
|
||||
*`events` = > `model`.complete prompt=Say hi stream=True *
|
||||
- [`ev`](`events`)
|
||||
*`t` = > json.get value=`ev` key=type *
|
||||
1. `t` == "delta"
|
||||
*`chunk` = > json.get value=`ev` key=text *
|
||||
> print text=`chunk`
|
||||
2. *
|
||||
*`_` = 1*
|
||||
**body = > llm.prompt path="fixtures/prompt-marqdo.md"**
|
||||
**body2 = > llm.prompt text=`raw`**
|
||||
```
|
||||
|
||||
Environment (create a **project-local** `.env`; do not commit secrets):
|
||||
### LLMResult
|
||||
|
||||
| Field | Meaning |
|
||||
|-------|---------|
|
||||
| `text` | Answer string |
|
||||
| `model` | Model id |
|
||||
| `finish` | Finish reason (`stop`, …) |
|
||||
| `backend` | e.g. `openai-compatible` / `ollama` |
|
||||
| `usage` | `{prompt_tokens,completion_tokens,total_tokens}` |
|
||||
| `tokens` | Alias of `usage.total_tokens` (agent metrics) |
|
||||
| `name` | Optional handle tag (`fast` / `reasoning`) |
|
||||
| `llm_calls` | `1` per ask |
|
||||
| `tool_calls` | List of OpenAI-shaped tool call objects (empty when none) |
|
||||
|
||||
### Compat
|
||||
|
||||
`complete` / `chat` still return a bare string (or event list when `stream=True`) so `ext/ai/agent` and older runbooks keep working. **New code should use `ask` / `stream` / `collect`.**
|
||||
|
||||
## Environment
|
||||
|
||||
```env
|
||||
OPENAI_API_KEY=sk-...
|
||||
OPENAI_BASE_URL=https://api.openai.com/v1
|
||||
OPENAI_MODEL=gpt-4o-mini
|
||||
MARQDO_LLM_FAST_MODEL=...
|
||||
MARQDO_LLM_REASONING_MODEL=...
|
||||
OLLAMA_HOST=http://127.0.0.1:11434/v1
|
||||
```
|
||||
|
||||
Fallbacks: `MARQDO_LLM_API_KEY`, `MARQDO_LLM_BASE_URL`, `MARQDO_LLM_MODEL`.
|
||||
Fallbacks: `MARQDO_LLM_API_KEY`, `MARQDO_LLM_BASE_URL`, `MARQDO_LLM_MODEL`. Ctor also accepts `api_key=` / `base_url=` / `model=` / `backend=`.
|
||||
|
||||
## Backends
|
||||
|
||||
| `backend=` | Module | Notes |
|
||||
|------------|--------|-------|
|
||||
| `openai-compatible` (default) | `ext/ai/llm/openai.mq.md` | Chat Completions HTTP + SSE |
|
||||
| `ollama` | defaults in ctor + same transport | Key defaults to `ollama`; host from `OLLAMA_HOST` |
|
||||
|
||||
## Tests
|
||||
|
||||
- Import smoke: `tests/ext/llm-import.mq.md`
|
||||
- Offline stream: `tests/ext/llm-stream-offline.mq.md` + `tests/lib/net-openai-sse.mq.md`
|
||||
- Live complete (local): `tests/ext/llm-complete.mq.md` + `tests/ext/.env`
|
||||
- Live agent `## step`: `tests/ext/agent-run-live.mq.md` (DeepSeek; gold: `ext_agent_run_live`)
|
||||
- Agent smoke (real `llm` handle): `tests/ext/agent-smoke.mq.md` (gold: `ext_agent_framework_smoke`)
|
||||
- Live complete: `tests/ext/llm-complete.mq.md` (gold: `ext_llm_complete_live`)
|
||||
|
||||
- `tests/ext/llm-import.mq.md`
|
||||
- `tests/ext/llm-ctor-offline.mq.md`
|
||||
- `tests/ext/llm-ask-offline.mq.md` (create + collect)
|
||||
- `tests/ext/llm-stream-offline.mq.md`
|
||||
- `tests/ext/llm-named-offline.mq.md` (fast/reasoning + prompt_load)
|
||||
- `tests/ext/llm-tool-calls-offline.mq.md` (`tool_calls_from` / Result field)
|
||||
- `tests/ext/llm-meta-offline.mq.md` · `llm-entry-prompt-offline.mq.md`
|
||||
- Live: `llm-complete.mq.md` · `llm-stream-live.mq.md`
|
||||
@@ -0,0 +1,29 @@
|
||||
# 三层归属与准入戒律(layers)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Accepted · 准入制度(问题一落地)** |
|
||||
| 日期 | 2026-09-22 |
|
||||
| 相关 | [core-surface.md](core-surface.md)(核心表面守卫)· [three-problems.md](../roadmap/three-problems.md) 问题一 · [three-problems-plan.md](../roadmap/three-problems-plan.md) T1.2 |
|
||||
|
||||
> **目的**:语言核心保持小——**Language / Runtime / Document 三层分明,生态功能不淹没语言身份**。本制度是机制不是宣言:违反即 CI 红 / PR 不评审。
|
||||
|
||||
## 1. 三层归属表
|
||||
|
||||
| 层 | 内容 | 仓库位置 |
|
||||
|----|------|----------|
|
||||
| **Language** | 标记映射、词法/行分类、AST、语义、升参、契约检查器、诊断 | `src/lex/` `src/parse/` `src/ast.rs` `src/interp.rs` `src/diagnostics.rs`(及 `contract`/`check`)· 宪法 [markdown-mapping-v0.3.md](markdown-mapping-v0.3.md) · [core-surface.md](core-surface.md) |
|
||||
| **Runtime** | stdlib、插件 ABI、WASM、debugger、view、catalog / schema / mcp CLI、字节码后端 | `lib/` `plugins/` `crates/` `src/catalog.rs` `src/view/` `src/debug/` `src/builtin.rs` `src/bytecode/` |
|
||||
| **Document** | 知识单元、AI skills、示例、官方扩展、用户站 | `*.mq.md` `skills/` `ext/` `public/` `examples/` `doc/` |
|
||||
|
||||
## 2. 准入戒律
|
||||
|
||||
1. **先填层归属**:任何新特性提案(issue / PR / 设计文档)必须先声明进哪层;未填 = 不评审。
|
||||
2. **Language 层变更 = ADR**:新增核心标记必须有 `doc/adr/` 编号 + 「为什么现有标记不够」+ [core-surface.md](core-surface.md) 增行 + 样例 + `CORE_CONSTRUCTS` 同步(缺任一 → CI 守卫红)。
|
||||
3. **核心计数只许持平或下降**:每个 release 的核心标记计数不得增长;确需增长即触发 Language 层复盘(ADR 里写明不可替代理由)。
|
||||
4. **默认进 ext**:能用扩展、stdlib、插件解决的,一律不进核心。
|
||||
5. **CHANGELOG 单列「核心表面 Δ」**:每个 release 写明增删(无 Δ 写「无」);发版 checklist(`.cursor/skills/marqdo-release/SKILL.md`)含此步。
|
||||
|
||||
## 3. PR 模板
|
||||
|
||||
`.github/PULL_REQUEST_TEMPLATE.md` 含「层归属」必填项与「核心表面」勾选——与本制度一一对应。
|
||||
@@ -616,3 +616,4 @@ FnDef.params:
|
||||
| 2026-09-13 | 决议空返回(§7.2)与提及性反引号(§4.4);形参改为「升参」与绑定图分离(§8.2) |
|
||||
| 2026-09-14 | §7.2:`---` 不再收束;返回语句结束函数体解析;空返回首选 `*None*` |
|
||||
| 2026-09-14 | 取元主语法改为 `[键](集合)`;脚注取元过渡保留 |
|
||||
| 2026-09-24 | Artifact Metadata Binding(`${…}` Phase 1,仅 frontmatter):见 [binding.md](binding.md) · ADR 0006 |
|
||||
@@ -130,7 +130,7 @@ Markdown **标记即语法**。输出**不是**语言架构原语,而是内置
|
||||
|
||||
## 5. Frontmatter 与跨文件
|
||||
|
||||
文件头 `---` 对为元信息。其中 `import bind:path.mq.md` / `导入` 为**文件导入**;`import bind:lib.member` 为**短名**(废除独立 `use` 与 frontmatter `>` 导入)。调用经裸点号路径(`time.parse`、`agent.agent`);实例方法在独立 `>` 调用须 `` > `var`.method ``(`*…*` 段内接收者裸名)。详见 [module-namespace.md](module-namespace.md)。
|
||||
文件头 `---` 对为 **Artifact Metadata**。其中 `import bind:path.mq.md` / `导入` 为**文件导入**;`import bind:lib.member` 为**短名**(废除独立 `use` 与 frontmatter `>` 导入)。其它 `key: value` 可为字面量或 **Binding**(`${env.NAME}` 等,加载期解析);见 [binding.md](binding.md)。调用经裸点号路径(`time.parse`、`agent.agent`);实例方法在独立 `>` 调用须 `` > `var`.method ``(`*…*` 段内接收者裸名)。详见 [module-namespace.md](module-namespace.md)。
|
||||
|
||||
**函数体内的** `---` / `***` 不解释为 frontmatter(见 §11)。
|
||||
|
||||
|
||||
+1373
File diff suppressed because it is too large.
Load diff
+933
@@ -0,0 +1,933 @@
|
||||
这是marqdo的智能体框架,我一直都不满意,我认为marqdo的智能体框架应该更好才对,因为marqdo本身就是一门极好的AI原生语言,针对这个智能体框架,我的构想是,每次执行,智能体都会沉淀mq.md文档,超过三次的调用就会自动总结优化出一份类似与记忆的mq.md文档,把每一步都拆解的非常好,类似与人类的某些学习过程,当完全学习到了一些内容,实际上后续就可以不再每次都聚精会神,而是用一种较低能耗运行,就像是我们沉淀下来的mq.md代码,其中可以使用非常少的大模型思考,或者使用类似Jev的决策模型,或者干脆只使用分支控制,之后调用即可,你觉得呢?
|
||||
|
||||
对,而且我现在觉得你这个构想**很可能比目前的 Agent 框架本身更接近 Marqdo 真正应该成为的东西**。
|
||||
|
||||
我仔细看了你上传的 `ext/ai/agent`。其实你已经无意中铺好了不少基础:现在的 `plan` 已经有 `reuse`、`near_match`、`soft_match`、`promote`、`improve_every=3` 和 `kb_dir=.marqdo/agent-kb`;命中已有 skill 后甚至可以走 `llm_free` 路径。 而且现有逻辑已经会把成功的 workbook promote 到 agent KB。
|
||||
|
||||
**所以我觉得你现在缺的不是“memory”,而是一个更深的概念:Learning → Compilation。**
|
||||
|
||||
---
|
||||
|
||||
# 1. 我会把你想做的东西定义成:Agent 的“认知编译”
|
||||
|
||||
你描述的人类学习过程非常准确:
|
||||
|
||||
```text
|
||||
第一次:
|
||||
仔细思考
|
||||
↓
|
||||
尝试
|
||||
↓
|
||||
犯错
|
||||
↓
|
||||
修正
|
||||
|
||||
第二次:
|
||||
仍然需要思考
|
||||
↓
|
||||
但已经有经验
|
||||
|
||||
第三次:
|
||||
开始形成套路
|
||||
|
||||
多次之后:
|
||||
“我知道怎么做了”
|
||||
↓
|
||||
不再重新思考
|
||||
↓
|
||||
直接执行
|
||||
```
|
||||
|
||||
传统 Agent:
|
||||
|
||||
```text
|
||||
User
|
||||
↓
|
||||
LLM
|
||||
↓
|
||||
Tool
|
||||
↓
|
||||
LLM
|
||||
↓
|
||||
Tool
|
||||
↓
|
||||
LLM
|
||||
```
|
||||
|
||||
每次任务都重新付出推理成本。
|
||||
|
||||
你想让 Marqdo 变成:
|
||||
|
||||
```text
|
||||
新任务
|
||||
│
|
||||
▼
|
||||
是否已经学会?
|
||||
/ \
|
||||
是 否
|
||||
│ │
|
||||
▼ ▼
|
||||
已编译 mq.md LLM 推理
|
||||
│ │
|
||||
▼ ▼
|
||||
低成本执行 执行 + 观察
|
||||
│
|
||||
▼
|
||||
沉淀 mq.md
|
||||
│
|
||||
▼
|
||||
验证 / 优化
|
||||
│
|
||||
▼
|
||||
编译成 Skill
|
||||
```
|
||||
|
||||
我认为这个想法**非常漂亮**。
|
||||
|
||||
---
|
||||
|
||||
# 2. 而且这和 Marqdo 的语言设计天然吻合
|
||||
|
||||
这可能是你整个项目最重要的一次“闭环”。
|
||||
|
||||
你前面定义:
|
||||
|
||||
> `.mq.md` = Markdown + 可执行程序 + 知识。
|
||||
|
||||
现在 Agent 又产生:
|
||||
|
||||
> `.mq.md` = Agent 学会的经验。
|
||||
|
||||
于是:
|
||||
|
||||
```text
|
||||
Agent
|
||||
│
|
||||
│ 学习
|
||||
▼
|
||||
.mq.md
|
||||
│
|
||||
┌───────┼────────┐
|
||||
↓ ↓ ↓
|
||||
知识 策略 程序
|
||||
│ │ │
|
||||
└───────┼────────┘
|
||||
▼
|
||||
Agent Skill
|
||||
│
|
||||
▼
|
||||
低成本执行
|
||||
```
|
||||
|
||||
这就非常有意思了。
|
||||
|
||||
**Agent 的 memory 不再是一段向量 embedding,也不只是聊天历史。**
|
||||
|
||||
它是:
|
||||
|
||||
> **可阅读、可修改、可验证、可执行的知识程序。**
|
||||
|
||||
这才是真正的 **Marqdo-native Agent**。
|
||||
|
||||
---
|
||||
|
||||
# 3. 你现在的 Agent 已经有这个雏形,但还没有真正“学习”
|
||||
|
||||
从你现在的代码看,已经存在:
|
||||
|
||||
```text
|
||||
reuse
|
||||
near_match
|
||||
soft_match
|
||||
promote
|
||||
improve_every = 3
|
||||
llm_free
|
||||
```
|
||||
|
||||
这些机制实际上已经在做:
|
||||
|
||||
> **经验复用。**
|
||||
|
||||
例如当前逻辑会查找已有 KB,如果命中 `llm_free` skill,就可以直接运行 resource,而不是再让父 Agent 做一轮 LLM 推理。
|
||||
|
||||
这是很好的基础。
|
||||
|
||||
但我认为:
|
||||
|
||||
> **Reuse ≠ Learning。**
|
||||
|
||||
现在更像:
|
||||
|
||||
```text
|
||||
成功任务
|
||||
↓
|
||||
保存
|
||||
↓
|
||||
以后查找
|
||||
↓
|
||||
复用
|
||||
```
|
||||
|
||||
而你现在构想的是:
|
||||
|
||||
```text
|
||||
成功任务
|
||||
↓
|
||||
观察多次执行
|
||||
↓
|
||||
抽象共同结构
|
||||
↓
|
||||
去掉偶然信息
|
||||
↓
|
||||
形成一般策略
|
||||
↓
|
||||
验证
|
||||
↓
|
||||
编译
|
||||
↓
|
||||
以后低成本执行
|
||||
```
|
||||
|
||||
这两者差距非常大。
|
||||
|
||||
---
|
||||
|
||||
# 4. 我甚至建议你不要叫它 Memory
|
||||
|
||||
我会给它一个更有 Marqdo 味道的名字:
|
||||
|
||||
## **Skill Compilation**
|
||||
|
||||
或者:
|
||||
|
||||
## **Experience Compilation**
|
||||
|
||||
甚至:
|
||||
|
||||
## **Cognitive Compilation**
|
||||
|
||||
整个过程:
|
||||
|
||||
```text
|
||||
Experience
|
||||
↓
|
||||
Episodic Trace
|
||||
↓
|
||||
Pattern Extraction
|
||||
↓
|
||||
Generalization
|
||||
↓
|
||||
Marqdo Skill
|
||||
↓
|
||||
Validation
|
||||
↓
|
||||
Compiled Skill
|
||||
```
|
||||
|
||||
然后 Agent 不再只是:
|
||||
|
||||
> “记住我以前做过什么。”
|
||||
|
||||
而是:
|
||||
|
||||
> **“把以前的思考编译成以后不需要思考的程序。”**
|
||||
|
||||
我觉得这是一个非常强的概念。
|
||||
|
||||
---
|
||||
|
||||
# 5. 例如真的可以这样工作
|
||||
|
||||
第一次:
|
||||
|
||||
用户:
|
||||
|
||||
> 把这个 CSV 中年龄大于 18 的用户筛出来,并按照年龄排序。
|
||||
|
||||
Agent:
|
||||
|
||||
```text
|
||||
LLM reasoning
|
||||
↓
|
||||
读取 CSV
|
||||
↓
|
||||
分析结构
|
||||
↓
|
||||
过滤
|
||||
↓
|
||||
排序
|
||||
↓
|
||||
输出
|
||||
```
|
||||
|
||||
产生:
|
||||
|
||||
```text
|
||||
.marqdo/agent-runs/xxx.mq.md
|
||||
```
|
||||
|
||||
第二次用户:
|
||||
|
||||
> 把另一个 CSV 中年龄大于 18 的用户筛出来,并按照年龄排序。
|
||||
|
||||
Agent 发现:
|
||||
|
||||
```text
|
||||
intent similarity = 0.94
|
||||
```
|
||||
|
||||
于是:
|
||||
|
||||
```text
|
||||
REUSE
|
||||
```
|
||||
|
||||
直接调用之前的程序。
|
||||
|
||||
---
|
||||
|
||||
第三、第四、第五次:
|
||||
|
||||
Agent 发现:
|
||||
|
||||
```text
|
||||
same structure
|
||||
different file
|
||||
different column names
|
||||
```
|
||||
|
||||
这时候不应该继续保存五个:
|
||||
|
||||
```text
|
||||
task1.mq.md
|
||||
task2.mq.md
|
||||
task3.mq.md
|
||||
task4.mq.md
|
||||
task5.mq.md
|
||||
```
|
||||
|
||||
而应该让 LLM 总结:
|
||||
|
||||
```markdown
|
||||
# Filter and Sort CSV Users
|
||||
|
||||
## Preconditions
|
||||
|
||||
- Input contains user records.
|
||||
- An age field exists.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Read the input table.
|
||||
2. Keep rows where `age > 18`.
|
||||
3. Sort by `age` ascending.
|
||||
|
||||
## Variables
|
||||
|
||||
- `input`
|
||||
- `age_field`
|
||||
- `output`
|
||||
|
||||
## Exceptions
|
||||
|
||||
- Missing age field
|
||||
- Invalid age
|
||||
|
||||
## Examples
|
||||
|
||||
| Input | Output |
|
||||
|---|---|
|
||||
| ... | ... |
|
||||
```
|
||||
|
||||
然后:
|
||||
|
||||
```text
|
||||
skill_status = compiled
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# 6. 更关键的是:这个 mq.md 可以逐渐“去 LLM 化”
|
||||
|
||||
这是你刚才说的:
|
||||
|
||||
> “其中可以使用非常少的大模型思考,或者使用类似 Jev 的决策模型,或者干脆只使用分支控制。”
|
||||
|
||||
**我非常赞同。**
|
||||
|
||||
我甚至认为这应该成为 Marqdo Agent 的核心架构。
|
||||
|
||||
可以设计成一个:
|
||||
|
||||
# Cognitive Cost Ladder
|
||||
|
||||
```text
|
||||
Level 0
|
||||
Deterministic Marqdo
|
||||
│
|
||||
▼
|
||||
Level 1
|
||||
Rule / Branch
|
||||
│
|
||||
▼
|
||||
Level 2
|
||||
Small decision model
|
||||
│
|
||||
▼
|
||||
Level 3
|
||||
Small LLM
|
||||
│
|
||||
▼
|
||||
Level 4
|
||||
Full LLM reasoning
|
||||
│
|
||||
▼
|
||||
Level 5
|
||||
Multi-agent exploration
|
||||
```
|
||||
|
||||
新问题:
|
||||
|
||||
```text
|
||||
Full LLM
|
||||
↓
|
||||
learn
|
||||
↓
|
||||
Marqdo Skill
|
||||
↓
|
||||
validation
|
||||
↓
|
||||
Level 2/1
|
||||
```
|
||||
|
||||
于是 Agent 会越来越“便宜”。
|
||||
|
||||
---
|
||||
|
||||
# 7. 这甚至给你提供了一个极漂亮的学习曲线
|
||||
|
||||
假设一个任务第一次需要:
|
||||
|
||||
```text
|
||||
8 次 LLM calls
|
||||
```
|
||||
|
||||
第二次:
|
||||
|
||||
```text
|
||||
5 次
|
||||
```
|
||||
|
||||
第三次:
|
||||
|
||||
```text
|
||||
3 次
|
||||
```
|
||||
|
||||
学习完成:
|
||||
|
||||
```text
|
||||
0 次
|
||||
```
|
||||
|
||||
或者:
|
||||
|
||||
```text
|
||||
1 次 small model
|
||||
```
|
||||
|
||||
这时候你就有一个非常漂亮的指标:
|
||||
|
||||
## Reasoning Amortization
|
||||
|
||||
定义:
|
||||
|
||||
$$
|
||||
C_n =
|
||||
\frac{
|
||||
\text{LLM inference cost over first } n \text{ executions}
|
||||
}{n}
|
||||
$$
|
||||
|
||||
随着执行次数增加:
|
||||
|
||||
```text
|
||||
Cost
|
||||
│\
|
||||
│ \
|
||||
│ \
|
||||
│ \
|
||||
│ \____
|
||||
│ ─────────
|
||||
└───────────────────
|
||||
executions
|
||||
```
|
||||
|
||||
**这甚至解决了你之前担心的 Marqdo 性能难以证明的问题。**
|
||||
|
||||
你之前不知道怎么证明:
|
||||
|
||||
> Marqdo 比 Python 更适合 AI。
|
||||
|
||||
但现在你可以研究:
|
||||
|
||||
> **Marqdo Agent 能否把高成本 LLM reasoning 逐渐编译成低成本 executable knowledge?**
|
||||
|
||||
这个是一个非常明确、可测量、而且很有研究价值的问题。
|
||||
|
||||
---
|
||||
|
||||
# 8. 但是这里有一个非常重要的危险
|
||||
|
||||
你说:
|
||||
|
||||
> “超过三次调用就自动总结。”
|
||||
|
||||
我建议:
|
||||
|
||||
**不要真的这么简单。**
|
||||
|
||||
三次可以作为:
|
||||
|
||||
> **触发学习候选**
|
||||
|
||||
但不能作为:
|
||||
|
||||
> **自动认为已经学会。**
|
||||
|
||||
否则会发生:
|
||||
|
||||
```text
|
||||
第一次:特殊情况
|
||||
第二次:特殊情况
|
||||
第三次:特殊情况
|
||||
```
|
||||
|
||||
LLM 总结:
|
||||
|
||||
```text
|
||||
General rule:
|
||||
...
|
||||
```
|
||||
|
||||
然后把偶然性错误“编译”进 Skill。
|
||||
|
||||
这会非常危险。
|
||||
|
||||
---
|
||||
|
||||
# 9. 我建议建立“经验 → 技能”的四阶段
|
||||
|
||||
## Stage 1 — Episode
|
||||
|
||||
每次任务产生:
|
||||
|
||||
```text
|
||||
task
|
||||
context
|
||||
actions
|
||||
tool calls
|
||||
observations
|
||||
errors
|
||||
result
|
||||
cost
|
||||
```
|
||||
|
||||
例如:
|
||||
|
||||
```text
|
||||
episode-001.mq.md
|
||||
episode-002.mq.md
|
||||
episode-003.mq.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Stage 2 — Candidate Skill
|
||||
|
||||
达到阈值之后:
|
||||
|
||||
```text
|
||||
3+ similar episodes
|
||||
```
|
||||
|
||||
让 LLM 总结:
|
||||
|
||||
```text
|
||||
candidate.mq.md
|
||||
```
|
||||
|
||||
但它还是:
|
||||
|
||||
> **candidate**
|
||||
|
||||
而不是正式 skill。
|
||||
|
||||
---
|
||||
|
||||
## Stage 3 — Verification
|
||||
|
||||
用历史任务重新运行:
|
||||
|
||||
```text
|
||||
episode 1
|
||||
episode 2
|
||||
episode 3
|
||||
```
|
||||
|
||||
测试:
|
||||
|
||||
```text
|
||||
candidate skill
|
||||
```
|
||||
|
||||
如果:
|
||||
|
||||
```text
|
||||
3 / 3 success
|
||||
```
|
||||
|
||||
再加新的 held-out task:
|
||||
|
||||
```text
|
||||
episode 4
|
||||
```
|
||||
|
||||
如果:
|
||||
|
||||
```text
|
||||
4 / 4
|
||||
```
|
||||
|
||||
才提高 confidence。
|
||||
|
||||
---
|
||||
|
||||
## Stage 4 — Compilation
|
||||
|
||||
最后:
|
||||
|
||||
```text
|
||||
candidate
|
||||
↓
|
||||
verified
|
||||
↓
|
||||
compiled
|
||||
```
|
||||
|
||||
此时才允许:
|
||||
|
||||
```text
|
||||
llm_free = true
|
||||
```
|
||||
|
||||
你目前的系统已经有 `llm_free` 命中后直接运行 resource 的机制。
|
||||
|
||||
我认为未来这个 flag **不应该是人工/静态指定的属性,而应该成为学习系统的产物。**
|
||||
|
||||
---
|
||||
|
||||
# 10. 甚至可以让 mq.md 自己记录“学习程度”
|
||||
|
||||
比如:
|
||||
|
||||
```yaml
|
||||
---
|
||||
type: skill
|
||||
status: compiled
|
||||
confidence: 0.97
|
||||
evidence: 17
|
||||
success_rate: 0.941
|
||||
llm_free: true
|
||||
created_from: 6c8...
|
||||
last_validated: 2026-09-23
|
||||
avg_reasoning_cost: 0.2
|
||||
---
|
||||
```
|
||||
|
||||
然后正文:
|
||||
|
||||
```markdown
|
||||
# CSV 成人用户筛选
|
||||
|
||||
## Intent
|
||||
|
||||
筛选年龄大于 18 岁的用户。
|
||||
|
||||
## Procedure
|
||||
|
||||
- 读取 CSV
|
||||
- 判断 age
|
||||
- 过滤
|
||||
- 排序
|
||||
|
||||
## Fallback
|
||||
|
||||
如果无法确定 age 字段:
|
||||
|
||||
> ask_llm
|
||||
|
||||
## Verification
|
||||
|
||||
| Case | Result |
|
||||
|---|---|
|
||||
| case-001 | ✓ |
|
||||
| case-002 | ✓ |
|
||||
| case-003 | ✓ |
|
||||
| case-004 | ✓ |
|
||||
```
|
||||
|
||||
注意这个设计里非常重要的一点:
|
||||
|
||||
> **Skill 本身可以包含一个 fallback 到 LLM 的路径。**
|
||||
|
||||
所以不是:
|
||||
|
||||
```text
|
||||
学会 / 不会
|
||||
```
|
||||
|
||||
而是:
|
||||
|
||||
```text
|
||||
确定部分 → deterministic
|
||||
不确定部分 → small model
|
||||
真正未知 → full LLM
|
||||
```
|
||||
|
||||
这就非常接近人类的实际认知过程。
|
||||
|
||||
---
|
||||
|
||||
# 11. 我甚至会把 Agent 的运行模型改成这样
|
||||
|
||||
```text
|
||||
User Task
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Skill Resolver │
|
||||
└────────┬────────┘
|
||||
│
|
||||
┌────────────┼────────────┐
|
||||
▼ ▼ ▼
|
||||
Exact Near Unknown
|
||||
Hit Hit
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
mq.md execute small model Full LLM
|
||||
│ │ │
|
||||
└────────────┼────────────┘
|
||||
▼
|
||||
Result
|
||||
│
|
||||
▼
|
||||
Episode
|
||||
│
|
||||
▼
|
||||
Experience Store
|
||||
│
|
||||
┌────────┴────────┐
|
||||
│ │
|
||||
repeated? novel?
|
||||
│
|
||||
▼
|
||||
Skill synthesis
|
||||
│
|
||||
▼
|
||||
Verification
|
||||
│
|
||||
▼
|
||||
Compilation
|
||||
│
|
||||
▼
|
||||
mq.md Skill
|
||||
```
|
||||
|
||||
**这就已经不是普通 Agent framework 了。**
|
||||
|
||||
它是一个:
|
||||
|
||||
> **Self-compiling Agent**
|
||||
|
||||
---
|
||||
|
||||
# 12. 而 Marqdo 在这里有一个其他 Agent framework 很难天然拥有的优势
|
||||
|
||||
因为“学习结果”不是:
|
||||
|
||||
```json
|
||||
{
|
||||
"memory": "..."
|
||||
}
|
||||
```
|
||||
|
||||
也不是:
|
||||
|
||||
```text
|
||||
embedding vector
|
||||
```
|
||||
|
||||
而是:
|
||||
|
||||
```text
|
||||
learned_skill.mq.md
|
||||
```
|
||||
|
||||
这意味着用户可以:
|
||||
|
||||
```text
|
||||
打开
|
||||
阅读
|
||||
修改
|
||||
运行
|
||||
测试
|
||||
分享
|
||||
Git commit
|
||||
diff
|
||||
fork
|
||||
```
|
||||
|
||||
甚至:
|
||||
|
||||
```text
|
||||
Agent A
|
||||
↓
|
||||
learned_skill.mq.md
|
||||
↓
|
||||
Agent B
|
||||
```
|
||||
|
||||
**Agent 的经验第一次成为一种真正可移植的软件 artifact。**
|
||||
|
||||
我认为这是你整个 Marqdo 项目里非常值得深入挖掘的东西。
|
||||
|
||||
---
|
||||
|
||||
# 13. 而且它和你前面说的“Executable Knowledge”完全闭环
|
||||
|
||||
你之前提出:
|
||||
|
||||
```text
|
||||
Executable Document
|
||||
↓
|
||||
Executable Knowledge Base
|
||||
```
|
||||
|
||||
现在再往前一步:
|
||||
|
||||
```text
|
||||
Executable Knowledge
|
||||
↓
|
||||
Agent Experience
|
||||
↓
|
||||
Agent Learning
|
||||
↓
|
||||
Compiled Knowledge
|
||||
↓
|
||||
Executable Skill
|
||||
```
|
||||
|
||||
于是整个 Marqdo 可以形成:
|
||||
|
||||
```text
|
||||
┌───────────────┐
|
||||
│ Human │
|
||||
└───────┬───────┘
|
||||
│
|
||||
▼
|
||||
Natural Task
|
||||
│
|
||||
▼
|
||||
AI Reasoning
|
||||
│
|
||||
▼
|
||||
Executable Work
|
||||
│
|
||||
▼
|
||||
.mq.md
|
||||
│
|
||||
┌───────┴───────┐
|
||||
▼ ▼
|
||||
Knowledge Skill
|
||||
│ │
|
||||
└───────┬───────┘
|
||||
▼
|
||||
Future Agent
|
||||
│
|
||||
▼
|
||||
Less Reasoning
|
||||
│
|
||||
▼
|
||||
More Execution
|
||||
```
|
||||
|
||||
这就是我现在最喜欢 Marqdo 的地方:
|
||||
|
||||
> **Agent 不只是使用知识库;Agent 会把自己的经验沉淀成知识库,然后知识库反过来改变 Agent 的计算方式。**
|
||||
|
||||
这是一个真正的闭环。
|
||||
|
||||
---
|
||||
|
||||
## 最后,我会建议你对现有 Agent 做一个非常明确的架构升级
|
||||
|
||||
你现在已经有:
|
||||
|
||||
```text
|
||||
reuse
|
||||
near_match
|
||||
soft_match
|
||||
promote
|
||||
improve_every=3
|
||||
llm_free
|
||||
```
|
||||
|
||||
所以**完全没必要推倒重来**。
|
||||
|
||||
我会把它演化成:
|
||||
|
||||
```text
|
||||
Current Agent
|
||||
│
|
||||
┌────────┴────────┐
|
||||
│ │
|
||||
Execution Experience
|
||||
│ │
|
||||
│ ▼
|
||||
│ Episode Store
|
||||
│ │
|
||||
│ ≥ N similar
|
||||
│ │
|
||||
│ ▼
|
||||
│ Skill Synthesis
|
||||
│ │
|
||||
│ ▼
|
||||
│ Skill Validation
|
||||
│ │
|
||||
│ ▼
|
||||
│ Skill Compilation
|
||||
│ │
|
||||
└─────────────┬───┘
|
||||
▼
|
||||
.mq.md Skill
|
||||
│
|
||||
┌────────┼────────┐
|
||||
▼ ▼ ▼
|
||||
Branch Small LLM Full LLM
|
||||
```
|
||||
|
||||
而且你当前已经有一个非常好的“终点”:**当 child 返回 value 时,可以 deterministic 地 solidify,而不是再次让父 LLM 发明 patch。** 当前代码已经明确这么处理。
|
||||
|
||||
所以我会把你的下一版 Agent 的核心 slogan 直接定成:
|
||||
|
||||
> **Think → Execute → Learn → Compile → Reuse**
|
||||
|
||||
或者更 Marqdo 一点:
|
||||
|
||||
> **Every successful thought can become executable knowledge.**
|
||||
|
||||
这比“Marqdo 有一个 Agent framework”强得多。
|
||||
|
||||
而且这件事**非常适合做你前面说的论文实验**:你甚至可以测一个 Agent 在重复任务上的 **LLM calls / tokens / latency / success rate 随经验次数变化的曲线**,直接验证“推理成本被 mq.md 技能逐渐摊销”的假设。这样 Marqdo 的 Agent、Executable Knowledge 和你之前苦恼的“性能怎么证明”三个问题,反而被同一个实验串起来了。
|
||||
+2626
File diff suppressed because it is too large.
Load diff
+541
@@ -0,0 +1,541 @@
|
||||
有时候可能用户不太好接入Jev,所以阶段4应该要允许用户使用小参数的模型代替,你认为呢?
|
||||
|
||||
我非常赞同,而且我甚至建议**不要把 Jev 作为阶段 4 的必选组件**。
|
||||
|
||||
更准确的设计应该是:
|
||||
|
||||
> **Jev 是一种可选的决策器(Decision Engine),小参数模型是另一种可选实现;Marqdo Agent 本身只规定“需要一个 Router/Policy”,而不规定 Router 必须由什么实现。**
|
||||
|
||||
这样会让整个框架开放很多。
|
||||
|
||||
### 1. 阶段 4 应该改成“Adaptive Routing”
|
||||
|
||||
原来的:
|
||||
|
||||
```text
|
||||
Phase 4
|
||||
↓
|
||||
Jev Router
|
||||
```
|
||||
|
||||
我建议改成:
|
||||
|
||||
```text
|
||||
Phase 4: Adaptive Routing
|
||||
│
|
||||
┌─────────┼─────────┐
|
||||
↓ ↓ ↓
|
||||
Marqdo Rule Jev Small LLM
|
||||
│ │ │
|
||||
└─────────┼─────────┘
|
||||
↓
|
||||
Route Policy
|
||||
```
|
||||
|
||||
甚至还可以允许:
|
||||
|
||||
```text
|
||||
Rule
|
||||
Small Model
|
||||
Jev
|
||||
Full LLM
|
||||
```
|
||||
|
||||
混合使用。
|
||||
|
||||
---
|
||||
|
||||
# 2. 为什么这个设计更合理
|
||||
|
||||
因为用户真正需要的不是:
|
||||
|
||||
> “我要接入 Jev。”
|
||||
|
||||
而是:
|
||||
|
||||
> “我希望 Agent 学会以后,不要每次都调用大模型。”
|
||||
|
||||
这两个概念应该分开。
|
||||
|
||||
所以 Marqdo 只定义接口:
|
||||
|
||||
```text
|
||||
agent.route(task, candidates)
|
||||
```
|
||||
|
||||
至于内部怎么实现,可以是:
|
||||
|
||||
### A. 纯 Marqdo
|
||||
|
||||
```text
|
||||
if task.type == "json":
|
||||
return json-repair
|
||||
|
||||
if task.type == "search":
|
||||
return corpus-search
|
||||
```
|
||||
|
||||
完全不需要模型。
|
||||
|
||||
---
|
||||
|
||||
### B. 小参数模型
|
||||
|
||||
比如用户已经有:
|
||||
|
||||
```text
|
||||
Qwen 0.5B
|
||||
Qwen 1.5B
|
||||
Llama 1B
|
||||
某个本地模型
|
||||
```
|
||||
|
||||
就可以:
|
||||
|
||||
```text
|
||||
task
|
||||
↓
|
||||
small model
|
||||
↓
|
||||
"json-repair"
|
||||
```
|
||||
|
||||
甚至可以让用户直接配置:
|
||||
|
||||
```yaml
|
||||
router:
|
||||
type: llm
|
||||
|
||||
provider: openai-compatible
|
||||
base_url: ...
|
||||
api_key: ...
|
||||
model: ...
|
||||
```
|
||||
|
||||
这和你之前设计 AI 斗蛐蛐时提出的 **OpenAI-compatible URL + Key** 思路其实也很契合。
|
||||
|
||||
---
|
||||
|
||||
### C. Jev
|
||||
|
||||
如果用户安装了 Jev:
|
||||
|
||||
```text
|
||||
task
|
||||
↓
|
||||
Jev
|
||||
↓
|
||||
policy
|
||||
```
|
||||
|
||||
那么就用 Jev。
|
||||
|
||||
---
|
||||
|
||||
### D. 大模型
|
||||
|
||||
甚至最开始没有任何 Router:
|
||||
|
||||
```text
|
||||
task
|
||||
↓
|
||||
large LLM
|
||||
↓
|
||||
decide
|
||||
```
|
||||
|
||||
也是合法的。
|
||||
|
||||
所以:
|
||||
|
||||
> **Router 是抽象能力,不是具体模型。**
|
||||
|
||||
---
|
||||
|
||||
# 3. 更重要的是:Router 本身也可以被学习
|
||||
|
||||
这里我觉得还能进一步提升你的设计。
|
||||
|
||||
一开始:
|
||||
|
||||
```text
|
||||
Task
|
||||
↓
|
||||
Large LLM
|
||||
↓
|
||||
选择 skill
|
||||
```
|
||||
|
||||
系统观察:
|
||||
|
||||
```text
|
||||
task A → json-repair
|
||||
task B → json-repair
|
||||
task C → json-repair
|
||||
task D → corpus-search
|
||||
```
|
||||
|
||||
之后可以自动生成:
|
||||
|
||||
```text
|
||||
router.mq.md
|
||||
```
|
||||
|
||||
例如:
|
||||
|
||||
```markdown
|
||||
---
|
||||
type: policy
|
||||
status: compiled
|
||||
---
|
||||
|
||||
## route
|
||||
|
||||
1. `task.type` == "json"
|
||||
*"json-repair"*
|
||||
|
||||
2. `task.type` == "search"
|
||||
*"corpus-search"*
|
||||
|
||||
3. *
|
||||
*"llm"*
|
||||
```
|
||||
|
||||
那么你甚至不需要 Jev。
|
||||
|
||||
**Marqdo 自己的控制流就是 Router。**
|
||||
|
||||
---
|
||||
|
||||
# 4. 所以我会重新设计 Cost Ladder
|
||||
|
||||
之前的:
|
||||
|
||||
```text
|
||||
L0 deterministic
|
||||
L1 rule
|
||||
L2 small model
|
||||
L3 small LLM
|
||||
L4 large LLM
|
||||
L5 exploration
|
||||
```
|
||||
|
||||
我建议改成:
|
||||
|
||||
```text
|
||||
┌────────────────────────┐
|
||||
│ Adaptive Router │
|
||||
└───────────┬────────────┘
|
||||
│
|
||||
┌─────────────────┼─────────────────┐
|
||||
↓ ↓ ↓
|
||||
Deterministic Small Model Large LLM
|
||||
│ │ │
|
||||
↓ ↓ ↓
|
||||
L0/L1 L2/L3 L4
|
||||
│
|
||||
↓
|
||||
Exploration L5
|
||||
```
|
||||
|
||||
这里最关键的是:
|
||||
|
||||
> **L2/L3 不应该规定具体模型。**
|
||||
|
||||
可以是:
|
||||
|
||||
```text
|
||||
Marqdo rules
|
||||
OR
|
||||
Jev
|
||||
OR
|
||||
0.5B model
|
||||
OR
|
||||
1B model
|
||||
OR
|
||||
3B model
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# 5. 我甚至建议用户可以配置多个 Router
|
||||
|
||||
例如:
|
||||
|
||||
```yaml
|
||||
router:
|
||||
candidates:
|
||||
|
||||
- name: rules
|
||||
type: marqdo
|
||||
|
||||
- name: local
|
||||
type: llm
|
||||
model: qwen-0.5b
|
||||
|
||||
- name: jev
|
||||
type: jev
|
||||
|
||||
- name: cloud
|
||||
type: llm
|
||||
model: ...
|
||||
```
|
||||
|
||||
然后 Agent 自己选择:
|
||||
|
||||
```text
|
||||
Task
|
||||
↓
|
||||
Router Resolver
|
||||
↓
|
||||
rules?
|
||||
↓
|
||||
confidence insufficient
|
||||
↓
|
||||
small model?
|
||||
↓
|
||||
confidence insufficient
|
||||
↓
|
||||
Jev?
|
||||
↓
|
||||
confidence insufficient
|
||||
↓
|
||||
large LLM
|
||||
```
|
||||
|
||||
这就形成一个**分级决策系统**。
|
||||
|
||||
---
|
||||
|
||||
# 6. 最漂亮的地方:Router 也可以被编译
|
||||
|
||||
最终甚至可以:
|
||||
|
||||
```text
|
||||
第一次:
|
||||
|
||||
LLM
|
||||
↓
|
||||
决定调用 Skill A
|
||||
|
||||
第二次:
|
||||
|
||||
LLM
|
||||
↓
|
||||
决定调用 Skill A
|
||||
|
||||
第三次:
|
||||
|
||||
LLM
|
||||
↓
|
||||
决定调用 Skill A
|
||||
|
||||
Learning
|
||||
↓
|
||||
发现稳定模式
|
||||
|
||||
Compile
|
||||
↓
|
||||
router.mq.md
|
||||
```
|
||||
|
||||
以后:
|
||||
|
||||
```text
|
||||
Task
|
||||
↓
|
||||
router.mq.md
|
||||
↓
|
||||
skill A
|
||||
↓
|
||||
execute
|
||||
```
|
||||
|
||||
**0 LLM。**
|
||||
|
||||
于是整个 Agent 的学习就不只是:
|
||||
|
||||
```text
|
||||
学习如何解决任务
|
||||
```
|
||||
|
||||
而是同时学习:
|
||||
|
||||
```text
|
||||
1. 学习如何解决任务
|
||||
2. 学习什么时候使用什么技能
|
||||
3. 学习什么时候需要大模型
|
||||
```
|
||||
|
||||
这三个层次组合起来,我认为比单纯引入 Jev 更有意思。
|
||||
|
||||
---
|
||||
|
||||
# 7. 最终可以形成三级编译
|
||||
|
||||
我现在反而觉得你的 Agent Framework 可以把整个“学习”定义成三个阶段:
|
||||
|
||||
```text
|
||||
Experience
|
||||
↓
|
||||
Skill Compilation
|
||||
↓
|
||||
Policy Compilation
|
||||
↓
|
||||
Execution Compilation
|
||||
```
|
||||
|
||||
### 第一层:Skill Compilation
|
||||
|
||||
```text
|
||||
LLM experience
|
||||
↓
|
||||
skill.mq.md
|
||||
```
|
||||
|
||||
解决:
|
||||
|
||||
> **怎么做?**
|
||||
|
||||
### 第二层:Policy Compilation
|
||||
|
||||
```text
|
||||
many task → skill mappings
|
||||
↓
|
||||
policy.mq.md
|
||||
```
|
||||
|
||||
解决:
|
||||
|
||||
> **什么时候做什么?**
|
||||
|
||||
### 第三层:Execution Compilation
|
||||
|
||||
```text
|
||||
skill + policy
|
||||
↓
|
||||
pure Marqdo execution
|
||||
```
|
||||
|
||||
解决:
|
||||
|
||||
> **能不能完全不思考直接执行?**
|
||||
|
||||
这时候整个 Agent 就变成:
|
||||
|
||||
```text
|
||||
Large LLM
|
||||
│
|
||||
Exploration
|
||||
↓
|
||||
Experience
|
||||
↓
|
||||
┌───────────────┐
|
||||
│ Skill Compile │
|
||||
└───────┬───────┘
|
||||
↓
|
||||
Skill MQ
|
||||
│
|
||||
┌───────┴───────┐
|
||||
↓ ↓
|
||||
Policy Compile Direct Execute
|
||||
↓
|
||||
Router MQ
|
||||
│
|
||||
↓
|
||||
Skill Selection
|
||||
│
|
||||
↓
|
||||
Execute
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# 8. 因此 Phase 4 我建议正式改名
|
||||
|
||||
不要叫:
|
||||
|
||||
> Jev Router
|
||||
|
||||
而叫:
|
||||
|
||||
> **Adaptive Policy / Routing Layer**
|
||||
|
||||
然后提供四种 backend:
|
||||
|
||||
```text
|
||||
router.backend = marqdo
|
||||
router.backend = small-llm
|
||||
router.backend = jev
|
||||
router.backend = llm
|
||||
```
|
||||
|
||||
甚至:
|
||||
|
||||
```text
|
||||
router.backend = auto
|
||||
```
|
||||
|
||||
让 Agent 根据可用环境自动选择。
|
||||
|
||||
---
|
||||
|
||||
## 9. 用户体验也会好很多
|
||||
|
||||
用户第一次安装 Marqdo:
|
||||
|
||||
```text
|
||||
marqdo agent run ...
|
||||
```
|
||||
|
||||
没有 Jev,也没有小模型。
|
||||
|
||||
完全没问题:
|
||||
|
||||
```text
|
||||
→ 使用配置的大模型
|
||||
```
|
||||
|
||||
用户后来配置:
|
||||
|
||||
```yaml
|
||||
router:
|
||||
backend: small-llm
|
||||
provider: openai-compatible
|
||||
model: qwen-0.5b
|
||||
```
|
||||
|
||||
Agent 就自动变成:
|
||||
|
||||
```text
|
||||
大模型:负责复杂推理 / 学习
|
||||
小模型:负责日常路由
|
||||
Marqdo:负责确定性执行
|
||||
```
|
||||
|
||||
如果用户安装 Jev:
|
||||
|
||||
```yaml
|
||||
router:
|
||||
backend: jev
|
||||
```
|
||||
|
||||
则:
|
||||
|
||||
```text
|
||||
Jev:负责路由
|
||||
大模型:负责复杂任务
|
||||
Marqdo:负责执行
|
||||
```
|
||||
|
||||
**用户完全不需要改变 Agent 的上层 API。**
|
||||
|
||||
---
|
||||
|
||||
我认为这个调整非常重要,因为它让 **Marqdo Agent 的核心创新从“依赖某个新模型”变成“模型无关的认知编译架构”**。
|
||||
|
||||
最终可以浓缩成一句话:
|
||||
|
||||
> **大模型负责探索,小模型/Jev 负责决策,Marqdo 负责执行;随着经验积累,探索和决策逐渐被编译成可执行的 `.mq.md`。**
|
||||
|
||||
这其实比“接入 Jev”本身更符合你最开始提出的 **Think → Execute → Learn → Compile → Reuse** 路线。
|
||||
+1173
File diff suppressed because it is too large.
Load diff
+1335
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,77 @@
|
||||
# Marqdo AI 栈统一开发方案(llm + agent)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Status | **Active** |
|
||||
| Date | 2026-09-24 |
|
||||
| Source | [doc/next/002–005](../next/) · [ext-llm.md](../design/ext-llm.md) · [ext-agent.md](../design/ext-agent.md) |
|
||||
|
||||
## 1. 一句话
|
||||
|
||||
> **Document → Prompt → LLM → Agent → Skill → Policy → Executable Knowledge**
|
||||
|
||||
大模型负责探索,小模型(`router_model` / 任意 llm 句柄)负责决策,Marqdo 负责执行;经验编译进 `.mq.md` 后推理成本下降。
|
||||
|
||||
## 2. 分层(勿混)
|
||||
|
||||
| 层 | 职责 | 落点 |
|
||||
|----|------|------|
|
||||
| **Intelligence primitive** | `ask` / `stream` / `collect`;结果对象 | `ext/ai/llm` |
|
||||
| **Agent runtime** | `run` / `step` / `plan` / `route` | `ext/ai/agent` |
|
||||
| **Memory / compile** | episode → skill → policy | `plugins/agent`(ABI;**不进** `src/host`) |
|
||||
| **Transport** | HTTP / SSE / provider | `ext/ai/llm/openai` · `ollama`;用户不可见 |
|
||||
|
||||
硬规则:`ext/**` 不调 `host_*`;agent/OKF/路由热路径只进 `plugins/agent`。
|
||||
|
||||
## 3. LLM(005)— 已/待
|
||||
|
||||
| 项 | 状态 |
|
||||
|----|------|
|
||||
| 语义 API:`ask` / `stream` / `collect`(`complete` 降为兼容原语) | **done** |
|
||||
| `create` 工厂;handle 带 `backend` | **done** |
|
||||
| LLMResult:`text` / `model` / `finish` / `backend` / `usage` / `tokens` | **done** |
|
||||
| Named models(`llm.fast` / `llm.reasoning`) | **done** |
|
||||
| Prompt document:`prompt` / `prompt_load` + `type: prompt` 正文 | **done**(无参 `ask`/`stream` → `sys.module_source` + `prompt_body`) |
|
||||
| 拆 `ext/ai/llm/openai.mq.md` + `ollama` 后端 | **done** |
|
||||
| tool_calls 完整 Result | **done**(`openai.tool_calls_from` + `LLMResult.tool_calls`;`llm-tool-calls-offline`) |
|
||||
| Transport → 可选 ABI 插件 (`plugins/llm`) | **done** |
|
||||
| Artifact Metadata Binding(`${env.*}` / `secret` · `sys.meta`) | **done**(Phase 1 Metadata-only + Phase 2 scope lift;[binding.md](../design/binding.md)) |
|
||||
|
||||
作者面优先:
|
||||
|
||||
```marqdo
|
||||
**answer = > llm.ask prompt="What is Marqdo?"**
|
||||
**events = > model.stream prompt=`p`**
|
||||
**text = > llm.collect events=`events`**
|
||||
**fast = > llm.fast**
|
||||
**prompt = > llm.prompt path="prompts/task.md"**
|
||||
```
|
||||
|
||||
## 4. Agent(002–004)— 已/待
|
||||
|
||||
| 项 | 状态 |
|
||||
|----|------|
|
||||
| Skill Compilation:episode → maybe_learn → `llm_free` | **done** |
|
||||
| Adaptive Routing:`marqdo` \| `small-llm` \| `llm` \| `auto`(`jev`→`small-llm` 别名) | **done** |
|
||||
| Policy Compilation → `policies/router.mq.md` | **done** |
|
||||
| `run` 串联 route → execute → record → learn | **done** |
|
||||
| Agent 默认走 `model.ask` + Result.usage 记成本 | **done**(`model_turn`) |
|
||||
| Prompt artifact / document-as-prompt | **done**(经 llm `path=` / `prompt`) |
|
||||
| 零 LLM 执行(policy+skill → `llm_calls=0`) | **done** |
|
||||
| `plan` 不覆盖已有 `llm_free` resource | **done** |
|
||||
| 真 MCP client(stdio) / resume checkpoint / 小模型路由请求面 | **done**(live 路由仍可选 skip) |
|
||||
|
||||
## 5. 联调顺序
|
||||
|
||||
1. **LLM Result → Agent metrics** — **done**
|
||||
2. **Named handles** — **done**
|
||||
3. **Prompt document** — **done**(`type: prompt` 剥 frontmatter;无参 `ask`/`stream` 经 `sys.module_source`)
|
||||
4. **Backend split** — **done**
|
||||
5. **Execution compilation** — **done**(`agent-zero-llm` 金样 + harness)
|
||||
6. **Metadata Binding** — **done**(Phase 1 + Phase 2 scope lift;`llm-meta-offline` · `meta-lift`)
|
||||
|
||||
## 6. 验收金样
|
||||
|
||||
- Agent:`tests/ext/agent-v2-learn.mq.md` · `agent-p4-route.mq.md` · `agent-zero-llm.mq.md` · `agent-plan-preserve.mq.md` · `agent-mcp-client.mq.md` · `agent-resume.mq.md` · `agent-small-llm-route.mq.md` · `scripts/agent-harness.sh`
|
||||
- LLM:`tests/ext/llm-ask-offline.mq.md` · `llm-stream-offline.mq.md` · `llm-ctor-offline.mq.md` · `llm-named-offline.mq.md` · `llm-meta-offline.mq.md` · `llm-entry-prompt-offline.mq.md` · `llm-tool-calls-offline.mq.md`
|
||||
- Binding:`tests/structure/meta-binding.mq.md` · `meta-lift.mq.md`
|
||||
@@ -0,0 +1,34 @@
|
||||
# 盲测实验(§5.2(b) 查询式 vs 全文档背诵)—— 2026-09-23
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 模型 | `deepseek-flash` @ `https://llm.cflmy.cn/v1` |
|
||||
| 样本 | 6 任务(业务 3 + QuixBugs 改写 3),oracle = stdout 精确匹配 |
|
||||
| 协议 | 系统提示词 = 硬规则 12 条 + 契约表三形状(两臂同文);**唯一变量 = 语法来源**:`query` 臂 MLSP 查询通道(≤3 次)vs `docs` 臂完整 markup 参考(背诵) |
|
||||
| 修复 | validate 诊断 + stderr 回喂,≤2 次修复;temperature=0,max_tokens=8000 |
|
||||
|
||||
## 汇总(判据:Q 通过率不劣于 D;首轮语法错误率 Q≤D;token Q≤70%%D;轮次不升)
|
||||
|
||||
| 臂 | 通过(一轮) | 通过(最终≤3) | 首轮语法错误 | 首轮契约诊断 | 平均修复轮(过例) | token 合计(prompt+completion) | 弃权/基础设施 |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| query | 0/3 | 0/3 | 2/3 | 1/3 | 0.00 | 69771(42478+27293) | 3 |
|
||||
| docs | 3/6 | 4/6 | 2/6 | 0/6 | 0.50 | 57276(33346+23930) | 0 |
|
||||
|
||||
## 逐例
|
||||
|
||||
| 例 | 臂 | 结果 | 修复轮 | 首轮语法错 | 首轮契约诊断 | token(p+c) | 查询 | 备注 |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| claims_reserve_settle | query | load-fail | 0 | 是 | 否 | 11987+8177 | 3 | {"error":{"code":"mlsp.parse_error","message":"<memory>: trailing input in expression: \"(claim - deductible, 0)\""},"id":1,"ok":false} \| stderr=error: /tmp/marqdo_blind_1694593_query_claims_reserve_settle.mq.md: trailing input in expression: "(claim - deductible, 0)" |
|
||||
| contract_milestone_pay | query | infra | 0 | 否 | 否 | 3970+4718 | 3 | 非 JSON / 空体: |
|
||||
| vendor_po_gate | query | load-fail | 0 | 是 | 否 | 9552+3686 | 3 | 回复中无程序 \| 开头: QUERY: 条件分支 写法 |
|
||||
| qb_gcd | query | run-fail | 0 | 否 | 是 | 9700+8488 | 3 | \| stderr=error: /tmp/marqdo_blind_1694593_query_qb_gcd.mq.md:26:1: unknown function `gcd(48` |
|
||||
| qb_max_sublist_sum | query | infra | 0 | 否 | 否 | 1253+134 | 3 | 非 JSON / 空体: |
|
||||
| qb_get_factors | query | infra | 0 | 否 | 否 | 6016+2090 | 3 | 非 JSON / 空体: |
|
||||
| claims_reserve_settle | docs | pass | 0 | 否 | 否 | 2699+2987 | 0 | queries=0 |
|
||||
| contract_milestone_pay | docs | pass | 0 | 否 | 否 | 2800+3202 | 0 | queries=0 |
|
||||
| vendor_po_gate | docs | pass | 0 | 否 | 否 | 2690+1264 | 0 | queries=0 |
|
||||
| qb_gcd | docs | pass | 2 | 否 | 否 | 8147+6970 | 0 | queries=0 |
|
||||
| qb_max_sublist_sum | docs | run-fail | 0 | 是 | 否 | 8518+3609 | 0 | \| stderr=error: /tmp/marqdo_blind_1694593_docs_qb_max_sublist_sum.mq.md:3:1: unknown function `[-2,` |
|
||||
| qb_get_factors | docs | run-fail | 0 | 是 | 否 | 8492+5898 | 0 | \| stderr=error: /tmp/marqdo_blind_1694593_docs_qb_get_factors.mq.md:20:1: missing argument for parameter `value` |
|
||||
|
||||
报告由 `tests/blind_experiment.rs` 生成;材料冻结于 `tests/blind/`(任务书 + docs-bundle),判据见 `doc/roadmap/perf-validation.md` §2。
|
||||
@@ -0,0 +1,34 @@
|
||||
# 盲测实验(§5.2(b) 查询式 vs 全文档背诵)—— 2026-09-23
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 模型 | `mimo-v2.6-pro` @ `https://llm.cflmy.cn/v1` |
|
||||
| 样本 | 6 任务(业务 3 + QuixBugs 改写 3),oracle = stdout 精确匹配 |
|
||||
| 协议 | 系统提示词 = 硬规则 12 条 + 契约表三形状(两臂同文);**唯一变量 = 语法来源**:`query` 臂 MLSP 查询通道(≤3 次)vs `docs` 臂完整 markup 参考(背诵) |
|
||||
| 修复 | validate 诊断 + stderr 回喂,≤2 次修复;temperature=0,max_tokens=8000 |
|
||||
|
||||
## 汇总(判据:Q 通过率不劣于 D;首轮语法错误率 Q≤D;token Q≤70%%D;轮次不升)
|
||||
|
||||
| 臂 | 通过(一轮) | 通过(最终≤3) | 首轮语法错误 | 首轮契约诊断 | 平均修复轮(过例) | token 合计(prompt+completion) | 弃权/基础设施 |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| query | 0/1 | 0/1 | 1/1 | 0/1 | 0.00 | 12428(10255+2173) | 5 |
|
||||
| docs | 0/0 | 0/0 | 0/0 | 0/0 | 0.00 | 0(0+0) | 6 |
|
||||
|
||||
## 逐例
|
||||
|
||||
| 例 | 臂 | 结果 | 修复轮 | 首轮语法错 | 首轮契约诊断 | token(p+c) | 查询 | 备注 |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| claims_reserve_settle | query | infra | 0 | 否 | 否 | 1424+324 | 3 | 非 JSON / 空体: |
|
||||
| contract_milestone_pay | query | infra | 0 | 否 | 否 | 0+0 | 0 | 非 JSON / 空体: |
|
||||
| vendor_po_gate | query | infra | 0 | 否 | 否 | 1417+213 | 3 | 非 JSON / 空体: |
|
||||
| qb_gcd | query | load-fail | 0 | 是 | 否 | 5011+1265 | 0 | {"error":{"code":"mlsp.parse_error","message":"<memory>: unexpected expression input: \"**y**\""},"id":1,"ok":false} \| stderr=error: /tmp/marqdo_blind_1710203_query_qb_gcd.mq.md: unexpected expression input: "**y**" |
|
||||
| qb_max_sublist_sum | query | infra | 0 | 否 | 否 | 1216+172 | 3 | 非 JSON / 空体: |
|
||||
| qb_get_factors | query | infra | 0 | 否 | 否 | 1187+199 | 3 | 非 JSON / 空体: |
|
||||
| claims_reserve_settle | docs | infra | 0 | 否 | 否 | 0+0 | 0 | 非 JSON / 空体: |
|
||||
| contract_milestone_pay | docs | infra | 0 | 否 | 否 | 0+0 | 0 | 非 JSON / 空体: |
|
||||
| vendor_po_gate | docs | infra | 0 | 否 | 否 | 0+0 | 0 | 非 JSON / 空体: |
|
||||
| qb_gcd | docs | infra | 0 | 否 | 否 | 0+0 | 0 | 非 JSON / 空体: |
|
||||
| qb_max_sublist_sum | docs | infra | 0 | 否 | 否 | 0+0 | 0 | 非 JSON / 空体: |
|
||||
| qb_get_factors | docs | infra | 0 | 否 | 否 | 0+0 | 0 | 非 JSON / 空体: {"error":{"message":"upstream connection failed","type":"server_error","param":n… |
|
||||
|
||||
报告由 `tests/blind_experiment.rs` 生成;材料冻结于 `tests/blind/`(任务书 + docs-bundle),判据见 `doc/roadmap/perf-validation.md` §2。
|
||||
@@ -39,6 +39,12 @@
|
||||
| **B0** | 官方示例演示 call_site/源码/OKF;Skill/文档;上下文管道可读 | **done** |
|
||||
| **B0.5** | 本机 live(`.env` → 开发模型)验证「模型用上了文档」 | **done**(`AGENT_LIVE` / `AGENT_HARNESS_LIVE`) |
|
||||
| **B1** | 评测 harness 锁差异化指标(constitution / dump / writeback 遮蔽) | **Active** |
|
||||
| **B2–B5** | 真 MCP / resume+HITL / stats / 并行 | 暂缓 |
|
||||
| **B1.5** | **Skill Compilation v2(Phase 1–3)**:episode → maybe_learn → compile → `llm_free`;`agent.run` / `resolve` / `metrics`;ABI 在 `plugins/agent`(`memory.rs`);金样 `agent-v2-learn` | **done** |
|
||||
| **B1.6 / P4** | **Adaptive Routing**(原 Jev Router 更名):`route` backends = `marqdo` \| `small-llm` \| `jev?` \| `llm` \| `auto`;Policy Compilation → `policies/router.mq.md`;金样 `agent-p4-route`;设计 [004.md](../next/004.md) | **done** |
|
||||
| **L0** | **LLM semantic surface(005)**:`ask` / `stream` / `collect` / `create`;`complete` 降为兼容;统一方案 [ai-stack.md](ai-stack.md) | **done** |
|
||||
| **B2** | 真 MCP stdio client(`agent_mcp_client` + fixture 仍作离线证据) | **done** |
|
||||
| **B3** | resume checkpoint(`.marqdo/agent-resume/`;HITL view 仍暂缓) | **done**(HITL 暂缓) |
|
||||
| **B4–B5** | stats / 并行;小模型 live 路由金样(无 key skip) | 部分 |
|
||||
| — | LLM named models / prompt-document | **done** |
|
||||
|
||||
示例:`examples/agent-pong/` · `examples/agent-okf-flywheel/`。
|
||||
示例:`examples/agent-pong/` · `examples/agent-okf-flywheel/`。v2 设计:[doc/next/002.md](../next/002.md) · [doc/next/003.md](../next/003.md) · P4:[doc/next/004.md](../next/004.md) · LLM:[doc/next/005.md](../next/005.md)。
|
||||
@@ -0,0 +1,118 @@
|
||||
# P4 验证汇总报告 — 2026-09-23
|
||||
|
||||
> 执行 [perf-validation.md](perf-validation.md)(自设计性能验证协议)的结果汇总。
|
||||
> 覆盖 three-problems.md 三大问题的**验证矩阵**(§5.2 a–e)。逐例明细见各实验原始报告(文末索引)。
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 协议 | [perf-validation.md](perf-validation.md)(实验 (a)–(e)) |
|
||||
| 判据源 | [three-problems.md](three-problems.md) §3.7 / 各问题 §2.4 |
|
||||
| 构建 | marqdo v1.0.7(本仓库工作树;金 20 + 全量回归通过后采样) |
|
||||
| 端点 | `llm.cflmy.cn/v1`(OpenAI 兼容;temperature=0) |
|
||||
| 模型 | 修复实验:`cflmy`、`mimo-v2.6-flash`;盲测:`deepseek-flash`、`mimo-v2.6-pro`(按用户指定) |
|
||||
| 回归 | 全量干净回归(无 API key 基线)**零失败** |
|
||||
|
||||
## 1. 五实验结果与判据对照
|
||||
|
||||
| 实验 | 判据 | 结果 | 判定 |
|
||||
|---|---|---|---|
|
||||
| **(a) 锚点消融**(T3.3,双模型×20例×双臂,三轮) | anchor 臂一轮修复率 ≥ 80%;护栏 0 次越界应用 | cflmy anchor **20/20=100%**(plain 17/20=85%);mimo-v2.6-flash anchor **20/20=100%**(plain 18/20=90%;容错加固中的第二轮 anchor 18/20=90% 亦 ≥80%);越界拒/越界应用 0/0 | ✅ 过 |
|
||||
| **(b) 盲测**(查询式 vs 全文档背诵,6任务×2臂) | 四判据:最终通过率 Q 不劣于 D(允许 −0 例);首轮语法错误率 Q ≤ D;平均修复轮次 Q ≤ D;token ≤ 70% of D(首轮契约诊断率为观测项) | deepseek-flash:docs 4/6 vs query 0/3(3 例端点 infra 剔除);首轮语法错 2/3 vs 2/6;token 69,771 vs 57,276。mimo-v2.6-pro:两跑均端点 infra(12/12、11/12),**不可评估** | ❌ 四判据未全立 ⇒ 按 §3.7 继续改协议 |
|
||||
| **(c) 契约消融**(四边界×三情形) | 带契约错值拦截 = 100%;带契约对值误报 = 0%;无契约路径零回归 | 拦截 4/4=**100%**;误报 0/4=**0%**;无契约 4/4 动态语义保持(含 `missing map key` 基线) | ✅ 过 |
|
||||
| **(d) 防漂移演练**(9 漂移 + 2 负对照) | 故意写错必报 = 100%;对照组误报 = 0% | 必报 9/9=**100%**(`unknown_type`×2、`param_drift`×2、`return_mismatch`、`arg_mismatch`、`unknown_key`、`contract.duplicate`、`contract.unknown_table`);负对照误报 0/2 | ✅ 过 |
|
||||
| **(e) 核心守卫演练**(3 种变异注入) | 单边改动必红;还原必绿 | 3/3 必红(`CORE_CONSTRUCTS` 私增、`core-surface.md` 私增、金样删除);还原全绿 | ✅ 过 |
|
||||
|
||||
**可复制结论(a/c/d/e 全过)**:锚点臂一轮修复率双模型 100%(判据 ≥80%,增量 +15pp/+10pp——易锚定样本近天花板,区分性留难样本扩展);契约系统"只在声明处拦截、零误报、不碰动态路径"成立;契约/实现互查防漂移 100% 必报;核心面单边膨胀被 CI 守卫机械阻断。
|
||||
|
||||
## 2. 实验 (c)(d)(e) 明细
|
||||
|
||||
### (c) 契约消融(tests/contract_ablation.rs,确定性)
|
||||
|
||||
四边界(调用实参 / 返回值 / 表绑定 / 键访问)各测三情形:
|
||||
|
||||
| 情形 | 口径 | 结果 |
|
||||
|---|---|---|
|
||||
| ① 带契约 + 错值 | 必须拦截(`contract.arg_mismatch` / `contract.return_mismatch` / `contract.field_mismatch` / `contract.unknown_key`) | 4/4 |
|
||||
| ② 带契约 + 对值 | 不得报契约错(同路径同形状) | 0/4 误报 |
|
||||
| ③ 无契约 + 错值 | 保持动态语义(不得出现 `contract.*`;走经典动态错误如 `missing map key`) | 4/4 |
|
||||
|
||||
取元一律用主推的 `[]()` 形(`[键](`集合`)`)验证;`集合[^键]` 过渡形保留兼容金样覆盖。
|
||||
|
||||
### (d) 防漂移演练(tests/drift_drill.rs,确定性)
|
||||
|
||||
9 个"故意写错"全部必报:类型词拼错、参数名漂移、升参未覆盖、返回越约、实参越约、未知键、重复契约表、错列表头契约表、运行时值越约。负对照 2 例(被绑定的数据表、纯动态程序)零误报。同时验证"契约表一经出现即为完整声明"(新升参未覆盖 ⇒ `contract.param_drift` warning)被按设计触发。
|
||||
|
||||
### (e) 核心守卫演练(人工变异,tests/core_surface.rs 必红验证)
|
||||
|
||||
| 变异 | 注入点 | 结果 |
|
||||
|---|---|---|
|
||||
| 私增 `ghost-construct` | `src/parse/mod.rs` `CORE_CONSTRUCTS` | 🔴 必红 |
|
||||
| 私增 `ghost-construct` 行 | `doc/design/core-surface.md` | 🔴 必红 |
|
||||
| 删除金样文件 | `tests/markup-v03/soft-emphasis.mq.md` | 🔴 必红 |
|
||||
|
||||
三次还原后 🟢 全绿。CI 守卫强制"清单与文档双改 + 全部金样存在"。
|
||||
|
||||
## 3. 实验 (b) 盲测:详况与诚实结论
|
||||
|
||||
### 协议
|
||||
|
||||
两臂系统提示词同文 = 硬规则 12 条 + 契约表三形状;**唯一变量 = 语法知识获取**:
|
||||
|
||||
- `query` 臂:MLSP 查询通道(`QUERY: <关键字>` ⇒ `mlsp syntax` 卡片,含戒律原文/样例/文档锚点;≤3 次)
|
||||
- `docs` 臂:完整 markup 参考背诵(`tests/blind/docs-bundle.md`,背诵式全语法表 + 中文内建面)
|
||||
|
||||
任务 6 例(`tests/blind/tasks/`):业务 3(保险理赔准备金 / 工程进度款 / 采购付款门槛,均出自律所实测任务集)+ QuixBugs 改写 3(gcd / 最大子段和 / 质因数分解);oracle = stdout 精确匹配(任务书自带期望输出,把推理难度控制掉,失败归因于**语言能力**)。修复回路 ≤2 次(validate 结构化诊断 + stderr 回喂)。
|
||||
|
||||
### deepseek-flash(完成)
|
||||
|
||||
| 臂 | 通过(一轮) | 通过(最终) | 首轮语法错 | 平均修复轮(过例) | token(p+c) | infra 剔除 |
|
||||
|---|---|---|---|---|---|---|
|
||||
| query | 0/3 | 0/3 | 2/3 | – | 69,771(42,478+27,293) | 3/6 |
|
||||
| docs | 3/6 | **4/6** | 2/6 | 0.50 | 57,276(33,346+23,930) | 0/6 |
|
||||
|
||||
- **docs 臂**:业务题 3/3 全部一轮通过(各 ~3k token);`qb_gcd` 修复 2 轮过;`qb_max_sublist_sum`、`qb_get_factors` 暴露真语法滑倒(列表字面量被当调用 `unknown function [-2,`;参数名漂移 `missing argument for parameter value`)。
|
||||
- **query 臂**:查询通道**被积极使用**(5/6 例用满 3 次,多词查询如 `条件分支 写法` 也能正确命中卡片——已单独验证查询匹配无缺陷);但 3 例被端点空体吃掉,其余 3 例败于真语法错(Python 式元组 `(a, b)` ⇒ `trailing input in expression`;Python 式调用 `gcd(48, 18)` ⇒ `unknown function gcd(48`)与 1 例查询后未交程序。
|
||||
|
||||
### mimo-v2.6-pro(两跑均不可评估)
|
||||
|
||||
首跑 **12/12** 全部"非 JSON / 空体"(与 deepseek 并行时段)。三向探针(小提示词×12000/4000、大提示词×12000)证实**网关随机空体**、与 `max_tokens`/提示词体量无稳定关系。加重试(8→12 次、退避 10→15 秒)并**单独**重跑后仍为 **11/12** 端点 infra(含两臂首聊即 `tok=(0,0)` 的空体);唯一执行到的 `qb_gcd` 例暴露真语法错(`**y**` 作语句 ⇒ `unexpected expression input`)。**结论:`mimo-v2.6-pro` 经本网关不可评估**,(b) 判据以 `deepseek-flash` 数据为准。原始报告:[blind-experiment-2026-09-23-mimo-v2.6-pro.md](blind-experiment-2026-09-23-mimo-v2.6-pro.md)。
|
||||
|
||||
### 判据对照(截至 deepseek-flash)
|
||||
|
||||
| 判据(协议 §2.4 原文口径) | 目标 | 实测(deepseek-flash,非 infra 口径) | 判定 |
|
||||
|---|---|---|---|
|
||||
| 最终通过率(≤3 轮)Q 不劣于 D(允许 −0 例) | ≥ D | 0/3 vs 4/6 | ❌ |
|
||||
| 首轮语法错误率 Q ≤ D | ≤ | 2/3 vs 2/6 | ❌ |
|
||||
| 平均修复轮次 Q ≤ D | ≤ | query 臂无通过例,不可比 | ⚠️ |
|
||||
| token 用量 Q ≤ 70% of D | ≤ 40,093 | 69,771(D 的 121.8%) | ❌ |
|
||||
| (观测项)首轮契约诊断率 Q ≤ D | – | 1/3 vs 0/6 | ➖ 观测 |
|
||||
|
||||
**诚实结论:在 deepseek-flash 上,"查询式"没有跑赢"全背诵"**——三项可判量化判据未达,按协议 §3.7 判定:**四判据未全成立 ⇒「AI 不必学语法」本轮未获实证,继续改协议**(不是证伪——infra 吞掉 query 臂一半样本,非 infra n=3)。要点:
|
||||
|
||||
1. **统计力被端点削弱**:query 臂 6 例中 3 例被空体吃掉(协议无从执行),非 infra 样本 n=3;mimo-v2.6-pro 整组不可用。判据是"有方向的强声明",当前证据**不足以支持也未证伪**其普适性,只能定格为"deepseek-flash × 本任务集 × 本协议版本"下的一次未达标观测。
|
||||
2. **失败模式是真发现**:查询臂的败例是**跨语言习惯泄漏**(元组表达式、Python 调用形)——查询卡片能回答"条件分支怎么写",但模型不查询"这里能不能用元组"。说明"AI 不必学语法"的瓶颈不在**查询机制**,而在**模型的自查纪律**(不知道自己不知道)。
|
||||
3. **token 判据方向存疑**:查询式省的是**系统提示词常驻**(背诵包),花的是**多轮往返**。6 例短程序任务里往返开销 > 常驻节省;该判据更适用于**长会话多文件**场景(一次背诵包 × N 轮 vs 每轮均摊查询),本轮任务形态低估了查询式的收益面。
|
||||
4. **docs 臂的 4/6 说明任务集有区分度**(不是全过/全不过的退化测量)。
|
||||
|
||||
## 4. 测量端基础设施记录(方法学附注)
|
||||
|
||||
- **LLM 端点不稳定**是本轮最大的干扰源:随机空体、reasoning 模型 `finish=length` 截断(思考吃满额度、正文为空)。对策:空体重试 5→12 次 + 退避 5→15 秒、截断自适应(`max_tokens` 加倍重试,封顶 32k)、infra 例从判据比率中剔除并单列。
|
||||
- **harness 缺陷两处**(已修,未污染最终数据):查询额度用满后的续篇被当最终回复(⇒ 假"无程序");程序提取失败不进修复回路(⇒ 少给模型一次标准修复机会)。修复后的协议对两臂完全一致。
|
||||
- **模型口径变化留痕**:原定 `cflmy`/`mimo-v2.6-flash` 组合按用户指示弃用,改 `mimo-v2.6-pro`/`deepseek-flash`;修复实验 (a) 的双模型数据仍是原组合(金 20 协议已完成,不重跑)。
|
||||
|
||||
## 5. 材料与复现
|
||||
|
||||
```bash
|
||||
# (a) 锚点消融(需要 API key)
|
||||
cargo test --test repair_experiment -- --ignored --nocapture # anchor/plain 双臂×20例
|
||||
# (b) 盲测(每模型一次;建议单独跑避免并发)
|
||||
OPENAI_MODEL=deepseek-flash MARQDO_EXP_DATE=2026-09-23 cargo test --test blind_experiment -- --ignored --nocapture
|
||||
# (c)(d) 确定性
|
||||
cargo test --test contract_ablation --test drift_drill
|
||||
# (e) 核心守卫(变异注入→必红→还原必绿,按 perf-validation.md §5 手法)
|
||||
cargo test --test core_surface
|
||||
```
|
||||
|
||||
- 任务书 / 背诵包(冻结,勿改):`tests/blind/`
|
||||
- 原始报告:[blind-experiment-2026-09-23-deepseek-flash.md](blind-experiment-2026-09-23-deepseek-flash.md) · [blind-experiment-2026-09-23-mimo-v2.6-pro.md](blind-experiment-2026-09-23-mimo-v2.6-pro.md)(首跑全 infra)· [repair-experiment-2026-09-22.md](repair-experiment-2026-09-22.md) · [repair-experiment-2026-09-22-mimo-v2.6-flash.md](repair-experiment-2026-09-22-mimo-v2.6-flash.md)
|
||||
- 判据协议:[perf-validation.md](perf-validation.md)(版本化于本仓库,实验前冻结)
|
||||
@@ -0,0 +1,125 @@
|
||||
# P4 性能验证设计(实验协议与判据)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Accepted · P4 执行依据** |
|
||||
| 日期 | 2026-09-23 |
|
||||
| 授权 | 用户授权**自行设计**本性能验证设计(P4 不再等待外部文档导入;`doc/next/002.md` 与性能验证无关) |
|
||||
| 上位判据 | [three-problems.md](three-problems.md) §5.1 判据矩阵 + §5.2 实验 (a)–(e);「判据矩阵全绿 + 实验数字可复现之前,三大问题一律视为未解决」 |
|
||||
| 工程任务 | [three-problems-plan.md](three-problems-plan.md) Phase 1–3(已完成);本文只负责**验证它们** |
|
||||
|
||||
> **本文定位**:把 three-problems.md §5.2 的五个实验操作化——每个实验给出**材料、协议、指标、判据**,全部机器可执行、数字可复现。实验产物 = 报告 [perf-validation-report-2026-09-23.md](perf-validation-report-2026-09-23.md)。**只认行为与数字,不认宣言。**
|
||||
|
||||
## 0. 总览
|
||||
|
||||
| 实验 | 验证对象 | 判据(全过 = 该问题的实证成立) | 执行方式 | 状态 |
|
||||
|------|----------|-------------------------------|----------|------|
|
||||
| (a) 锚点修复消融 | 问题三 · 可靠反馈 | 一轮修复率 ≥ 80%(anchor 臂),护栏 0 越界 | 模型在环 `tests/repair_experiment.rs` | ✅ 已完成(三轮双模型) |
|
||||
| (b) 盲测 | 问题三 · 不必学语法 | 查询组错误率 ≤ 全文档组;token 显著更少;修复轮次不升 | 模型在环 `tests/blind_experiment.rs` | 见报告 |
|
||||
| (c) 契约消融 | 问题二 · 语义校验 | 拦截率 = 100%;误报率 = 0%;无契约路径零回归 | 确定性 `tests/contract_ablation.rs` | 见报告 |
|
||||
| (d) 防漂移演练 | 问题二 · 契约可信 | 故意写错契约必报率 = 100% | 确定性 `tests/drift_drill.rs` | 见报告 |
|
||||
| (e) 核心守卫演练 | 问题一 · 核心要小 | 三种变异注入各必红 → 还原必绿 | 变异脚本(见 §4) | 见报告 |
|
||||
|
||||
判定口径([three-problems.md](three-problems.md) §5.1):**任何一项判据不成立,对应问题即「未解决」**——不许用文档宣言充数。
|
||||
|
||||
---
|
||||
|
||||
## 1. 实验 (a):锚点修复消融(问题三 · 可靠反馈)
|
||||
|
||||
**假设**:诊断带 `doc_anchor`/`doc_quote`/`suggestion`(锚点诊断)比裸诊断显著提高一轮修复率。
|
||||
|
||||
- **对照**:`anchor` 臂(全量诊断字段)vs `plain` 臂(剥掉 doc_anchor/doc_quote/suggestion)。
|
||||
- **材料**:golden 20 例(10 骨架 × 2 变体:arg/ret/field/key/typo/drift/dup),易锚定错误。
|
||||
- **协议**:`validate` → `repair_targets` → 模型行编辑 → `repair_apply`(越界必拒/abstain)→ 复验;≤2 轮。
|
||||
- **指标**:一轮修复率、最终修复率、平均尝试、越界拒、弃权。
|
||||
- **判据**:anchor 臂一轮修复率 ≥ 80%;护栏 0 次越界应用(越界必拒由 `tests/repair_loop.rs` 硬断言兜底)。
|
||||
- **执行**:`OPENAI_MODEL=… cargo test --test repair_experiment -- --ignored --nocapture`。
|
||||
- **结果**:三轮双模型全过(cflmy anchor 100% / mimo 90%→100%,plain 85%–100%,护栏 0 越界)——
|
||||
详见 [repair-experiment-2026-09-22.md](repair-experiment-2026-09-22.md) 与
|
||||
[repair-experiment-2026-09-22-mimo-v2.6-flash.md](repair-experiment-2026-09-22-mimo-v2.6-flash.md)。**✅ PASS**。
|
||||
- **诚实注记**:样本为易锚定错误(两臂近天花板),锚点增量的区分性证据留待难样本扩展;不影响本判据达标。
|
||||
|
||||
## 2. 实验 (b):盲测——查询式 vs 全文档背诵(问题三 · 超级核心)
|
||||
|
||||
**假设**:「零语法文档 + MLSP 查询」组(Q)成绩**不劣于**「全文档背诵」组(D),且 prompt token 显著更少、修复轮次不升([three-problems.md](three-problems.md) §3.7 最终判据)。
|
||||
|
||||
### 2.1 对照(唯一变量 = 语法知识的获取方式)
|
||||
|
||||
| 臂 | 系统提示词内容 | 语法知识来源 |
|
||||
|----|----------------|--------------|
|
||||
| **Q 查询组** | 硬规则(12 条)+ 契约表三形状 + MLSP 查询通道说明 | 按需 `QUERY: <关键字>`(≤3 次,harness 代答 `mlsp syntax` 卡片) |
|
||||
| **D 全文档组** | 硬规则(12 条)+ 契约表三形状 + **完整 markup 参考**(`tests/blind/docs-bundle.md`:v0.3 全语法表 + 中文内建面) | 上下文预载(背诵) |
|
||||
|
||||
两臂共同点:硬规则与契约表形状**同文**(受控变量,不测契约知识差异);修复回路相同(`mlsp validate` 诊断回喂,≤2 轮);模型 / 温度 / 任务 / 提示词模板相同。
|
||||
|
||||
### 2.2 材料:任务集(6 例,业务 3 + 算法 3)
|
||||
|
||||
| id | 类型 | 任务 | 功能 oracle(stdout 精确匹配) |
|
||||
|----|------|------|-------------------------------|
|
||||
| `claims_reserve_settle` | 业务 | 保险准备金(免赔/共保/分项限额/追偿/地板) | `28000` |
|
||||
| `contract_milestone_pay` | 业务 | 工程里程碑应付款(保留金/逾期罚/变更/税/地板) | `37354` |
|
||||
| `vendor_po_gate` | 业务 | 采购 PO 准付闸(折扣/加急/预算帽/合规费/地板) | `814` |
|
||||
| `qb_gcd` | 算法 | 最大公约数;打印 gcd(48,18) gcd(1071,462) gcd(0,5) gcd(17,5) 各一行 | `6` `21` `5` `1` |
|
||||
| `qb_max_sublist_sum` | 算法 | 最大子段和;三组输入各打印一行 | `6` `6` `-1` |
|
||||
| `qb_get_factors` | 算法 | 质因数分解,因数以 `*` 连接;12 / 13 / 60 各一行 | `2*2*3` `13` `2*2*3*5` |
|
||||
|
||||
业务任务书复用 MarqdoThesis `formal/exp3-bizreq/data/tasks/`(权威规则文本原样进入提示词);算法任务书为 QuixBugs 改写(从「修 bug」改「实现 + 固定输出」,消除参考源码变量)。
|
||||
|
||||
### 2.3 协议(每例)
|
||||
|
||||
1. 提示词 = 任务书 + 「输出完整 `.mq.md` 源码于单个 ```markdown 围栏」;
|
||||
2. **查询阶段**(仅 Q 臂):模型可输出 `QUERY: <关键字>` 行,harness 以 `mlsp syntax` 卡片作答并继续;累计 ≤3 次;
|
||||
3. **执行阶段**:`marqdo run` → stdout == oracle 即通过;失败则回喂 `mlsp validate` 结构化诊断 + stderr,允许 ≤2 次修复;
|
||||
4. 记录每轮 token(API `usage.prompt_tokens + completion_tokens` 累计)。
|
||||
|
||||
### 2.4 指标与判据
|
||||
|
||||
| 指标 | 口径 | 判据 |
|
||||
|------|------|------|
|
||||
| 功能通过率(一轮 / 最终≤3 轮) | stdout 与 oracle 精确匹配 | Q 最终通过率**不劣于** D(允许 −0 例) |
|
||||
| 首轮语法错误率 | 首轮 `validate`/run 出现 parse 类失败的例数占比 | Q ≤ D |
|
||||
| 首轮契约诊断率 | 首轮出现 `contract.*` 诊断的例数占比 | Q ≤ D(观测项) |
|
||||
| 平均修复轮次 | 通过前的修复次数(含首轮生成记 0) | Q ≤ D |
|
||||
| token 用量 | 逐例累计 prompt+completion | Q **显著更少**(≤ 70% of D) |
|
||||
|
||||
**判定**:四条判据全成立 ⇒ 「AI 不必学语法」实证成立;否则按 §3.7「继续改协议」,不得以改进文档充数。
|
||||
- **执行**:`OPENAI_MODEL=… cargo test --test blind_experiment -- --ignored --nocapture`(每模型一轮)。
|
||||
|
||||
## 3. 实验 (c):契约消融(问题二 · 语义校验)
|
||||
|
||||
**假设**:契约只在**有契约处**拦截错误值(P1:无契约 = 全动态),且不误报。
|
||||
|
||||
- **材料**:`tests/contract_ablation.rs` 内联四边界用例(调用实参 / 返回 / 表绑定 / 字段访问)×三形态:
|
||||
① 带契约 + 错值 → **必拦**(`contract.*` 诊断);② 带契约 + 对值 → **零误报**(正常产出);③ 同程序**删契约表** + 错值 → **零契约诊断**(动态语义原样)。
|
||||
- **指标与判据**:
|
||||
- 拦截率 = ①被拦 / ①总数 = **100%**;
|
||||
- 误报率 = ②误报 / ②总数 = **0%**;
|
||||
- 无契约零回归 = ③零 `contract.*` 诊断且输出 = 动态语义基线;且全量 `cargo test` 绿(441+ 既有测试是无契约路径的主回归面)。
|
||||
- **执行**:`cargo test --test contract_ablation`。
|
||||
|
||||
## 4. 实验 (d):防漂移演练(问题二 · 契约可信)
|
||||
|
||||
**假设**:契约写错**必**被机器抓(不静默信任契约)——three-problems.md §2.8「故意写错契约的金样必红」。
|
||||
|
||||
- **材料**:`tests/drift_drill.rs`——故意写错契约的 N 例(类型 typo `nubmer`、未知对象类型、契约参数名 ≠ 升参名 ×2、返回契约明显不符、重复契约表、错位契约表)+ **负对照**(被绑定的 `字段|类型` 数据表 = web_db_migration 式,必须**不**被当契约吃掉)。
|
||||
- **指标与判据**:
|
||||
- 必报率 = 漂移例被诊断(`check` 报错/警告或运行时 `contract.*`)/ 漂移例总数 = **100%**;
|
||||
- 负对照误报 = **0**。
|
||||
- **执行**:`cargo test --test drift_drill`。
|
||||
|
||||
## 5. 实验 (e):核心守卫演练(问题一 · 核心要小)
|
||||
|
||||
**假设**:核心表面任何**单侧**变更(代码/文档/金样不同步)→ CI 必红——three-problems.md §1.5「新增标记不更新清单时 CI 必须红」。
|
||||
|
||||
- **材料**:`tests/core_surface.rs` 守卫 + 三种变异注入(脚本化,`git checkout` 还原):
|
||||
1. `src/parse/mod.rs` `CORE_CONSTRUCTS` 增一行、不改文档 → **必红**;
|
||||
2. `doc/design/core-surface.md` 表增一行、不改代码 → **必红**;
|
||||
3. 金样文件改名(路径失效)→ **必红**。
|
||||
- **指标与判据**:3/3 变异注入后守卫测试红;还原后全绿。**必红率 = 100%**。
|
||||
- **执行**:报告内嵌演练记录(命令 + 测试输出摘录)。
|
||||
|
||||
## 6. 报告与复现
|
||||
|
||||
- 报告:[perf-validation-report-2026-09-23.md](perf-validation-report-2026-09-23.md)((a) 引用既有两份实验报告;(b)–(e) 本次产出)。
|
||||
- 复现命令全部写入报告各节;模型在环实验记录模型名与端点,判据数字允许 ±1 例波动(模型非确定性),**判据线本身不放宽**。
|
||||
- 汇总结论回填 [three-problems-plan.md](three-problems-plan.md) 落地记录;§5.1 判据矩阵逐行标注 ✅/❌。
|
||||
@@ -0,0 +1,64 @@
|
||||
# 修复实验(§5.2(a) 锚点消融)—— 2026-09-22
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 模型 | `mimo-v2.6-flash` @ `https://llm.cflmy.cn/v1` |
|
||||
| 样本 | golden 20 例(易锚定错误) |
|
||||
| 协议 | `validate` → `repair_targets` → 模型行编辑 → `repair_apply`(越界必拒/abstain)→ 复验;≤2 轮 |
|
||||
| 对照 | `anchor` = 诊断带 `doc_anchor`/`doc_quote`/`suggestion`;`plain` = 全部剥掉 |
|
||||
|
||||
## 汇总(设计目标:一轮修复率 ≥ 80%)
|
||||
|
||||
| 臂 | 一轮修复率 | 最终修复率(≤2轮) | 平均尝试 | 越界拒(护栏) | 弃权 |
|
||||
|---|---|---|---|---|---|
|
||||
| anchor | 20/20(100%) | 20/20(100%) | 1.00 | 0 | 0 |
|
||||
| plain | 18/20(90%) | 19/20(95%) | 1.05 | 0 | 1 |
|
||||
|
||||
## 逐例
|
||||
|
||||
| 例 | 臂 | 结果 | 轮次 | 越界拒 | 备注 |
|
||||
|---|---|---|---|---|---|
|
||||
| arg-num-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| arg-num-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| arg-num-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| arg-num-b | plain | 一轮修复 | 1 | 0 | |
|
||||
| arg-text-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| arg-text-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| arg-text-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| arg-text-b | plain | 一轮修复 | 1 | 0 | |
|
||||
| drift-cover-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| drift-cover-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| drift-cover-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| drift-cover-b | plain | 一轮修复 | 1 | 0 | |
|
||||
| drift-rename-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| drift-rename-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| drift-rename-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| drift-rename-b | plain | 一轮修复 | 1 | 0 | |
|
||||
| dup-delete-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| dup-delete-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| dup-delete-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| dup-delete-b | plain | 一轮修复 | 1 | 0 | |
|
||||
| field-nullable-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| field-nullable-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| field-nullable-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| field-nullable-b | plain | 一轮修复 | 1 | 0 | |
|
||||
| field-type-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| field-type-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| field-type-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| field-type-b | plain | 一轮修复 | 1 | 0 | |
|
||||
| key-fix-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| key-fix-a | plain | 未修复 | 1 | 0 | 模型弃权/输出不可解析: [{"op":"replace","line":13","text":"*`用户`[^name]*"}] |
|
||||
| key-fix-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| key-fix-b | plain | 二轮修复 | 2 | 0 | 修复后仍有诊断: {"code":"marqdo.error","contract_ref":null,"doc_anchor":null,"doc_quote":null,"message":"missing map key `phone`","severity":"error","span":{"col":1,"file":"/tmp/marqdo_repair_1666049_plain_key-fix-b.mq.md","line":14},"suggestion":null} |
|
||||
| ret-type-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| ret-type-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| ret-type-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| ret-type-b | plain | 一轮修复 | 1 | 0 | |
|
||||
| type-typo-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| type-typo-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| type-typo-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| type-typo-b | plain | 一轮修复 | 1 | 0 | |
|
||||
|
||||
## 护栏核对
|
||||
|
||||
- 每次越界编辑都以 `mlsp.repair_out_of_scope` + `abstain` 被拒,且 `repair_apply` 全有或全无(源零字节不改)——由 `tests/repair_loop.rs` 的确定性断言覆盖;本实验计数仅作记录。
|
||||
@@ -0,0 +1,64 @@
|
||||
# 修复实验(§5.2(a) 锚点消融)—— 2026-09-22
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 模型 | `cflmy` @ `https://llm.cflmy.cn/v1` |
|
||||
| 样本 | golden 20 例(易锚定错误) |
|
||||
| 协议 | `validate` → `repair_targets` → 模型行编辑 → `repair_apply`(越界必拒/abstain)→ 复验;≤2 轮 |
|
||||
| 对照 | `anchor` = 诊断带 `doc_anchor`/`doc_quote`/`suggestion`;`plain` = 全部剥掉 |
|
||||
|
||||
## 汇总(设计目标:一轮修复率 ≥ 80%)
|
||||
|
||||
| 臂 | 一轮修复率 | 最终修复率(≤2轮) | 平均尝试 | 越界拒(护栏) | 弃权 |
|
||||
|---|---|---|---|---|---|
|
||||
| anchor | 20/20(100%) | 20/20(100%) | 1.00 | 0 | 0 |
|
||||
| plain | 17/20(85%) | 17/20(85%) | 0.85 | 0 | 0 |
|
||||
|
||||
## 逐例
|
||||
|
||||
| 例 | 臂 | 结果 | 轮次 | 越界拒 | 备注 |
|
||||
|---|---|---|---|---|---|
|
||||
| arg-num-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| arg-num-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| arg-num-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| arg-num-b | plain | 一轮修复 | 1 | 0 | |
|
||||
| arg-text-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| arg-text-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| arg-text-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| arg-text-b | plain | 一轮修复 | 1 | 0 | |
|
||||
| drift-cover-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| drift-cover-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| drift-cover-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| drift-cover-b | plain | 一轮修复 | 1 | 0 | |
|
||||
| drift-rename-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| drift-rename-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| drift-rename-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| drift-rename-b | plain | 一轮修复 | 1 | 0 | |
|
||||
| dup-delete-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| dup-delete-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| dup-delete-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| dup-delete-b | plain | 一轮修复 | 1 | 0 | |
|
||||
| field-nullable-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| field-nullable-a | plain | 未修复 | 0 | 0 | LLM 调用失败(重试后): 非 JSON / 空体: |
|
||||
| field-nullable-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| field-nullable-b | plain | 未修复 | 0 | 0 | LLM 调用失败(重试后): 非 JSON / 空体: |
|
||||
| field-type-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| field-type-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| field-type-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| field-type-b | plain | 一轮修复 | 1 | 0 | |
|
||||
| key-fix-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| key-fix-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| key-fix-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| key-fix-b | plain | 未修复 | 0 | 0 | LLM 调用失败(重试后): 非 JSON / 空体: |
|
||||
| ret-type-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| ret-type-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| ret-type-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| ret-type-b | plain | 一轮修复 | 1 | 0 | |
|
||||
| type-typo-a | anchor | 一轮修复 | 1 | 0 | |
|
||||
| type-typo-a | plain | 一轮修复 | 1 | 0 | |
|
||||
| type-typo-b | anchor | 一轮修复 | 1 | 0 | |
|
||||
| type-typo-b | plain | 一轮修复 | 1 | 0 | |
|
||||
|
||||
## 护栏核对
|
||||
|
||||
- 每次越界编辑都以 `mlsp.repair_out_of_scope` + `abstain` 被拒,且 `repair_apply` 全有或全无(源零字节不改)——由 `tests/repair_loop.rs` 的确定性断言覆盖;本实验计数仅作记录。
|
||||
@@ -0,0 +1,225 @@
|
||||
# 三大问题落地计划(core 收口 · 渐进式契约 · MLSP)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **本期开发依据 · 执行中** |
|
||||
| 日期 | 2026-09-22 |
|
||||
| 设计 | [three-problems.md](three-problems.md)(解决方向与验证设计;判据矩阵 §5.1) |
|
||||
| 范围 | **工程任务 = Phase 1–3**;验证实验(three-problems.md §5.2)协议见 [perf-validation.md](perf-validation.md)(2026-09-23 用户授权自行设计并执行) |
|
||||
| 约定 | 每任务带机器可验证验收;**任务完成 = 验收全绿 + 全量回归绿**;只改宣言不改行为 = 未完成 |
|
||||
|
||||
## 0. 本期范围与依赖
|
||||
|
||||
| 阶段 | 定位 | 本期 |
|
||||
|------|------|------|
|
||||
| **Phase 1** 核心收口 + 诊断地基 | 一切的地基 | ✅ 启动 |
|
||||
| **Phase 2** 渐进式契约层 | 校验层(schema 源头 = 表格/段落) | ✅ 本期主目标 |
|
||||
| **Phase 3** MLSP(AI 语言服务) | **下一步的超级核心**(AI 不必学语法 + 可靠反馈闭环) | ⏭ 下期启动,本期只留接口 |
|
||||
| **Phase 4** 验证实验 | 性能验证(非开发任务) | ❌ 不认领 |
|
||||
|
||||
依赖链(**跳步会倒逼「另造一套错误 / 另写一套 schema」的遮盖式捷径**):
|
||||
|
||||
```text
|
||||
T1.4 Diagnostic ──是──> T2.x 契约诊断的唯一出口
|
||||
T2.x 契约 ───────是──> T3.1 schema 注册表的主数据源
|
||||
T1.1 core-surface ─是─> T3.1 语言面 + T1.3 守卫
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 · 核心收口 + 诊断地基(本期启动)
|
||||
|
||||
### T1.1 核心表面清单
|
||||
|
||||
- **产出**:`doc/design/core-surface.md`
|
||||
- **实现要点**:
|
||||
- 两张表:**行构造**(heading-unit `#`/`##`、call-line `>`、branch-list `1.`、loop-list `-`、param `+ 名`、table、fence、hr、writeback、narrative、blank)与**内联标记**(ident `` `名` ``、bold-code `**…**`、italic-return `*…*`、empty-return、link-index `[键](集合)`、bracket-call `[函数]`、footnote-index `[^键]` 过渡);
|
||||
- 每行含:**id**(英文 kebab,守卫用)、Markdown 外形、语义、**戒律**(边界 + 为什么不能是普通 Markdown)、**金样**(`tests/markup-v03/…` 路径);
|
||||
- 与 [markdown-mapping-v0.3.md](../design/markdown-mapping-v0.3.md) §5 / §10 对齐;词法大类(`src/lex/mod.rs` `LineKind`)另列一节说明与构造表的关系。
|
||||
- **验收**:每行金样文件真实存在(T1.3 断言);id 全集 = T1.3 `CORE_CONSTRUCTS` 全集。
|
||||
|
||||
### T1.2 三层归属与准入戒律
|
||||
|
||||
- **产出**:`doc/design/layers.md` + `.github/PULL_REQUEST_TEMPLATE.md`(无则新建)
|
||||
- **实现要点**:
|
||||
- 归属表(含仓库路径):Language = `src/lex` `src/parse` `src/ast` `src/interp` `src/diagnostics` + 契约检查器;Runtime = `lib/` `plugins/` `crates/` `src/catalog.rs` `src/view` `src/debug` + 未来的 check/schema/mcp;Document = `*.mq.md` `skills/` `ext/` `public/` `examples/`;
|
||||
- 准入戒律:新特性先填层归属;进 Language 必答「为什么现有标记不够」;默认进 ext;核心标记计数每 release 只许持平或下降;
|
||||
- 发布 checklist:CHANGELOG 增「核心表面 Δ」栏(写入 `skills/marqdo-release/SKILL.md` 一步)。
|
||||
- **验收**:PR 模板含「层归属」必填项;layers.md 与 core-surface.md 互相引用闭合。
|
||||
|
||||
### T1.3 核心表面守卫(CI)
|
||||
|
||||
- **产出**:`tests/core_surface.rs` + `src/parse/mod.rs` 增 `pub const CORE_CONSTRUCTS: &[(&str, &str)]`(id、外形;**紧邻分类代码维护**)
|
||||
- **实现要点**:`include_str!` 解析 core-surface.md 表格 id 列 ↔ `CORE_CONSTRUCTS` **双向集合相等**;每行「金样」路径 `Path::exists()` 断言。
|
||||
- **验收(反遮盖核心)**:
|
||||
- 新增 `CORE_CONSTRUCTS` 条目不改文档 → **红**;
|
||||
- 文档加行不改代码 → **红**;
|
||||
- 删除/改名金样文件 → **红**。
|
||||
|
||||
### T1.4 诊断对象扩展(Phase 2/3 的唯一出口)
|
||||
|
||||
- **产出**:`src/diagnostics.rs` 扩展 + `src/main.rs` `run --json`
|
||||
- **实现要点**:
|
||||
- `Diagnostic` 增 `code`(如 `parse.*` / `contract.*` / `runtime.*`)、`severity`、`suggestion`、`doc_anchor`、`doc_quote`、`contract_ref`(Phase 2 起填值);`::new/at` 签名不变(全仓调用零改动),新增 builder 方法;
|
||||
- `serde::Serialize` + `to_json()`;**Display 保持 `path:line:col: message`**(人类输出零回归);
|
||||
- `run --json`:顶层错误路径把 Diagnostic 序列化输出(stderr JSON 行)。
|
||||
- **验收**:现有错误金样文本输出**零变化**;新金样断言 JSON `code`/`span` 非空;`cargo test` 全绿。
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 · 渐进式契约层(本期主目标)
|
||||
|
||||
### T2.1 契约 AST + 提取器
|
||||
|
||||
- **产出**:`src/contract.rs`(`Contract` / `ParamContract` / `FieldContract` + 类型词汇 `text/number/bool/list/map/any/对象名`);`src/parse/`(提取);`src/ast.rs`(挂载,带 span)
|
||||
- **实现要点**:按 [three-problems.md](three-problems.md) §2.3/§2.4——**未绑定文档表** + 列头关键字(`参数/返回/字段/类型/说明`)+ 允许位置(函数体首个可执行行前 / 绑定前 / 对象体首成员前);**被绑定的表永不视为契约表**。
|
||||
- **验收**:三附着面金样 + 「绑定表不吃」反例金样(web_db_migration 式 `字段|类型|可空` 绑定数据表必须原样是数据);未知类型名(`nubmer`)必产诊断。
|
||||
|
||||
### T2.2 四边界运行时校验
|
||||
|
||||
- **产出**:`src/interp.rs` 接入四边界——调用实参 / 字段访问(`[键](集合)`、`.field`)/ 表绑定 / `*…*` 返回
|
||||
- **实现要点**:只在**该边界有契约**时校验(无契约 = 全动态,P1);诊断一律走 T1.4 出口(`code=contract.*`)。
|
||||
- **验收**:有契约 + 错值 → 必产诊断;**无契约路径全量回归零变化**;three-problems.md §5.2(c) 消融材料可产。
|
||||
|
||||
### T2.3 `marqdo check` + `--json`
|
||||
|
||||
- **产出**:`src/check.rs` + `src/main.rs` `Commands::Check`
|
||||
- **实现要点**:静态**按需**校验(只查有契约处);不做全局推导。
|
||||
- **验收**:契约样例工程 `check` 退出码非 0 且 JSON 合协议;干净工程 0 退出零输出。
|
||||
|
||||
### T2.4 锚点诊断
|
||||
|
||||
- **产出**:契约诊断填 `doc_anchor`(契约表行段 `file#Lx-Ly`)+ `doc_quote`(表行原文)
|
||||
- **验收**:金样断言锚点非空且行号回读命中契约表;three-problems.md §5.2(a) 材料就绪。
|
||||
|
||||
### T2.5 契约 vs 实现互查(防漂移)
|
||||
|
||||
- **产出**:check + 运行时提示——契约参数名 ≠ 升参名 / 未知类型 / 返回契约明显不符 → 诊断
|
||||
- **验收**:故意写错契约的金样**必红**(three-problems.md §5.2(d) 材料就绪)。
|
||||
|
||||
### T2.6 运行时内省 `get_schema`(可延后,P3)
|
||||
|
||||
- **产出**:`src/builtin.rs` 内省 builtin(契约是数据,P3)
|
||||
- **验收**:demo 运行中取回自身函数契约并打印。
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 · MLSP(下一步的超级核心;本期只留接口)
|
||||
|
||||
### T3.1 `marqdo schema`(schema 注册表,三来源零手写)
|
||||
|
||||
- **产出**:`src/schema.rs` + `Commands::Schema`——语言面(core-surface 机械转 JSON)+ 函数/类型面(**契约** + stdlib/ext 签名)+ 项目面(`catalog.yaml`)
|
||||
- **验收**:两次生成 byte-identical;stdlib 主函数覆盖率 ≥ 90%。
|
||||
|
||||
### T3.2 `marqdo mcp`(AI 工具面)
|
||||
|
||||
- **产出**:MCP server(参考 `plugins/agent/src/mcp_server.rs`)——`syntax_list` / `syntax_get` / `fn_schema` / `type_schema` / `validate` / `repair` / `run`
|
||||
- **验收**:MCP 客户端冒烟:`fn_schema` → `validate` → `repair` 全链路。
|
||||
|
||||
### T3.3 查询式 AI Skill
|
||||
|
||||
- **产出**:`skills/marqdo/` 改写——十条硬规则 + 查询工作流(construct → validate → 修 → run);细节一律走 `syntax_get` / `fn_schema`
|
||||
- **验收**:skill 体积减半以上;金样全绿。
|
||||
|
||||
### T3.4 修复回路
|
||||
|
||||
- **产出**:repair 规则库 + 语句级 patch(AST span 对齐;预算 N=3;防循环)
|
||||
- **验收**:注入 bug 金样一轮修复率可测(three-problems.md §5.2(a) 执行就绪)。
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 · 验证实验(**协议 = [perf-validation.md](perf-validation.md)**)
|
||||
|
||||
- 2026-09-23 用户授权**自行设计**性能验证设计并执行验证;`doc/next/002.md` 与性能验证无关(此前「待导入」表述已纠正)。
|
||||
- five 实验 (a)–(e) 的**材料由 Phase 1–3 的验收金样直接供给**;协议 / 指标 / 判据见 [perf-validation.md](perf-validation.md) §1–§5;
|
||||
实验产物 = [perf-validation-report-2026-09-23.md](perf-validation-report-2026-09-23.md)。
|
||||
|
||||
---
|
||||
|
||||
## 完成定义
|
||||
|
||||
| 阶段 | 完成定义 |
|
||||
|------|----------|
|
||||
| Phase 1 | T1.1–T1.4 验收全绿 |
|
||||
| Phase 2 | T2.1–T2.5 验收全绿(T2.6 可延后) |
|
||||
| Phase 3 | T3.1–T3.4 验收全绿 |
|
||||
| 三大问题「已解决」 | [three-problems.md](three-problems.md) §5.1 判据矩阵全绿 + 实验数字可复现(Phase 4) |
|
||||
|
||||
## 落地记录
|
||||
|
||||
### Phase 2 · 渐进式契约层(2026-09-22 完成 T2.1–T2.5;T2.6 按计划延后至 Phase 3)
|
||||
|
||||
- **T2.1** `src/contract.rs`(类型词汇/三形状契约表解析/合并/四边界校验诊断)+
|
||||
`src/parse/mod.rs::extract_function_contract`(位置 + 形状双重判定,**在升参推断之前**执行——
|
||||
契约表的单元不会被误升参;契约表从 body 移除,元数据永不执行)。
|
||||
反例守住:**绑定**的 `字段|类型` 数据表(web_db_migration 风格)绝不视为契约。
|
||||
- **T2.2** 四边界全接:调用实参(普通/库路径/方法三条调用路径)、返回、表绑定、字段访问
|
||||
(键存在性**先于**取值,契约诊断优先于 `missing map key`)。列语义按「几何即类型」:标量或全匹配列向量。
|
||||
- **T2.3/T2.5** `marqdo check [FILE] [--json]`:契约行 vs 升参推断形参双向互查
|
||||
(错行 ⇒ Error;新升参未覆盖 ⇒ Warning)、未知类型名(附最近建议)、错位契约;Error ⇒ 非零退出。
|
||||
- **T2.4** 全部契约诊断带 `doc_anchor`(`file.mq.md#L7-L13`)+ `doc_quote`(契约表原文)+
|
||||
`contract_ref`(`declared` = 契约声明摘要);`run --json` / `check --json` 同源输出。
|
||||
- **附带修复(错误出口唯一)**:`src/load.rs` 原来把 parse 错误 `anyhow!("{label}: {e}")` 拍平成字符串,
|
||||
结构化诊断在 import 装载链路丢失;改为 `label_error` 保留 Diagnostic(补文件标签),任何装载层都不许拍平错误。
|
||||
- 验收:`tests/contracts.rs` 12 例(提取 4 + 四边界 4 + check 3 + 正向对照 1)全绿;
|
||||
全量回归零失败(含 262 gold)。
|
||||
- **T2.6(契约导出 `get_schema`)**:按计划延后至 Phase 3 与 MLSP `schema` 一并实现。
|
||||
|
||||
### Phase 3 · MLSP(2026-09-22 完成 T3.1–T3.4 全部工程项 + T2.6)
|
||||
|
||||
- **T3.1** `marqdo mlsp` 子命令:行分隔 JSON over stdio,请求 `{"id","method","params"}` →
|
||||
响应 `{"id","ok","result"|"error"}`(永不 panic,错误也是结构化 JSON)。
|
||||
- **T3.2** 五个 handler 全部实现(`src/mlsp.rs`):
|
||||
`locate`(构造卡片 / 符号定位 + 契约)、`syntax`(关键字/别名 → 卡片;语法唯一事实来源 =
|
||||
`parse::CORE_CONSTRUCTS`,MLSP 只加检索别名不定义语法)、`validate`(parse + 契约 check → 结构化诊断数组)、
|
||||
`repair_targets`(有界修复靶点:`strategy=anchored-local`、`on_violation=abstain`——越界必拒护栏先行)、
|
||||
`schema`(**T2.6 完成**:单元名 → 形参/返回/字段契约 + 升参清单 + 锚点)。
|
||||
- 验收:`tests/mlsp.rs` 7 例全绿(语法问答/未知查询/符号定位/漂移校验/schema 导出/有界修复/结构化错误);
|
||||
stdio 冒烟通过;全量回归零失败。
|
||||
- **T3.4(修复回路)**(2026-09-22 完成工程项)——`src/repair.rs` **机械护栏**(有界行编辑
|
||||
replace/delete/insert:触碰行必须落在靶点范围内,越界 ⇒ 整体拒绝 + 越界清单 + abstain,源零字节不改;
|
||||
表尾追加允许 end+1)+ `mlsp repair_apply` 出口(护栏的 MLSP 面)+ **golden 20 例**
|
||||
(10 骨架 × 2 变体:arg/ret/field/key/typo/drift/dup;金修复经完整修复循环复现——诊断 → 靶点 →
|
||||
行编辑自动推导 → 范围自检(「易锚定」的机器验证)→ 应用 → 复验)+ 护栏负例(越界必拒/部分越界全拒)。
|
||||
**模型实验完成(2026-09-22 · `tests/repair_experiment.rs` · 双模型三轮 @ llm.cflmy.cn)**:
|
||||
§5.2(a) 锚点消融——`anchor` 臂(诊断带 `doc_anchor`/`doc_quote`/`suggestion`)**一轮修复率最高 20/20 = 100%**(目标 ≥80% ✅);
|
||||
`plain` 臂(去锚)85%–100%。三轮汇总:
|
||||
|
||||
| 轮 | 模型 | anchor 一轮 | anchor 最终 | plain 一轮 | plain 最终 | 护栏越界 |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 1 | `cflmy`(Qwen3.8-Flash-Next) | **20/20(100%)** | 100% | 17/20(85%) | 17/20 | 0 |
|
||||
| 2 | `mimo-v2.6-flash`(容错加固中) | 18/20(90%) | 19/20(95%) | 19/20(95%) | 19/20 | 0 |
|
||||
| 3 | `mimo-v2.6-flash`(容错补齐后) | **20/20(100%)** | 100% | 18/20(90%) | 19/20(95%) | 0 |
|
||||
|
||||
**"≥80% 一轮修复率"三轮双臂全部达标(最低 85%)**;护栏 0 越界(越界必拒/abstain 由确定性测试
|
||||
`tests/repair_loop.rs` 硬断言覆盖)。实验同时是 `validate`→`repair_targets`→`repair_apply` 功能面的真实消费者。
|
||||
**"不稳定"定性为三类测量端问题,均已加固**(与 Marqdo 语义无关):① 网关空体抖动 → curl `--retry` + 5s 退避;
|
||||
② 推理模型 token 预算被 `reasoning_content` 吃光 → `max_tokens` 2000→8000;③ 模型输出格式畸形 → 解析容错
|
||||
(数组/单对象/NDJSON 多对象流/字符串化元素/字符串型行号/围栏/数字后多余引号的确定性修补)。
|
||||
锚点增量观察:易锚定样本两臂都高(天花板效应),cflmy 上 anchor +20pp 明显、mimo 上持平——
|
||||
**锚点价值的区分性验证留 Phase 4**(难样本/信息受限消融)。
|
||||
报告:[repair-experiment-2026-09-22.md](repair-experiment-2026-09-22.md)(cflmy)、
|
||||
[repair-experiment-2026-09-22-mimo-v2.6-flash.md](repair-experiment-2026-09-22-mimo-v2.6-flash.md)(mimo 终测)。
|
||||
- **T3.3(AI Skill 换骨:背语法 → 查 MLSP)**(2026-09-22 完成)——`marqdo` / `marqdo-dev` 两件套改为
|
||||
查询式工作流:写码前 `mlsp syntax/locate`、提交前 `mlsp validate` + `marqdo check`;
|
||||
渐进式契约三种表格形状入语法面;错误修复照 `doc_anchor` 局部改(越界必弃权)。
|
||||
|
||||
## Phase 4 · 验证执行记录(2026-09-23,协议 [perf-validation.md](perf-validation.md))
|
||||
|
||||
- **(a) 锚点修复消融** ✅ PASS——判据「anchor 一轮 ≥80% + 护栏 0 越界」:cflmy anchor 20/20=100%(plain 85%)、
|
||||
mimo-v2.6-flash anchor 20/20=100%(plain 90%),越界拒/越界应用 0/0。易锚定样本近天花板,
|
||||
增量区分性留难样本扩展(协议 §1 诚实注记)。
|
||||
- **(b) 盲测(查询式 vs 全文档背诵)** ❌ 四判据未全立 ⇒ 按 three-problems.md §3.7 **继续改协议**——
|
||||
deepseek-flash:docs 4/6 vs query 0/3(端点 infra 剔除 3/6);首轮语法错 2/3 vs 2/6;token 69,771 vs 57,276。
|
||||
mimo-v2.6-pro 两跑均端点 infra(12/12、11/12,**不可评估**;结果回填 [perf-validation-report-2026-09-23.md](perf-validation-report-2026-09-23.md) §3)。
|
||||
**真发现**:查询机制本身无缺陷(多词查询亦正确命中卡片),败例是**跨语言习惯泄漏**(元组表达式/Python 调用形)
|
||||
——模型"不知道自己不知道"而不去查。协议改进主攻方向:**自查触发机制**(validate 前置自检、
|
||||
陌生构造强制查询),而非继续改进卡片。
|
||||
- **(c) 契约消融** ✅ PASS——拦截率 4/4=100%、误报率 0/4=0%、无契约零回归 4/4(四边界:调用实参/返回/表绑定/键访问)。
|
||||
- **(d) 防漂移演练** ✅ PASS——必报率 9/9=100%(9 类故意写错全数诊断)、负对照误报 0/2。
|
||||
- **(e) 核心守卫演练** ✅ PASS——3/3 变异注入必红(清单私增/文档私增/金样删除)、还原必绿。
|
||||
- **判据矩阵回填**:three-problems.md §5.1 逐行标注——问题 1 ✅、问题 2 ✅、**问题 3 ❌(未解决)**。
|
||||
- **测量端记录**:LLM 网关随机空体为最大干扰(重试 5→12 次、退避 5→15 秒、截断自适应 max_tokens);
|
||||
harness 修复两处协议性 bug(查询额度用满误判、提取失败不进修复回路)后采样;
|
||||
模型口径按用户指示由 `cflmy`/`mimo-v2.6-flash` 改 `mimo-v2.6-pro`/`deepseek-flash`(实验 (a) 数据为原组合,不重跑)。
|
||||
- **全量干净回归**(无 API key 基线)零失败。
|
||||
@@ -0,0 +1,393 @@
|
||||
# Marqdo 三大问题:解决方向与验证设计
|
||||
|
||||
——证明「被解决」,而不是「被遮盖」
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **设计 · 已导出本期开发任务:[three-problems-plan.md](three-problems-plan.md);验证实验不属于开发任务** |
|
||||
| 日期 | 2026-09-22 |
|
||||
| 来源 | [001.md](../next/001.md) 评审讨论提炼的三大问题;用户决议:**问题二走「渐进式契约 / 标注类型」**(拒绝照搬静态类型,也拒绝放弃语义校验);**问题三(AI 可靠反馈 + AI 不必学语法)是下一步的超级核心** |
|
||||
| 相关 | [markdown-mapping-v0.3.md](../design/markdown-mapping-v0.3.md)(语法宪法)· [tables-maps-footnotes.md](tables-maps-footnotes.md)(表格几何)· [catalog-cli.md](../design/catalog-cli.md) · [ai-skill.md](../design/ai-skill.md) · [objects.md](../design/objects.md) · [interpreter.md](interpreter.md) |
|
||||
|
||||
> **本文定位:三大问题的解决方向与验证设计。** 给三大问题定义「什么算被解决」(机器可验证判据,见 §5.1)与验证实验(修复率 / 盲测 / 消融 / 防漂移演练,见 §5.2),把「真正解决」与「遮盖过去」区分开。文中的解决步骤(各节 S 表)是**设计方向**;本期开发任务、排期与验收以 [three-problems-plan.md](three-problems-plan.md) 为准。**验证实验(§5.2)协议见 [perf-validation.md](perf-validation.md)**(2026-09-23 用户授权自行设计并执行)。
|
||||
>
|
||||
> **一句话:语言核心收口是地基;渐进式契约是校验层,schema 直接长在表格与说明段落里;AI 语言服务消费这两者形成可靠闭环。** 三个问题必须一起解——契约是 AI 接口的数据源,核心收口是契约语法不膨胀的前提。
|
||||
|
||||
**贯穿戒律(防止「遮盖式解决」):每个阶段的验收必须是机器可验证的行为变化。只产出宣言文档、不改变工具行为的,一律视为未完成。**
|
||||
|
||||
---
|
||||
|
||||
## 0. 问题清单与依赖
|
||||
|
||||
| # | 问题 | 定位 | 一句话方案 |
|
||||
|---|------|------|-----------|
|
||||
| 1 | 语言核心正在被生态功能淹没,identity 不清 | **地基** | 三层归属表 + 准入戒律 + 核心表面守卫(CI 断言) |
|
||||
| 2 | 「静态类型列为非目标」被误读为「不需要语义校验」;但照搬 Java/TS 静态类型会毁掉低摩擦 | **校验层** | **渐进式契约**:schema 写在表格 / 说明段落里;按需契约校验;诊断带文档锚点 |
|
||||
| 3 | AI 反馈不可靠(只有给人看的文本);AI 还在背千行语法文档 | **超级核心** | **Marqdo Language Server for AI(MLSP)**:结构化诊断协议 + 可查询 schema + 修复回路 |
|
||||
|
||||
依赖链:
|
||||
|
||||
```text
|
||||
问题 1(核心收口:契约语法不许越界进核心)
|
||||
└─→ 问题 2(契约层:schema 源头 = 表格 / 说明段落)
|
||||
└─→ 问题 3(schema 注册表 = 契约 + 核心表面 + catalog)
|
||||
└─→ 实证(修复实验 + 消融基准 → README 愿景 6)
|
||||
```
|
||||
|
||||
共享地基先行项:**统一诊断协议(Diagnostic)**——问题 2 的契约校验与问题 3 的 AI 反馈都以它为出口,因此它排在最前(见 §3.3 与 §4)。
|
||||
|
||||
---
|
||||
|
||||
## 1. 问题一:语言核心必须小
|
||||
|
||||
### 1.1 诊断
|
||||
|
||||
Marqdo 现在同时是:Markdown 语言 + literate programming + 知识库 + catalog + web + WASM + 浏览器 + LLM + agent + MCP + 量子 + 线代 + 插件 ABI。工程上这很酷;语言设计上的危险是:
|
||||
|
||||
> **核心语言的 identity 被生态功能淹没;新功能随手进核心 → 语法面暗中膨胀。**
|
||||
|
||||
真正缺的不是「收敛宣言」,而是**三样机制**:
|
||||
|
||||
1. 一份穷举的**核心表面清单**——现在「什么算核心」没有唯一明文;
|
||||
2. 一张**三层归属表**——新提案无处登记,默认哪儿都塞;
|
||||
3. 一条**越界即可见的守卫**——核心语法暗增长没有任何机器报警。
|
||||
|
||||
### 1.2 目标 / 非目标
|
||||
|
||||
**目标**:Language / Runtime / Document 三层分离;核心标记维持**约 15 个**的封闭集合,生态增长不再增加语言面。
|
||||
|
||||
**非目标**:删掉 web/quantum/agent 等功能(它们不是错,层次混放才是错);重写解释器;给核心加模块可见性等系统编程特性。
|
||||
|
||||
### 1.3 三层归属表(S1 产出物,此处先锁定语义)
|
||||
|
||||
| 层 | 内容 | 仓库位置 |
|
||||
|----|------|----------|
|
||||
| **Language** | 标记映射、词法/行分类、AST、语义、升参、**契约检查器**、诊断 | `src/`(解释器核心)· `doc/design/markdown-mapping-v0.3.md` |
|
||||
| **Runtime** | stdlib、插件 ABI、WASM、debugger、view、catalog / schema CLI、MCP | `lib/` · `plugins/` · `crates/` · `src/bin` |
|
||||
| **Document** | 知识单元、skills、示例、public 站、agent 工作流、官方扩展 | `*.mq.md` · `skills/` · `ext/` · `public/` · `examples/` |
|
||||
|
||||
核心表面清单(S1 的枚举基线,来自宪法 §5,逐项进 `core-surface.md`):
|
||||
|
||||
叙述(含 `` `名` `` 声明/引用)· `**…**` 代码 · `*…*` 返回 · 空返回形(`*None*` 等)· `>` 调用 · `#` / `##` 标题单元 · 体内 `1.` 分支 · `-` 循环 · GFM 表格 = 集合 · `[键](集合)` 取元 · `修饰… [函数名]` 方括号调用 · 脚注取元(过渡)· ` ```lang ` 外联 · `---` 叙述分隔 · `` + `名` `` 形参(兼容形)。
|
||||
|
||||
### 1.4 解决方向(设计 · 任务化见 plan)
|
||||
|
||||
| 步 | 动作 | 产出 | 验收(机器可验证) |
|
||||
|----|------|------|--------------------|
|
||||
| S1 | 写 `doc/design/core-surface.md`:穷举核心标记,每个标记一行「戒律」(语义、边界、**为什么不能是普通 Markdown**) | 核心表面唯一清单 | 清单项 ↔ `tests/markup-v03/` 金样逐项有例;缺项即测试红 |
|
||||
| S2 | 写 `doc/design/layers.md`:§1.3 归属表 + **准入戒律**——任何新特性提案必须先填「进哪层」;进 Language 层必须答「为什么现有标记不够」,默认答案是「进 ext」 | 归属与准入制度 | 仓库 PR 模板加「层归属」必填项 |
|
||||
| S3 | 核心表面守卫:Rust 测试断言「词法分类枚举 ↔ core-surface.md 表格」一致(脚本解析 markdown 表格比对枚举名) | `tests/core_surface.rs` | 故意加一个新标记不更新清单 → CI 红 |
|
||||
| S4 | 核心变更强制走 ADR:新增核心标记必须有 `doc/adr/` 编号 + 反例说明;每个 release 的 CHANGELOG 单列「核心表面 Δ」 | ADR 戒律 | 发版 checklist 含「核心表面 Δ」栏 |
|
||||
| S5 | 文档收口:README/doc 目录按三层重排;`doc/design/` 只收 Language 层决策,Runtime 进 `doc/roadmap/`,生态面进各自 ext 文档 | 重排后的文档树 | README 目录树与三层表一一对应 |
|
||||
|
||||
**观测指标**(不是 KPI,是体检):核心标记计数每个 release 只许持平或下降;连续 3 个版本增长即触发 Language 层复盘。
|
||||
|
||||
### 1.5 遮盖 vs 解决
|
||||
|
||||
| 遮盖式做法(禁止) | 本方案的做法 | 判据 |
|
||||
|--------------------|--------------|------|
|
||||
| 写一篇「语言核心很重要」的宣言 | 清单 + 归属表 + **CI 守卫** | 新增标记不更新清单时 CI 必须红 |
|
||||
| 「以后尽量不加语法」的口头约定 | ADR 强制 + CHANGELOG 核心表面 Δ | 没有 ADR 编号的核心变更不允许合并 |
|
||||
| 把 ext 功能砍掉显得小 | 不砍功能,只**归层** | web/quantum/agent 全部保留且测试通过 |
|
||||
|
||||
---
|
||||
|
||||
## 2. 问题二:渐进式契约 / 标注类型
|
||||
|
||||
### 2.1 分歧点澄清(本方案的立场)
|
||||
|
||||
GPT 一侧强调「AI 可验证语言需要强语义校验」,容易被听成「上静态类型」。**这两件事不一样**:
|
||||
|
||||
> 「放弃全程序静态编译类型」**≠** 「放弃语义检查、字段解析、契约校验」。
|
||||
|
||||
照搬 Java/TS 式静态类型会破坏「自然叙述优先、低摩擦」的体验——Marqdo 会变得笨重,丢掉对业务人员 / agent 即兴脚本的友好性。**目标不是变成静态语言,而是一套附着在文档单元上的可查询契约系统,作为 AI 闭环里的校验层。**
|
||||
|
||||
渐进式契约的四条特征:
|
||||
|
||||
1. **动态默认**——不强制任何标注,保持动态,适合快速原型与 agent 即兴脚本;
|
||||
2. **契约即文档**——schema 可选地写在**表格、文档段落**里,不引入第二套标注语言;
|
||||
3. **按需校验**——不做全局类型推导;在调用边界 / 字段访问时校验「当前值 vs 契约」;
|
||||
4. **诊断即文档链接**——错误不只报类型错,还定位到文档里对应的说明段落,给 AI 的诊断自带文档上下文。
|
||||
|
||||
**独有优势(TS/Python 做不到)**:契约的 schema 源头就是表格与函数说明段落本身——不是额外写一套 JSON Schema。知识、契约、数据、代码在同一份 `.mq.md` 里,**契约即知识**。这是 Knowledge-Code Coherence 的第一个可落地实证:契约说 `int` 实际来了 `text` 时,被机器抓住的不是「文档过期」,而是**知识不一致**。
|
||||
|
||||
### 2.2 设计原则
|
||||
|
||||
| # | 原则 | 含义 |
|
||||
|---|------|------|
|
||||
| P1 | 无契约 = 全动态 | 没写契约的单元行为**一字不变**(零回归是硬验收) |
|
||||
| P2 | 不新增核心标记 | 契约语法全部骑在现有 Markdown 外形上(问题一戒律直接约束本节) |
|
||||
| P3 | 契约是数据 | 契约表本身就是 Marqdo 集合,运行时可内省(`get_schema` 一类的 lib 函数免费获得) |
|
||||
| P4 | 双向防漂移 | 不只「值 vs 契约」,还查「契约 vs 实现」(§2.7-S5) |
|
||||
| P5 | 诊断带锚点 | 凡契约诊断必含:契约表所在文件 + 行段 + 原文引用 |
|
||||
|
||||
### 2.3 契约的三个附着面
|
||||
|
||||
**A. 函数契约** —— `##` 函数体内、首个可执行行之前的**未绑定文档表**:
|
||||
|
||||
```markdown
|
||||
## 加一
|
||||
|
||||
加一函数:把输入加一。
|
||||
|
||||
| 参数 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `n` | number | 输入值 |
|
||||
|
||||
| 返回 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| — | number | n+1 |
|
||||
|
||||
对于输入变量`n`,执行**n=n+1**接着返回*n*。
|
||||
```
|
||||
|
||||
参数仍由 v0.3 升参推断(宪法 §8),契约**不改变**形参机制,只在存在时校验——零摩擦兼容存量代码。
|
||||
|
||||
**B. 集合 / 表格契约** —— 紧邻 `` `名` = `` 绑定之前的未绑定文档表:
|
||||
|
||||
```markdown
|
||||
| 字段 | 类型 | 可空 |
|
||||
|------|------|------|
|
||||
| id | number | 否 |
|
||||
| name | text | 否 |
|
||||
|
||||
`用户` =
|
||||
|
||||
| 行 | id | name |
|
||||
|----|----|------|
|
||||
| 1 | 1 | 甲 |
|
||||
```
|
||||
|
||||
**C. 对象契约** —— `# 对象` 体内首个成员之前的 `| 字段 | 类型 |` 表;同时被 catalog 提取为条目 schema(见 §3.5 项目面)。
|
||||
|
||||
**先例**:schema 长在表格里今天已经发生——`web_db_migration` 的 `| 字段 | 类型 | 可空 |` 列就是以数据形式传递的 DDL schema。契约系统把这个模式升格为**通用约定**。
|
||||
|
||||
### 2.4 形态选型(为什么是「契约表」)
|
||||
|
||||
| 候选 | 形态 | 判决 |
|
||||
|------|------|------|
|
||||
| **P-A 契约表**(选定) | 列头关键字(`参数`/`返回`/`字段`/`类型`/`说明`)的未绑定文档表 | ✅ 与「GFM 表格即数据」哲学一致;catalog 可提取;人读无负担 |
|
||||
| P-B 叙述内标注 | `` `n`: int `` 写在散文里 | ❌ **否决**:改动行内 `` `名` `` 词法 = 核心标记变更,违反问题一戒律与 P2 |
|
||||
| P-C frontmatter | YAML 声明契约 | ⚠️ 仅作文件级汇总的可选补充(serde_yaml 已在解析元信息);函数粒度不够,不作主形态 |
|
||||
|
||||
**消歧规则(防「契约表」误吃数据表)**:
|
||||
|
||||
1. **被绑定的表永不视为契约表**——`名` = 后的表是数据(宪法 §5:未绑定才「求值后丢弃」,即文档表)。`web_db_migration` 的列头恰好叫 `字段|类型|可空` 但它是绑定数据,不受影响;
|
||||
2. 契约表必须是**未绑定文档表** + 列头命中关键字集,且出现在允许位置(§2.3 A/B/C);
|
||||
3. 同一位置出现多张契约表:参数表与返回表各认各的列头;其余重复 → 诊断(防两份契约打架)。
|
||||
|
||||
### 2.5 类型词汇(刻意做小)
|
||||
|
||||
`text` · `number` · `bool` · `list` · `map` · `any` · 对象名(与 `# 对象` / `_type` 对齐)。
|
||||
|
||||
- 词汇表**只有这一行**;扩展类型词汇 = Language 层变更,走 ADR(问题一 S4)。
|
||||
- 类型名拼错(如 `nubmer`)→ **必须报诊断**(§2.7 验收),否则契约本身成为新的漂移源。
|
||||
|
||||
### 2.6 校验时机(按需四边界,不做全局推导)
|
||||
|
||||
| 边界 | 校验什么 | 时机 |
|
||||
|------|----------|------|
|
||||
| 调用实参 | 实参值 vs 参数契约(类型 + 可选 required) | 运行时调用瞬间;`check` 静态可查处一并查 |
|
||||
| 字段访问 | `[键](集合)` / `obj.field` vs 字段契约(键存在性 + 类型) | 访问瞬间 |
|
||||
| 表绑定 | `` `名` = `` 表的几何/单元格 vs 集合契约 | 绑定瞬间 |
|
||||
| 返回 | `*…*` 值 vs 返回契约 | 返回瞬间 |
|
||||
|
||||
无契约 → 该边界直接跳过(P1)。校验器**不是**编译器:不做跨函数全局推导、不要求全程序一致。
|
||||
|
||||
### 2.7 解决方向(设计 · 任务化见 plan)
|
||||
|
||||
| 步 | 动作 | 产出 | 验收(机器可验证) |
|
||||
|----|------|------|--------------------|
|
||||
| S1 | 契约提取器:按 §2.3/§2.4 规则解析未绑定文档表 → `Contract` 对象,挂入 AST 单元(`Param` 已有 `span`,契约同级带 `span`) | AST + 提取器 | 金样:三附着面各一份提取断言(含「绑定表不吃」反例) |
|
||||
| S2 | 运行时边界校验:§2.6 四边界接入 `Contract` | 运行时校验 | 有契约 + 错值 → **必**产出 §3.3 诊断;无契约金样**全绿零回归** |
|
||||
| S3 | `marqdo check`:静态按需校验(只查有契约处),`--json` 输出诊断 | CLI | 契约样例工程 `check` 退出码非 0 且诊断 JSON 合协议 |
|
||||
| S4 | 诊断锚点:契约错误必带 `doc_anchor` + `doc_quote`(契约表原文行) | 诊断增强 | 金样断言诊断 JSON 里锚点字段非空、行号可回读原文 |
|
||||
| S5 | **契约 vs 实现互查**(P4):契约参数名 ≠ 升参推断名、契约类型名未知、契约返回 vs 语句结构明显不符 → 诊断 | 防漂移检查 | 故意写错契约的金样**必须**红(而不是静默信任契约) |
|
||||
| S6 | 运行时内省:`lib` 提供 `get_schema 单元=…`(P3:契约是数据) | lib 函数 | demo:运行中取回自身函数契约表并打印 |
|
||||
|
||||
### 2.8 遮盖 vs 解决
|
||||
|
||||
| 遮盖式做法(禁止) | 本方案的做法 | 判据 |
|
||||
|--------------------|--------------|------|
|
||||
| 把「静态类型」继续列为非目标就收工 | 明确「≠ 放弃语义校验」,落**边界校验器** | 运行时行为变化可测(§2.7-S2) |
|
||||
| 照搬 Java/TS 全程序静态类型 | 渐进式:无标注零摩擦,按需四边界 | 无契约金样零回归 |
|
||||
| 另写一套 JSON Schema 文件 | schema = 表格 / 说明段落本身 | 仓库里**不允许**出现手写的第二份 schema;契约只能从 `.mq.md` 提取 |
|
||||
| 契约写了就永远正确 | 契约 vs 升参互查 + 未知类型名必报 | 故意写错契约的金样必红 |
|
||||
| 错误只报 `type mismatch` 文本 | 诊断带文档锚点 + 原文引用 | 金样断言锚点字段 |
|
||||
|
||||
---
|
||||
|
||||
## 3. 问题三(超级核心):AI 可靠反馈 + AI 不必学语法细节
|
||||
|
||||
### 3.1 两个子问题
|
||||
|
||||
**A. 可靠反馈**:今天解释器诊断是给人看的文本(`path:line:col` + message,见 ai-skill「按诊断迭代」)。AI 拿到它要「猜」哪里错、对应文档哪句话、怎么改。GPT 设想的闭环——
|
||||
|
||||
```text
|
||||
LLM → Marqdo → 语义校验 → 结构化 Error → LLM 修复 → 合格程序
|
||||
```
|
||||
|
||||
——的第一块砖就是**结构化诊断**,而且诊断必须自带文档上下文(问题二 §2.1-4 的锚点在这里被 AI 消费)。
|
||||
|
||||
**B. 不必学语法**:今天 `skills/marqdo` 是「面向模型的压缩操作手册」= **背诵式**。正确形态是**查询式**:
|
||||
|
||||
```text
|
||||
AI ── discover ─→ construct ─→ validate ─→ execute ─→ inspect
|
||||
```
|
||||
|
||||
AI 不阅读一千行文档然后猜,而是按需调 `get_syntax("branch")`、`fn_schema("print")`、`validate(program)`。**这才是 Marqdo 与普通 DSL 拉开距离的地方。**
|
||||
|
||||
### 3.2 总形态:Marqdo Language Server for AI(MLSP)
|
||||
|
||||
一套机器可读的语言服务协议。工具面(首版最小集):
|
||||
|
||||
| 工具 | 入参 | 返回 | 数据源 |
|
||||
|------|------|------|--------|
|
||||
| `syntax_list` / `syntax_get` | — / 构造名 | 核心构造表 / 单构造戒律 | core-surface(问题一 S1,机械转 JSON) |
|
||||
| `fn_schema` | 函数名 | 参数/返回契约 + 说明 + 锚点 | 契约(问题二)+ stdlib/ext 签名 |
|
||||
| `type_schema` | 对象名 | 字段契约 + 方法表 | `# 对象` 契约 + catalog |
|
||||
| `validate` | 程序文本/文件 | `Diagnostic[]` | 契约校验器 + 语义检查 |
|
||||
| `repair` | 程序 + 诊断 | 建议 patch(限定行级) | 诊断规则库 |
|
||||
| `run` | 入口 | 结果或结构化错误 | 运行时 |
|
||||
|
||||
交付通道(按序):**`marqdo mcp`**(MCP server,复用 agent ext 的 MCP 经验)→ CLI `--json`(任何宿主可用)→ `skills/marqdo` 改造成「教 AI 查询」(§3.6)。
|
||||
|
||||
### 3.3 先决:统一诊断协议(可靠反馈的地基)
|
||||
|
||||
所有层(解释器、check、契约、插件错误)统一产出:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": "contract.arg_mismatch",
|
||||
"severity": "error",
|
||||
"message": "参数 `n` 期望 number,实际 text",
|
||||
"span": { "file": "demo.mq.md", "line": 12, "col": 3, "heading": ["加一"] },
|
||||
"doc_anchor": "demo.mq.md#L4-L7",
|
||||
"doc_quote": "| `n` | number | 输入值 |",
|
||||
"contract_ref": { "unit": "加一", "kind": "param", "name": "n", "declared": "number" },
|
||||
"suggestion": "把实参改为数字;若契约有误,改契约表的类型列"
|
||||
}
|
||||
```
|
||||
|
||||
规则:
|
||||
|
||||
1. 凡可定位必带 `span`(AST 已有 span,宪法 §10.2);
|
||||
2. 凡契约错误必带 `doc_anchor` + `doc_quote`——**错误直接定位到文档里对应的说明段落**;
|
||||
3. 凡可修复建议必带 `suggestion`(给修复回路的原料);
|
||||
4. 人类文本输出保持现状,`--json` 并行提供——人机同读同写,诊断也一样。
|
||||
|
||||
### 3.4 修复回路(repair loop)
|
||||
|
||||
```text
|
||||
validate(p) → diagnose → patch(p, diag) → validate(p') → … (预算 N=3 轮)
|
||||
```
|
||||
|
||||
- **限定行级 patch**,不允许整文件重写(防止 AI「修复」时破坏叙述);
|
||||
- 尝试预算与重复诊断去重,防循环;
|
||||
- 实验指标:**一轮修复率**(带锚点 vs 不带锚点两组对比)——这是「可靠反馈」的直接证据。
|
||||
|
||||
### 3.5 Schema 注册表(三来源,零手写第二份)
|
||||
|
||||
| 面 | 来源 | 说明 |
|
||||
|----|------|------|
|
||||
| 语言面 | `core-surface.md` 机械转 JSON | 构造戒律原文进 `description` |
|
||||
| 函数 / 类型面 | **契约**(问题二产物)+ stdlib/ext 签名 | schema 源头 = 表格与说明段落(§2.1 独有优势) |
|
||||
| 项目面 | `catalog.yaml`(catalog 自动生成) | 知识单元、链接关系(知识图的边) |
|
||||
|
||||
格式 `marqdo-schema.json`(示意):
|
||||
|
||||
```json
|
||||
{
|
||||
"language": "marqdo",
|
||||
"version": "0.3",
|
||||
"constructs": [{ "name": "branch", "syntax": "1. … 2. … N. *", "rule": "自 1. 递增;重出 1. 开新判断" }],
|
||||
"functions": [{ "name": "加一", "params": [{ "name": "n", "type": "number", "desc": "输入值", "anchor": "#L4" }], "returns": { "type": "number" } }]
|
||||
}
|
||||
```
|
||||
|
||||
命令:`marqdo schema [PATH]`(catalog 的兄弟命令,同样**只生成不手改**)。
|
||||
|
||||
### 3.6 AI Skill:从「背诵」改「查询」
|
||||
|
||||
`skills/marqdo` 改造原则:
|
||||
|
||||
- `SKILL.md` 只留**十条不可违反的硬规则**(叙述默认安全、`**` 代码 `*` 返回、表格几何、命名/双语约定等)+ MLSP 工具用法;
|
||||
- 语法细节、API 明细**一律走 `syntax_get` / `fn_schema`**,不再进上下文;
|
||||
- 工作流改为:construct(凭硬规则写)→ `validate` → 按诊断修 → `run`。
|
||||
|
||||
### 3.7 解决方向(设计 · 任务化见 plan)
|
||||
|
||||
| 步 | 动作 | 验收(机器可验证) |
|
||||
|----|------|--------------------|
|
||||
| S1 | Diagnostic 结构落地(Rust)+ 解释器诊断改造 + `run --json` / `check --json` | 现有错误金样转 JSON 后 `code`/`span` 全非空 |
|
||||
| S2 | 契约诊断接入(消费 §2.7-S4 的锚点) | §2.8 判据全部成立 |
|
||||
| S3 | `marqdo schema` 生成器(三来源汇聚) | 生成物 diff 稳定(两次生成 byte-identical);stdlib 主函数覆盖率 ≥ 90% |
|
||||
| S4 | `marqdo mcp`:暴露 §3.2 工具面 | 用 MCP 客户端冒烟:`fn_schema` → `validate` → `repair` 全链路 |
|
||||
| S5 | `skills/marqdo` 改造为查询式(保留硬规则十条) | skill 体积减半以上;既有 `marqdo run` 金样仍全绿 |
|
||||
| S6 | 修复回路 + 实验执行(实验设计见 §5.2) | 实验报告(进 MarqdoThesis,对接 README 愿景 6) |
|
||||
|
||||
**「解决而不是遮盖」的最终判据(问题三)**:盲测组(零语法文档 + 工具)的成绩**不劣于**全文档背诵组,且 token 显著更少、修复轮次不升——达不到就继续改协议,**不许**用「再写一篇更好的文档」充数。
|
||||
|
||||
### 3.8 遮盖 vs 解决
|
||||
|
||||
| 遮盖式做法(禁止) | 本方案的做法 | 判据 |
|
||||
|--------------------|--------------|------|
|
||||
| 再写一份更全的 AI 语法文档 | 查询协议 + schema 注册表 | 盲测实验成立 |
|
||||
| 让 AI「多读 reference.md」 | `syntax_get` / `fn_schema` 按需取 | skill 体积减半且成绩不降 |
|
||||
| 报错文本润色一下 | 统一 Diagnostic JSON + 文档锚点 | 诊断金样断言锚点字段 |
|
||||
| 主观感觉「AI 好用了」 | 一轮修复率 / 盲测 / token 三组数字 | 实验报告可复现(MarqdoThesis) |
|
||||
|
||||
---
|
||||
|
||||
## 4. 路线顺序(依赖序 · 排期见 [three-problems-plan.md](three-problems-plan.md))
|
||||
|
||||
> 本表是**依赖顺序**,不是排期表——任务化与本期范围见 [three-problems-plan.md](three-problems-plan.md)。
|
||||
|
||||
| 顺序 | 内容 | 产出 | 依赖 |
|
||||
|------|------|------|------|
|
||||
| **M0 核心收口** | 问题一 S1–S5 | core-surface.md · layers.md · CI 守卫 | — |
|
||||
| **M1 诊断地基** | 问题三 S1(Diagnostic + `--json`) | 统一诊断协议 | M0 |
|
||||
| **M2 契约层** | 问题二 S1–S6 | 提取器 · 四边界校验 · `check` · 锚点 · 防漂移 | M1 |
|
||||
| **M3 AI 语言服务** | 问题三 S2–S5 | schema 生成 · `marqdo mcp` · 查询式 skill · 修复回路 | M2 |
|
||||
| **M4 实证** | 问题三 S6 | 修复实验 + 盲测 + 消融(对接 README 愿景 6) | M3 |
|
||||
|
||||
顺序理由:Diagnostic 先于契约(契约诊断是它的第一个消费者);契约先于 MLSP(契约是 schema 注册表的主数据源)——**任何一个跳步都会倒逼「另写一套 schema / 另造一套错误」的遮盖式捷径**。
|
||||
|
||||
---
|
||||
|
||||
## 5. 验证矩阵与实验设计(本文核心)
|
||||
|
||||
### 5.1 判据矩阵:什么算「已解决」
|
||||
|
||||
| 问题 | 解决的定义(机器可验证) | 遮盖式收场(一句话就能写完的,禁止) | 验证手段 | 判定(2026-09-23) |
|
||||
|------|--------------------------|--------------------------------------|----------|--------------------|
|
||||
| 1 核心要小 | 新标记不更新 core-surface → **CI 红** | 「我们重视语言核心」 | CI 守卫演练:故意加标记不更新清单,必须红 | ✅ (e) 3/3 必红→还原必绿 |
|
||||
| 2 语义校验 | 四边界校验器上线;契约写错**必**被机器抓;无契约路径零回归 | 「静态类型是非目标」/照搬 TS 类型/另写 JSON Schema | 契约金样 + 防漂移金样 + 全量回归 | ✅ (c) 拦截 100%/误报 0%/零回归;(d) 必报 9/9 |
|
||||
| 3 AI 反馈 / 语法 | 盲测组错误率 ≤ 全文档组且 token 显著更少;一轮修复率可复现 | 「再写更好的文档」 | §5.2 实验 (a)(b)(d) | ❌ (a) 过(双模型 100%);**(b) 四判据未全立** ⇒ 按 §3.7 继续改协议 |
|
||||
|
||||
> 2026-09-23 实测数字与判据对照:[perf-validation-report-2026-09-23.md](perf-validation-report-2026-09-23.md)。
|
||||
> 按本矩阵口径:**问题 1、2 实证成立;问题 3 未解决**(查询式盲测未跑赢全背诵——详见报告 §3 的四条要点,
|
||||
> 其中「模型不查自己不知道的语法」是协议改进的主攻方向)。
|
||||
|
||||
### 5.2 实验设计(性能 / 可靠性验证)
|
||||
|
||||
| 实验 | 对照组 | 指标 | 材料 |
|
||||
|------|--------|------|------|
|
||||
| (a) 锚点修复 | 诊断带 `doc_anchor` vs 不带 | 一轮修复率、平均修复轮次 | 注入已知 bug 的金样集 |
|
||||
| (b) 盲测(零语法文档) | 只给 MLSP 工具 vs 给全文档背诵(现 `skills/marqdo`) | 语法错误率、契约违规率、token 消耗 | 任务集(QuixBugs 类 + 业务逻辑类) |
|
||||
| (c) 契约消融 | 契约开 / 关 | 契约拦截率、误报率、无契约路径零回归 | 同上 |
|
||||
| (d) 防漂移演练 | 契约故意写错(类型 typo / 参数名与升参不符) | 必报率 = 100% | 契约金样 |
|
||||
| (e) 核心守卫演练 | 故意加新标记不更新清单 | CI 红 = 必须 | — |
|
||||
|
||||
实验基建:复用 MarqdoThesis `formal/exp1–3` 的 runner / oracle / 容错重试;报告即 README 愿景 6 的「可靠性基准」材料。**判据矩阵全绿 + 实验数字可复现之前,三大问题一律视为未解决**——这是「不遮盖」的最终保险。
|
||||
|
||||
---
|
||||
|
||||
## 6. 未决问题(进 M0/M2 的 ADR 逐条锁定)
|
||||
|
||||
1. 契约表消歧的边界案例:紧邻绑定的未绑定表(注释表)紧贴数据表时的归属规则要不要「隔空行才算分离」?(M2-S1 金样定夺)
|
||||
2. 参数表与返回表合并成单表(首列 `参数`/`返回` 混排)是否更可读?(M2-S1 对比样例后定)
|
||||
3. 对象字段契约:与 [objects.md](../design/objects.md) 「不把表当对象定义语言」的非目标如何划界——契约是**可选校验元数据**,不定义对象布局(对象仍是运行时 map);需 ADR 明文,避免与 [tables-maps-footnotes.md](tables-maps-footnotes.md) 非目标条目互相打架。
|
||||
4. 类型词汇是否收 `对象名` 之外的复合形(如 `list<number>`)?——默认**不收**,需要时走 ADR(P2 + 问题一 S4)。
|
||||
5. `marqdo check` 独立子命令 vs `run --check` 并存?(M2-S3 定,倾向独立 + `run` 默认轻查)
|
||||
6. `repair` 的 patch 边界:行级 patch 的具体格式(unified diff vs 语句级替换)?(M3-S4 定,倾向语句级——与 AST span 对齐)
|
||||
+1
-1
@@ -1 +1 @@
|
||||
1.0.6
|
||||
1.1.1
|
||||
+518
-24
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: ext/ai/agent
|
||||
description: Document-driven agent — step / plan; tools via lib/subtask; runtime via ABI v2 agent plugin.
|
||||
description: Document-driven agent v2 — step / plan / run; Skill Compilation via ABI plugin (episodes → learn → llm_free).
|
||||
import llm:ext/ai/llm.mq.md
|
||||
import json:lib/json.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
@@ -49,6 +49,37 @@ The runbook `.mq.md` is ground truth (code as documentation). Default budgets ke
|
||||
|
||||
---
|
||||
|
||||
## model_turn
|
||||
+ `model`
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
|
||||
One LLM turn via the semantic surface (`ask` / `stream`+`collect`). Returns `{text,llm_calls,tokens}` for episode metrics.
|
||||
|
||||
1. `stream`
|
||||
**evs = > model.stream prompt=`prompt` echo=`echo`**
|
||||
**text = > llm.collect events=`evs`**
|
||||
**tokens = 0**
|
||||
2. *
|
||||
**r = > model.ask prompt=`prompt`**
|
||||
**text = [text](r)**
|
||||
**tokens = [tokens](r)**
|
||||
1. `tokens`
|
||||
**_ = 1**
|
||||
2. *
|
||||
**tokens = 0**
|
||||
|
||||
`out` =
|
||||
|
||||
| text | llm_calls | tokens |
|
||||
|------|-----------|--------|
|
||||
| `text` | 1 | `tokens` |
|
||||
|
||||
*out*
|
||||
|
||||
---
|
||||
|
||||
## dump_step_context
|
||||
+ `agent`
|
||||
+ `task`=""
|
||||
@@ -424,6 +455,128 @@ Call a named tool from a MCP-shaped fixture (`results` / `calls` map). Optional
|
||||
|
||||
---
|
||||
|
||||
## mcp_connect
|
||||
+ `command`
|
||||
+ `args`=None
|
||||
+ `session`="default"
|
||||
+ `name`="mcp"
|
||||
|
||||
Open a stdio MCP client session (spawn `command` + `args`, JSON-RPC initialize). Evidence only — workbook remains authority.
|
||||
|
||||
**p = > plugin.native_path name="agent"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
2. *
|
||||
> print text=ext/ai/agent: native agent plugin not found
|
||||
> sys.exit code=1
|
||||
|
||||
1. `args`
|
||||
*> agent_mcp_client action="connect" command=`command` args=`args` session=`session` name=`name`*
|
||||
2. *
|
||||
*> agent_mcp_client action="connect" command=`command` session=`session` name=`name`*
|
||||
|
||||
## mcp_client_list
|
||||
+ `session`="default"
|
||||
+ `command`=None
|
||||
+ `args`=None
|
||||
|
||||
List tools from an open MCP session, or one-shot when `command=` is set (no session retained).
|
||||
|
||||
1. `command`
|
||||
1. `args`
|
||||
*> agent_mcp_client action="list" command=`command` args=`args`*
|
||||
2. *
|
||||
*> agent_mcp_client action="list" command=`command`*
|
||||
2. *
|
||||
*> agent_mcp_client action="list" session=`session`*
|
||||
|
||||
## mcp_client_call
|
||||
+ `name`=None
|
||||
+ `session`="default"
|
||||
+ `arguments`=None
|
||||
+ `command`=None
|
||||
+ `args`=None
|
||||
+ `tool`=None
|
||||
|
||||
Call a tool on an open MCP session, or one-shot when `command=` is set. Prefer `tool=` for oneshot when `name` would collide with server name.
|
||||
|
||||
1. `command`
|
||||
1. `tool`
|
||||
**tn = tool**
|
||||
2. *
|
||||
**tn = name**
|
||||
1. `args`
|
||||
*> agent_mcp_client action="call" command=`command` args=`args` tool=`tn` arguments=`arguments`*
|
||||
2. *
|
||||
*> agent_mcp_client action="call" command=`command` tool=`tn` arguments=`arguments`*
|
||||
2. *
|
||||
*> agent_mcp_client action="call" session=`session` name=`name` arguments=`arguments`*
|
||||
|
||||
## mcp_disconnect
|
||||
+ `session`="default"
|
||||
|
||||
Close a stdio MCP client session.
|
||||
|
||||
*> agent_mcp_client action="close" session=`session`*
|
||||
|
||||
---
|
||||
|
||||
## resume_save
|
||||
+ `id`
|
||||
+ `goal`=None
|
||||
+ `task`=None
|
||||
+ `status`=running
|
||||
+ `round`=0
|
||||
+ `workbook`=None
|
||||
+ `events`=None
|
||||
+ `route`=None
|
||||
+ `result`=None
|
||||
+ `resume_dir`=.marqdo/agent-resume
|
||||
|
||||
Persist a plan/run checkpoint as `.marqdo/agent-resume/<id>.json`.
|
||||
|
||||
**p = > plugin.native_path name="agent"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
2. *
|
||||
> print text=ext/ai/agent: native agent plugin not found
|
||||
> sys.exit code=1
|
||||
|
||||
*> agent_resume_save id=`id` goal=`goal` task=`task` status=`status` round=`round` workbook=`workbook` events=`events` route=`route` result=`result` resume_dir=`resume_dir`*
|
||||
|
||||
## resume_load
|
||||
+ `id`
|
||||
+ `resume_dir`=.marqdo/agent-resume
|
||||
|
||||
Load a resume checkpoint (or `{ok:false,found:false}`).
|
||||
|
||||
*> agent_resume_load id=`id` resume_dir=`resume_dir`*
|
||||
|
||||
## resume_clear
|
||||
+ `id`
|
||||
+ `resume_dir`=.marqdo/agent-resume
|
||||
|
||||
Delete a resume checkpoint.
|
||||
|
||||
*> agent_resume_clear id=`id` resume_dir=`resume_dir`*
|
||||
|
||||
## resume_list
|
||||
+ `resume_dir`=.marqdo/agent-resume
|
||||
|
||||
List checkpoint ids under `resume_dir`.
|
||||
|
||||
*> agent_resume_list resume_dir=`resume_dir`*
|
||||
|
||||
## resume
|
||||
+ `id`
|
||||
+ `resume_dir`=.marqdo/agent-resume
|
||||
|
||||
Author entry: load checkpoint for continuation. Returns the checkpoint map (`found`/`round`/`workbook`/…). Callers continue with `plan`/`run` using saved fields; clear when done.
|
||||
|
||||
*> resume_load id=`id` resume_dir=`resume_dir`*
|
||||
|
||||
---
|
||||
|
||||
## extract_call_args
|
||||
+ `reply`=None
|
||||
|
||||
@@ -1264,7 +1417,7 @@ Parent Plan-and-Move prompt. Short protocol only — no long monologues.
|
||||
+ `events`
|
||||
+ `max_acts`=6
|
||||
|
||||
LLM Plan → CALL/READ/DECISION loop. Returns `{decision,reply,observation,events}`.
|
||||
LLM Plan → CALL/READ/DECISION loop. Returns `{decision,reply,observation,events,llm_calls,tokens}`.
|
||||
|
||||
**model = > json.get value=`agent` key="model"**
|
||||
**last_obs = observation**
|
||||
@@ -1272,6 +1425,8 @@ LLM Plan → CALL/READ/DECISION loop. Returns `{decision,reply,observation,event
|
||||
**decision = None**
|
||||
**last_reply = None**
|
||||
**evs_all = events**
|
||||
**llm_calls = 0**
|
||||
**tokens = 0**
|
||||
|
||||
- `left` > 0
|
||||
1. `decision`
|
||||
@@ -1279,11 +1434,17 @@ LLM Plan → CALL/READ/DECISION loop. Returns `{decision,reply,observation,event
|
||||
2. *
|
||||
**ctx = > build_plan_context agent=`agent` goal=`goal` observation=`last_obs` explore_attempt=`explore_attempt` explore_n=`explore_n` phase=`phase`**
|
||||
1. `stream`
|
||||
**evs = > model.complete prompt=`ctx` stream=True echo=`echo`**
|
||||
**last_reply = > llm.stream_result events=`evs`**
|
||||
**evs = > model.stream prompt=`ctx` echo=`echo`**
|
||||
**last_reply = > llm.collect events=`evs`**
|
||||
**evs_all = > plan_merge_deltas events=`evs_all` from=`evs` stream=`stream`**
|
||||
**llm_calls = llm_calls + 1**
|
||||
2. *
|
||||
**last_reply = > model.complete prompt=`ctx`**
|
||||
**turn = > model_turn model=`model` prompt=`ctx` stream=False echo=`echo`**
|
||||
**last_reply = > json.get value=`turn` key="text"**
|
||||
**c = > json.get value=`turn` key="llm_calls"**
|
||||
**t = > json.get value=`turn` key="tokens"**
|
||||
**llm_calls = llm_calls + c**
|
||||
**tokens = tokens + t**
|
||||
**last_reply = > trim value=`last_reply`**
|
||||
**act = > extract_plan_act reply=`last_reply`**
|
||||
**kind = > json.get value=`act` key="kind"**
|
||||
@@ -1314,6 +1475,8 @@ LLM Plan → CALL/READ/DECISION loop. Returns `{decision,reply,observation,event
|
||||
**out = > json.set map=`out` key="reply" value=`last_reply`**
|
||||
**out = > json.set map=`out` key="observation" value=`last_obs`**
|
||||
**out = > json.set map=`out` key="events" value=`evs_all`**
|
||||
**out = > json.set map=`out` key="llm_calls" value=`llm_calls`**
|
||||
**out = > json.set map=`out` key="tokens" value=`tokens`**
|
||||
*out*
|
||||
|
||||
---
|
||||
@@ -1515,6 +1678,316 @@ Clear the plugin session bag for this agent id.
|
||||
> agent_history_clear id=`id`
|
||||
****
|
||||
|
||||
## resolve
|
||||
+ `task`
|
||||
+ `kb_dir`=.marqdo/agent-kb
|
||||
+ `near_match`=True
|
||||
+ `near_threshold`=0.78
|
||||
|
||||
Skill resolver (v2): exact → alias → canonical → near. Returns `mode` / `resource` / `llm_free` / `confidence`. Plugin ABI — no core host_* calls.
|
||||
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
*> agent_resolve task=`task` kb_dir=`kb_dir` near_match=`near_match` near_threshold=`near_threshold` tools=`tools`*
|
||||
|
||||
## route
|
||||
+ `task`
|
||||
+ `backend`=auto
|
||||
+ `kb_dir`=.marqdo/agent-kb
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
+ `threshold`=0.78
|
||||
|
||||
Adaptive Routing (Phase 4): abstract router — `marqdo` | `small-llm` | `llm` | `auto`. `jev` is accepted as a **compat alias** of `small-llm` (not a first-class backend). `auto` cascades policy → kb → small-llm request. May return `needs_llm=True` + `prompt` for the caller to complete with `router_model` (any llm handle), then `route_apply`.
|
||||
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
*> agent_route task=`task` backend=`backend` kb_dir=`kb_dir` memory_dir=`memory_dir` threshold=`threshold` tools=`tools`*
|
||||
|
||||
## route_apply
|
||||
+ `task`
|
||||
+ `reply`
|
||||
+ `kb_dir`=.marqdo/agent-kb
|
||||
+ `backend`=small-llm
|
||||
|
||||
Apply a router model line (`SKILL:<slug>` / `EXPLORE`) to a kb skill hit.
|
||||
|
||||
*> agent_route_apply task=`task` reply=`reply` kb_dir=`kb_dir` backend=`backend`*
|
||||
|
||||
## compile_policy
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
+ `min_evidence`=3
|
||||
+ `force`=False
|
||||
|
||||
Policy Compilation: stable episode skill selections → `policies/router.mq.md` (deterministic Marqdo rules).
|
||||
|
||||
*> agent_compile_policy memory_dir=`memory_dir` min_evidence=`min_evidence` force=`force`*
|
||||
|
||||
## maybe_compile_policy
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
+ `min_evidence`=3
|
||||
|
||||
Learn gate for the Adaptive Router policy layer.
|
||||
|
||||
*> agent_maybe_compile_policy memory_dir=`memory_dir` min_evidence=`min_evidence`*
|
||||
|
||||
## record_episode
|
||||
+ `task`
|
||||
+ `status`=ok
|
||||
+ `result`=None
|
||||
+ `mode`=explore
|
||||
+ `skill`=none
|
||||
+ `observation`=None
|
||||
+ `workbook`=None
|
||||
+ `cache`=None
|
||||
+ `match`=None
|
||||
+ `llm_calls`=0
|
||||
+ `tool_calls`=0
|
||||
+ `error`=None
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
|
||||
Persist one run as `.mq.md` under `memory_dir/episodes/YYYY/MM/` (v2 Phase 1). Memory is executable documentation, not a vector bag.
|
||||
|
||||
*> agent_record_episode task=`task` status=`status` result=`result` mode=`mode` skill=`skill` observation=`observation` workbook=`workbook` cache=`cache` match=`match` llm_calls=`llm_calls` tool_calls=`tool_calls` error=`error` memory_dir=`memory_dir`*
|
||||
|
||||
## maybe_learn
|
||||
+ `task`
|
||||
+ `kb_dir`=.marqdo/agent-kb
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
+ `improve_every`=3
|
||||
+ `promote`=True
|
||||
|
||||
Offline learn gate: when ≥ `improve_every` successful episodes share a task hash, generalize → synthesize → compile → optional OKF promote (`llm_free`). Also attempts Policy Compilation for Adaptive Routing.
|
||||
|
||||
**skill_learn = > agent_maybe_learn task=`task` kb_dir=`kb_dir` memory_dir=`memory_dir` improve_every=`improve_every` promote=`promote`**
|
||||
**policy_learn = > agent_maybe_compile_policy memory_dir=`memory_dir` min_evidence=`improve_every`**
|
||||
**out = > json.parse text={"learned":false}**
|
||||
**sl = > json.get value=`skill_learn` key="learned"**
|
||||
**out = > json.set map=`out` key="learned" value=`sl`**
|
||||
**out = > json.set map=`out` key="skill" value=`skill_learn`**
|
||||
**out = > json.set map=`out` key="policy" value=`policy_learn`**
|
||||
*out*
|
||||
|
||||
## compile
|
||||
+ `task`
|
||||
+ `kb_dir`=.marqdo/agent-kb
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
+ `min_evidence`=3
|
||||
+ `promote`=True
|
||||
+ `force`=False
|
||||
|
||||
Compile a candidate skill into versioned `skills/<slug>/vN.mq.md` and optionally promote into agent-kb.
|
||||
|
||||
*> agent_compile_skill task=`task` kb_dir=`kb_dir` memory_dir=`memory_dir` min_evidence=`min_evidence` promote=`promote` force=`force`*
|
||||
|
||||
## metrics
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
|
||||
Reasoning amortization stats over recorded episodes (`avg_llm_calls`, `compiled_runs`, …).
|
||||
|
||||
*> agent_metrics memory_dir=`memory_dir`*
|
||||
|
||||
## run
|
||||
+ `task`
|
||||
+ `kb_dir`=.marqdo/agent-kb
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
+ `learn`=True
|
||||
+ `verify`=True
|
||||
+ `writeback`=True
|
||||
+ `reuse`=True
|
||||
+ `near_match`=True
|
||||
+ `near_threshold`=0.78
|
||||
+ `promote`=True
|
||||
+ `improve_every`=3
|
||||
+ `explore_n`=3
|
||||
+ `max_rounds`=4
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
+ `trace`=False
|
||||
+ `force`=False
|
||||
+ `router`=auto
|
||||
+ `router_model`=None
|
||||
+ `resume_id`=None
|
||||
+ `resume_dir`=.marqdo/agent-resume
|
||||
|
||||
Primary author entry (v2 + P4): Adaptive `route` → execute (policy/kb skill) or `plan` → record episode → maybe_learn (skill + policy). `router` = `auto`|`marqdo`|`small-llm`|`llm` (`jev`→`small-llm` alias). Optional `router_model` (llm handle) answers `needs_llm` prompts — small models welcome.
|
||||
|
||||
**routed = > `self`.route task=`task` backend=`router` kb_dir=`kb_dir` memory_dir=`memory_dir` threshold=`near_threshold`**
|
||||
**route_matched = > json.get value=`routed` key="matched"**
|
||||
**needs_llm = > json.get value=`routed` key="needs_llm"**
|
||||
**exec_mode = "explore"**
|
||||
**llm_calls = 0**
|
||||
**tokens = 0**
|
||||
**out = None**
|
||||
**route_backend = > json.get value=`routed` key="backend"**
|
||||
|
||||
1. `route_matched`
|
||||
**path = > json.get value=`routed` key="resource"**
|
||||
1. `path`
|
||||
**aw = > await_workbook path=`path`**
|
||||
**code = > json.get value=`aw` key="code"**
|
||||
**child_val = > json.get value=`aw` key="value"**
|
||||
**last_obs = > json.get value=`aw` key="observation"**
|
||||
1. `code` == 0
|
||||
> agent_kb_record_hit kb_dir=`kb_dir` goal=`task` improve_every=`improve_every`
|
||||
**lf = > json.get value=`routed` key="llm_free"**
|
||||
1. `lf`
|
||||
**exec_mode = "compiled"**
|
||||
2. *
|
||||
**exec_mode = "routed"**
|
||||
**out = > json.parse text={"status":"ok"}**
|
||||
**out = > json.set map=`out` key="status" value="ok"**
|
||||
**out = > json.set map=`out` key="goal" value=`task`**
|
||||
**out = > json.set map=`out` key="result" value=`child_val`**
|
||||
**out = > json.set map=`out` key="workbook" value=`path`**
|
||||
**out = > json.set map=`out` key="cache" value="hit"**
|
||||
**rmode = > json.get value=`routed` key="mode"**
|
||||
**out = > json.set map=`out` key="match" value=`rmode`**
|
||||
**sk = > json.get value=`routed` key="skill"**
|
||||
**out = > json.set map=`out` key="skill" value=`sk`**
|
||||
**out = > json.set map=`out` key="observation" value=`last_obs`**
|
||||
**out = > json.set map=`out` key="summary" value="adaptive route hit"**
|
||||
**out = > json.set map=`out` key="route" value=`routed`**
|
||||
**llm_calls = 0**
|
||||
2. *
|
||||
**route_matched = False**
|
||||
2. *
|
||||
**route_matched = False**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `route_matched`
|
||||
**_ = 1**
|
||||
2. `needs_llm`
|
||||
1. `router_model`
|
||||
**prompt = > json.get value=`routed` key="prompt"**
|
||||
**turn = > model_turn model=`router_model` prompt=`prompt` stream=`stream` echo=`echo`**
|
||||
**reply = > json.get value=`turn` key="text"**
|
||||
**llm_calls = > json.get value=`turn` key="llm_calls"**
|
||||
**tokens = > json.get value=`turn` key="tokens"**
|
||||
**applied = > `self`.route_apply task=`task` reply=`reply` kb_dir=`kb_dir` backend=`router`**
|
||||
**am = > json.get value=`applied` key="matched"**
|
||||
1. `am`
|
||||
**path = > json.get value=`applied` key="resource"**
|
||||
**aw = > await_workbook path=`path`**
|
||||
**code = > json.get value=`aw` key="code"**
|
||||
**child_val = > json.get value=`aw` key="value"**
|
||||
**last_obs = > json.get value=`aw` key="observation"**
|
||||
1. `code` == 0
|
||||
**exec_mode = "routed"**
|
||||
**out = > json.parse text={"status":"ok"}**
|
||||
**out = > json.set map=`out` key="status" value="ok"**
|
||||
**out = > json.set map=`out` key="goal" value=`task`**
|
||||
**out = > json.set map=`out` key="result" value=`child_val`**
|
||||
**out = > json.set map=`out` key="workbook" value=`path`**
|
||||
**out = > json.set map=`out` key="cache" value="soft-hit"**
|
||||
**out = > json.set map=`out` key="match" value="routed"**
|
||||
**sk = > json.get value=`applied` key="skill"**
|
||||
**out = > json.set map=`out` key="skill" value=`sk`**
|
||||
**out = > json.set map=`out` key="observation" value=`last_obs`**
|
||||
**out = > json.set map=`out` key="summary" value="router model selected skill"**
|
||||
**out = > json.set map=`out` key="route" value=`applied`**
|
||||
**route_matched = True**
|
||||
2. *
|
||||
**route_matched = False**
|
||||
2. *
|
||||
**route_matched = False**
|
||||
2. *
|
||||
**route_matched = False**
|
||||
3. *
|
||||
**_ = 1**
|
||||
|
||||
1. `route_matched`
|
||||
**_ = 1**
|
||||
2. *
|
||||
**resolved = > `self`.resolve task=`task` kb_dir=`kb_dir` near_match=`near_match` near_threshold=`near_threshold`**
|
||||
**mode = > json.get value=`resolved` key="mode"**
|
||||
**llm_free = > json.get value=`resolved` key="llm_free"**
|
||||
1. `llm_free`
|
||||
**path = > json.get value=`resolved` key="resource"**
|
||||
**aw = > await_workbook path=`path`**
|
||||
**code = > json.get value=`aw` key="code"**
|
||||
**child_val = > json.get value=`aw` key="value"**
|
||||
**last_obs = > json.get value=`aw` key="observation"**
|
||||
1. `code` == 0
|
||||
> agent_kb_record_hit kb_dir=`kb_dir` goal=`task` improve_every=`improve_every`
|
||||
**exec_mode = "compiled"**
|
||||
**out = > json.parse text={"status":"ok"}**
|
||||
**out = > json.set map=`out` key="status" value="ok"**
|
||||
**out = > json.set map=`out` key="goal" value=`task`**
|
||||
**out = > json.set map=`out` key="result" value=`child_val`**
|
||||
**out = > json.set map=`out` key="workbook" value=`path`**
|
||||
**out = > json.set map=`out` key="cache" value="hit"**
|
||||
**out = > json.set map=`out` key="match" value=`mode`**
|
||||
**sk = > json.get value=`resolved` key="skill"**
|
||||
**out = > json.set map=`out` key="skill" value=`sk`**
|
||||
**out = > json.set map=`out` key="observation" value=`last_obs`**
|
||||
**out = > json.set map=`out` key="summary" value="compiled skill (llm_free)"**
|
||||
**llm_calls = 0**
|
||||
2. *
|
||||
**out = > `self`.plan goal=`task` kb_dir=`kb_dir` writeback=`writeback` reuse=`reuse` near_match=`near_match` near_threshold=`near_threshold` promote=`promote` improve_every=`improve_every` explore_n=`explore_n` max_rounds=`max_rounds` stream=`stream` echo=`echo` trace=`trace` force=`force`**
|
||||
**exec_mode = "explore"**
|
||||
**llm_calls = 1**
|
||||
2. *
|
||||
**out = > `self`.plan goal=`task` kb_dir=`kb_dir` writeback=`writeback` reuse=`reuse` near_match=`near_match` near_threshold=`near_threshold` promote=`promote` improve_every=`improve_every` explore_n=`explore_n` max_rounds=`max_rounds` stream=`stream` echo=`echo` trace=`trace` force=`force`**
|
||||
**cache = > json.get value=`out` key="cache"**
|
||||
1. `cache` == hit
|
||||
**exec_mode = "compiled"**
|
||||
**llm_calls = 0**
|
||||
2. `cache` == soft-hit
|
||||
**exec_mode = "near"**
|
||||
**llm_calls = 0**
|
||||
3. *
|
||||
**exec_mode = "explore"**
|
||||
**llm_calls = 1**
|
||||
|
||||
**st = > json.get value=`out` key="status"**
|
||||
**res = > json.get value=`out` key="result"**
|
||||
**wb_path = > json.get value=`out` key="workbook"**
|
||||
**obs = > json.get value=`out` key="observation"**
|
||||
**sk2 = > json.get value=`out` key="skill"**
|
||||
**cache2 = > json.get value=`out` key="cache"**
|
||||
**match2 = > json.get value=`out` key="match"**
|
||||
**err = > json.get value=`out` key="error"**
|
||||
|
||||
1. `verify`
|
||||
1. `st` == "ok"
|
||||
**verification = "status: passed"**
|
||||
2. *
|
||||
**verification = "status: failed"**
|
||||
2. *
|
||||
**verification = "status: skipped"**
|
||||
|
||||
**ep = > `self`.record_episode task=`task` status=`st` result=`res` mode=`exec_mode` skill=`sk2` observation=`obs` workbook=`wb_path` cache=`cache2` match=`match2` llm_calls=`llm_calls` tokens=`tokens` error=`err` memory_dir=`memory_dir`**
|
||||
|
||||
**learning = None**
|
||||
1. `learn`
|
||||
1. `st` == "ok"
|
||||
**learning = > `self`.maybe_learn task=`task` kb_dir=`kb_dir` memory_dir=`memory_dir` improve_every=`improve_every` promote=`promote`**
|
||||
2. *
|
||||
**learning = None**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
**execution = > json.parse text={"mode":"explore","llm_calls":0,"tokens":0}**
|
||||
**execution = > json.set map=`execution` key="mode" value=`exec_mode`**
|
||||
**execution = > json.set map=`execution` key="llm_calls" value=`llm_calls`**
|
||||
**execution = > json.set map=`execution` key="tokens" value=`tokens`**
|
||||
**execution = > json.set map=`execution` key="skill" value=`sk2`**
|
||||
**execution = > json.set map=`execution` key="router" value=`route_backend`**
|
||||
**out = > json.set map=`out` key="execution" value=`execution`**
|
||||
**out = > json.set map=`out` key="episode" value=`ep`**
|
||||
**out = > json.set map=`out` key="route" value=`routed`**
|
||||
1. `learning`
|
||||
**out = > json.set map=`out` key="learning" value=`learning`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `resume_id`
|
||||
**_ckpt = > resume_save id=`resume_id` goal=`task` task=`task` status=`st` round=1 workbook=`wb_path` result=`res` route=`routed` resume_dir=`resume_dir`**
|
||||
**out = > json.set map=`out` key="resume" value=`_ckpt`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
*out*
|
||||
|
||||
## step
|
||||
+ `task`
|
||||
+ `writeback`=True
|
||||
@@ -1539,17 +2012,20 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
|
||||
**reads_left = max_reads**
|
||||
**reply = None**
|
||||
**decision = None**
|
||||
**llm_calls = 0**
|
||||
**tokens = 0**
|
||||
|
||||
- `reads_left` > 0
|
||||
1. `decision`
|
||||
**reads_left = 0**
|
||||
2. *
|
||||
**ctx = > build_step_context agent=`self` task=`task` source_depth=`source_depth` skill_depth=`skill_depth`**
|
||||
1. `stream`
|
||||
**evs = > model.complete prompt=`ctx` stream=True echo=`echo`**
|
||||
**reply = > llm.stream_result events=`evs`**
|
||||
2. *
|
||||
**reply = > model.complete prompt=`ctx`**
|
||||
**turn = > model_turn model=`model` prompt=`ctx` stream=`stream` echo=`echo`**
|
||||
**reply = > json.get value=`turn` key="text"**
|
||||
**c = > json.get value=`turn` key="llm_calls"**
|
||||
**t = > json.get value=`turn` key="tokens"**
|
||||
**llm_calls = llm_calls + c**
|
||||
**tokens = tokens + t**
|
||||
**reply = > trim value=`reply`**
|
||||
**rk = > extract_plan_read reply=`reply`**
|
||||
1. `rk` == source
|
||||
@@ -1578,11 +2054,12 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
|
||||
**task_s = > json.stringify value=`task`**
|
||||
**fp = "Tool CALL was rejected (not on the tools whitelist). Do NOT emit CALL. Answer the user task directly with the final answer only.\n\nUser task:\n"**
|
||||
**fp = fp + task_s**
|
||||
1. `stream`
|
||||
**evs_retry = > model.complete prompt=`fp` stream=True echo=`echo`**
|
||||
**reply = > llm.stream_result events=`evs_retry`**
|
||||
2. *
|
||||
**reply = > model.complete prompt=`fp`**
|
||||
**turn2 = > model_turn model=`model` prompt=`fp` stream=`stream` echo=`echo`**
|
||||
**reply = > json.get value=`turn2` key="text"**
|
||||
**c = > json.get value=`turn2` key="llm_calls"**
|
||||
**t = > json.get value=`turn2` key="tokens"**
|
||||
**llm_calls = llm_calls + c**
|
||||
**tokens = tokens + t**
|
||||
**out = > json.set map=`out` key="status" value="ok"**
|
||||
**out = > json.set map=`out` key="decision" value=`decision`**
|
||||
**out = > json.set map=`out` key="denied_tool" value=`tool_name`**
|
||||
@@ -1596,17 +2073,21 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
|
||||
**fp = fp + "ran via subtask and returned:"**
|
||||
**fp = fp + tool_s**
|
||||
**fp = fp + ". Reply to the user briefly."**
|
||||
1. `stream`
|
||||
**evs2 = > model.complete prompt=`fp` stream=True echo=`echo`**
|
||||
**reply = > llm.stream_result events=`evs2`**
|
||||
2. *
|
||||
**reply = > model.complete prompt=`fp`**
|
||||
**turn2 = > model_turn model=`model` prompt=`fp` stream=`stream` echo=`echo`**
|
||||
**reply = > json.get value=`turn2` key="text"**
|
||||
**c = > json.get value=`turn2` key="llm_calls"**
|
||||
**t = > json.get value=`turn2` key="tokens"**
|
||||
**llm_calls = llm_calls + c**
|
||||
**tokens = tokens + t**
|
||||
**out = > json.set map=`out` key="tool" value=`tool_name`**
|
||||
**out = > json.set map=`out` key="tool_result" value=`tool_s`**
|
||||
**out = > json.set map=`out` key="result" value=`reply`**
|
||||
2. *
|
||||
**out = > json.set map=`out` key="result" value=`reply`**
|
||||
|
||||
**out = > json.set map=`out` key="llm_calls" value=`llm_calls`**
|
||||
**out = > json.set map=`out` key="tokens" value=`tokens`**
|
||||
|
||||
**as_turn = > json.parse text={"role":"assistant"}**
|
||||
**as_turn = > json.set map=`as_turn` key="content" value=`reply`**
|
||||
> agent_history_append id=`id` item=`as_turn`
|
||||
@@ -1644,6 +2125,8 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
+ `trace`=False
|
||||
+ `resume_id`=None
|
||||
+ `resume_dir`=.marqdo/agent-resume
|
||||
|
||||
Multi-step with OKF agent-kb. Default workbook is `kb_dir/resources/<slug>.mq.md`. While task file count `< explore_n` and skill is not llm_free, force a new explore variant under `kb_dir/explore/<slug>/`. Code-first: llm_free hits skip parent LLM. File children return via `# main`; `plan` exposes that as `result`.
|
||||
|
||||
@@ -1799,11 +2282,12 @@ Reuse lookup: exact → alias → canonicalize → optional local n-gram `near`
|
||||
**soft_prompt = > build_soft_match_prompt goal=`goal` tasks=`tasks`**
|
||||
**model = > json.get value=`self` key="model"**
|
||||
1. `stream`
|
||||
**evs = > model.complete prompt=`soft_prompt` stream=True echo=`echo`**
|
||||
**soft_reply = > llm.stream_result events=`evs`**
|
||||
**evs = > model.stream prompt=`soft_prompt` echo=`echo`**
|
||||
**soft_reply = > llm.collect events=`evs`**
|
||||
**events = > plan_merge_deltas events=`events` from=`evs` stream=`stream`**
|
||||
2. *
|
||||
**soft_reply = > model.complete prompt=`soft_prompt`**
|
||||
**soft_turn = > model_turn model=`model` prompt=`soft_prompt` stream=False echo=`echo`**
|
||||
**soft_reply = > json.get value=`soft_turn` key="text"**
|
||||
**soft_reply = > trim value=`soft_reply`**
|
||||
**soft_dec = > extract_plan_decision reply=`soft_reply`**
|
||||
**events = > plan_append_decision events=`events` decision=`soft_dec` stream=`stream`**
|
||||
@@ -1898,18 +2382,28 @@ Reuse lookup: exact → alias → canonicalize → optional local n-gram `near`
|
||||
**b = > json.get value=`parts` key="b"**
|
||||
**c = > json.get value=`parts` key="c"**
|
||||
**path = workbook_dir + a + slug + b + ts + c**
|
||||
> fs.write_text path=`path` text=`skel`
|
||||
2. `explore`
|
||||
**parts = > json.parse text={"a":"/explore/","b":"/","c":".mq.md"}**
|
||||
**a = > json.get value=`parts` key="a"**
|
||||
**b = > json.get value=`parts` key="b"**
|
||||
**c = > json.get value=`parts` key="c"**
|
||||
**path = kb_dir + a + slug + b + explore_attempt + c**
|
||||
**ex = > fs.exists path=`path`**
|
||||
1. `ex`
|
||||
**_ = 1**
|
||||
2. *
|
||||
> fs.write_text path=`path` text=`skel`
|
||||
3. *
|
||||
**parts = > json.parse text={"a":"/resources/","b":".mq.md"}**
|
||||
**a = > json.get value=`parts` key="a"**
|
||||
**b = > json.get value=`parts` key="b"**
|
||||
**path = kb_dir + a + slug + b**
|
||||
> fs.write_text path=`path` text=`skel`
|
||||
**ex = > fs.exists path=`path`**
|
||||
1. `ex`
|
||||
**_ = 1**
|
||||
2. *
|
||||
> fs.write_text path=`path` text=`skel`
|
||||
|
||||
1. `confirm`
|
||||
**out = > json.parse text={"status":"pending"}**
|
||||
|
||||
+295
-68
@@ -1,23 +1,26 @@
|
||||
---
|
||||
title: ext/ai/llm
|
||||
description: OpenAI-compatible LLM object (English). Import ext/ai/llm.mq.md.
|
||||
description: >-
|
||||
Marqdo LLM intelligence primitive — ask / stream / collect / named handles.
|
||||
OpenAI-compatible and Ollama HTTP stay behind the semantic surface.
|
||||
import sys:lib/sys.mq.md
|
||||
import net:lib/net.mq.md
|
||||
import json:lib/json.mq.md
|
||||
import table:lib/table.mq.md
|
||||
import fs:lib/fs.mq.md
|
||||
import text:lib/text.mq.md
|
||||
import openai:ext/ai/llm/openai.mq.md
|
||||
---
|
||||
|
||||
## load_env
|
||||
+ `path`=None
|
||||
|
||||
Load `.env` from cwd (optional named arg `path=`). Does not override existing variables.
|
||||
Load `.env` from cwd (optional `path=`). Does not override existing variables.
|
||||
|
||||
*> sys.load_dotenv path=`path`*
|
||||
|
||||
## stream_result
|
||||
## collect
|
||||
+ `events`
|
||||
|
||||
Reduce a `complete stream=True` event list to the final `done.result` text (or `error.message`). Unknown types (e.g. `reasoning`) are ignored.
|
||||
Collapse a stream event list to final answer text (`done.result` or `error.message`). Prefer this over raw SSE fields. Alias: `stream_result`.
|
||||
|
||||
**answer = ""**
|
||||
|
||||
@@ -28,97 +31,321 @@ Reduce a `complete stream=True` event list to the final `done.result` text (or `
|
||||
2. `t` == "error"
|
||||
**answer = [message](ev)**
|
||||
3. `t` == "delta"
|
||||
4. *
|
||||
4. `t` == "text"
|
||||
**chunk = [text](ev)**
|
||||
1. `chunk`
|
||||
**answer = answer + chunk**
|
||||
2. *
|
||||
**_ = 1**
|
||||
5. *
|
||||
**_ = 1**
|
||||
|
||||
*answer*
|
||||
|
||||
# llm
|
||||
## stream_result
|
||||
+ `events`
|
||||
|
||||
Construct an LLM handle from `OPENAI_*` / `MARQDO_LLM_*`.
|
||||
Compatibility alias for `collect`.
|
||||
|
||||
**api_key = > sys.env_get name="OPENAI_API_KEY"**
|
||||
1. not `api_key`
|
||||
**api_key = > sys.env_get name="MARQDO_LLM_API_KEY"**
|
||||
*> collect events=`events`*
|
||||
|
||||
1. not `api_key`
|
||||
> print text=ext/ai/llm: set OPENAI_API_KEY or MARQDO_LLM_API_KEY
|
||||
## usage_from
|
||||
+ `data`
|
||||
|
||||
Extract `{prompt_tokens,completion_tokens,total_tokens}` from an OpenAI-shaped response map (or empty zeros).
|
||||
|
||||
*> openai.usage_from data=`data`*
|
||||
|
||||
## create
|
||||
+ `model`=None
|
||||
+ `base_url`=None
|
||||
+ `api_key`=None
|
||||
+ `backend`=openai-compatible
|
||||
+ `name`=None
|
||||
|
||||
Factory for an LLM handle (same as constructing `# llm`). Env fallbacks apply inside the constructor. Optional `name=` tags the handle (`fast` / `reasoning`).
|
||||
|
||||
**h = > llm model=`model` base_url=`base_url` api_key=`api_key` backend=`backend` name=`name`**
|
||||
*h*
|
||||
|
||||
## fast
|
||||
+ `model`=None
|
||||
+ `base_url`=None
|
||||
+ `api_key`=None
|
||||
|
||||
Named small/fast handle. Model from arg, else `MARQDO_LLM_FAST_MODEL`, else default create model.
|
||||
|
||||
1. `model`
|
||||
**mdl = model**
|
||||
2. *
|
||||
**mdl = > sys.env_get name="MARQDO_LLM_FAST_MODEL"**
|
||||
|
||||
*> create model=`mdl` base_url=`base_url` api_key=`api_key` name="fast" backend="openai-compatible"*
|
||||
|
||||
## reasoning
|
||||
+ `model`=None
|
||||
+ `base_url`=None
|
||||
+ `api_key`=None
|
||||
|
||||
Named strong/reasoning handle. Model from arg, else `MARQDO_LLM_REASONING_MODEL`, else default create model.
|
||||
|
||||
1. `model`
|
||||
**mdl = model**
|
||||
2. *
|
||||
**mdl = > sys.env_get name="MARQDO_LLM_REASONING_MODEL"**
|
||||
|
||||
*> create model=`mdl` base_url=`base_url` api_key=`api_key` name="reasoning" backend="openai-compatible"*
|
||||
|
||||
## prompt_body
|
||||
+ `src`
|
||||
|
||||
If `src` starts with YAML frontmatter declaring `type: prompt` / `类型: 提示`, return the body after the closing `---`; otherwise return `src` unchanged.
|
||||
|
||||
**sw = > text.starts_with text=`src` prefix="---"**
|
||||
1. not `sw`
|
||||
*src*
|
||||
2. *
|
||||
**parts = > split value=`src` sep="\n---\n"**
|
||||
**n = > len value=`parts`**
|
||||
1. `n` < 2
|
||||
*src*
|
||||
2. *
|
||||
**fm = > at value=`parts` index=0**
|
||||
**is_p = > text.contains text=`fm` sub="type: prompt"**
|
||||
1. not `is_p`
|
||||
**is_p = > text.contains text=`fm` sub="类型: 提示"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. `is_p`
|
||||
**body = > at value=`parts` index=1**
|
||||
*> text.str_trim s=`body`*
|
||||
2. *
|
||||
*src*
|
||||
|
||||
## prompt_load
|
||||
+ `path`
|
||||
|
||||
Load a prompt artifact (`.mq.md` / `.md`) from disk for `ask`.
|
||||
|
||||
**raw = > fs.read_text path=`path`**
|
||||
*> prompt_body src=`raw`*
|
||||
|
||||
## prompt
|
||||
+ `path`=None
|
||||
+ `text`=None
|
||||
|
||||
Prompt artifact helper: `path=` loads a file; `text=` strips `type: prompt` frontmatter from an in-memory string.
|
||||
|
||||
1. `path`
|
||||
*> prompt_load path=`path`*
|
||||
2. `text`
|
||||
*> prompt_body src=`text`*
|
||||
3. *
|
||||
> print text=ext/ai/llm.prompt: pass path= or text=
|
||||
> sys.exit code=1
|
||||
|
||||
**base_url = > sys.env_get name="OPENAI_BASE_URL"**
|
||||
1. not `base_url`
|
||||
**base_url = > sys.env_get name="MARQDO_LLM_BASE_URL"**
|
||||
1. not `base_url`
|
||||
**base_url = "https://api.openai.com/v1"**
|
||||
## ask
|
||||
+ `prompt`=None
|
||||
+ `path`=None
|
||||
+ `model`=None
|
||||
+ `base_url`=None
|
||||
+ `api_key`=None
|
||||
|
||||
**model = > sys.env_get name="OPENAI_MODEL"**
|
||||
1. not `model`
|
||||
**model = > sys.env_get name="MARQDO_LLM_MODEL"**
|
||||
1. not `model`
|
||||
**model = "gpt-4o-mini"**
|
||||
Module convenience: create a default handle and return **answer text**. Pass `path=` to load a prompt document, `prompt=` text, or omit both to use the entry `.mq.md` as a prompt artifact (`type: prompt` body via `sys.module_source`).
|
||||
|
||||
1. `path`
|
||||
**p = > prompt_load path=`path`**
|
||||
2. `prompt`
|
||||
**p = prompt**
|
||||
3. *
|
||||
**raw = > sys.module_source**
|
||||
1. not `raw`
|
||||
> print text=ext/ai/llm.ask: pass prompt= or path= (or run a prompt document as entry)
|
||||
> sys.exit code=1
|
||||
2. *
|
||||
**p = > prompt_body src=`raw`**
|
||||
|
||||
**m = > create model=`model` base_url=`base_url` api_key=`api_key`**
|
||||
**r = > m.ask prompt=`p`**
|
||||
*[text](r)*
|
||||
|
||||
# llm
|
||||
+ `model`=None
|
||||
+ `base_url`=None
|
||||
+ `api_key`=None
|
||||
+ `backend`=openai-compatible
|
||||
+ `name`=None
|
||||
|
||||
LLM handle — intelligence primitive, not an HTTP client.
|
||||
|
||||
Resolution: explicit args → entry-scope / Artifact Metadata (`sys.meta_get`) → `sys.env_get` → defaults.
|
||||
Prefer declaring `model` / `api_key` / `base_url` in frontmatter Binding, then use the lifted names.
|
||||
|
||||
1. `api_key`
|
||||
**key = api_key**
|
||||
2. *
|
||||
**key = > sys.meta_get key="api_key"**
|
||||
1. not `key`
|
||||
**key = > sys.env_get name="OPENAI_API_KEY"**
|
||||
1. not `key`
|
||||
**key = > sys.env_get name="MARQDO_LLM_API_KEY"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `backend` == ollama
|
||||
1. not `key`
|
||||
**key = "ollama"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. not `key`
|
||||
> print text=ext/ai/llm: set OPENAI_API_KEY or MARQDO_LLM_API_KEY (or pass api_key= / metadata api_key)
|
||||
> sys.exit code=1
|
||||
3. *
|
||||
**_ = 1**
|
||||
|
||||
1. `base_url`
|
||||
**url = base_url**
|
||||
2. *
|
||||
**url = > sys.meta_get key="base_url"**
|
||||
1. `url`
|
||||
**_ = 1**
|
||||
2. `backend` == ollama
|
||||
**url = > sys.env_get name="OLLAMA_HOST"**
|
||||
1. not `url`
|
||||
**url = "http://127.0.0.1:11434/v1"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
3. *
|
||||
**url = > sys.env_get name="OPENAI_BASE_URL"**
|
||||
1. not `url`
|
||||
**url = > sys.env_get name="MARQDO_LLM_BASE_URL"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. not `url`
|
||||
**url = "https://api.openai.com/v1"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `model`
|
||||
**mdl = model**
|
||||
2. *
|
||||
**mdl = > sys.meta_get key="model"**
|
||||
1. `mdl`
|
||||
**_ = 1**
|
||||
2. *
|
||||
**mdl = > sys.env_get name="OPENAI_MODEL"**
|
||||
1. not `mdl`
|
||||
**mdl = > sys.env_get name="MARQDO_LLM_MODEL"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. not `mdl`
|
||||
1. `backend` == ollama
|
||||
**mdl = "llama3.2"**
|
||||
2. *
|
||||
**mdl = "gpt-4o-mini"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `name`
|
||||
**nm = name**
|
||||
2. *
|
||||
**nm = None**
|
||||
|
||||
`h` =
|
||||
|
||||
| api_key | base_url | model | suffix | bearer |
|
||||
|---------|----------|-------|--------|--------|
|
||||
| `api_key` | `base_url` | `model` | /chat/completions | "Bearer " |
|
||||
| _type | backend | api_key | base_url | model | suffix | bearer | name |
|
||||
|-------|---------|---------|----------|-------|--------|--------|------|
|
||||
| llm | `backend` | `key` | `url` | `mdl` | /chat/completions | "Bearer " | `nm` |
|
||||
|
||||
*h*
|
||||
|
||||
## ask
|
||||
+ `prompt`=None
|
||||
+ `path`=None
|
||||
|
||||
Semantic ask → **LLMResult** (`text`, `model`, `finish`, `backend`, `usage`, `name`, `tool_calls`). Prefer `[text](result)` for the answer string. Omit `prompt=` / `path=` to use the entry document (`sys.module_source` + `prompt_body`).
|
||||
|
||||
1. `path`
|
||||
**p = > prompt_load path=`path`**
|
||||
2. `prompt`
|
||||
**p = prompt**
|
||||
3. *
|
||||
**raw = > sys.module_source**
|
||||
1. not `raw`
|
||||
> print text=ext/ai/llm: ask needs prompt= or path= (or entry prompt document)
|
||||
> sys.exit code=1
|
||||
2. *
|
||||
**p = > prompt_body src=`raw`**
|
||||
|
||||
**pack = > self.complete_result prompt=`p` stream=False echo=False**
|
||||
**text = > json.get value=`pack` key="text"**
|
||||
**usage = > json.get value=`pack` key="usage"**
|
||||
**finish = > json.get value=`pack` key="finish"**
|
||||
**tc = > json.get value=`pack` key="tool_calls"**
|
||||
1. not `tc`
|
||||
**tc = > json.parse text=[]**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
`result` =
|
||||
|
||||
| text | model | finish | backend | usage | name | llm_calls | tokens | tool_calls |
|
||||
|------|-------|--------|---------|-------|------|-----------|--------|------------|
|
||||
| `text` | [model](self) | `finish` | [backend](self) | `usage` | [name](self) | 1 | [total_tokens](usage) | `tc` |
|
||||
|
||||
*result*
|
||||
|
||||
## stream
|
||||
+ `prompt`=None
|
||||
+ `path`=None
|
||||
+ `echo`=False
|
||||
|
||||
Semantic stream: return the event list (use `llm.collect` to reduce to text). Omit `prompt=` / `path=` to use the entry document.
|
||||
|
||||
1. `path`
|
||||
**p = > prompt_load path=`path`**
|
||||
2. `prompt`
|
||||
**p = prompt**
|
||||
3. *
|
||||
**raw = > sys.module_source**
|
||||
1. not `raw`
|
||||
> print text=ext/ai/llm: stream needs prompt= or path= (or entry prompt document)
|
||||
> sys.exit code=1
|
||||
2. *
|
||||
**p = > prompt_body src=`raw`**
|
||||
|
||||
*> self.complete prompt=`p` stream=True echo=`echo`*
|
||||
|
||||
## complete_result
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
|
||||
Transport pack via `ext/ai/llm/openai` (Ollama uses the same OpenAI-compatible path). Non-stream → `{text,usage,finish}`; stream → `{events}`.
|
||||
|
||||
*> openai.chat_completions base_url=[base_url](self) api_key=[api_key](self) model=[model](self) prompt=`prompt` stream=`stream` echo=`echo` suffix=[suffix](self) bearer=[bearer](self)*
|
||||
|
||||
## complete
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
|
||||
Chat completion using `self` handle fields.
|
||||
|
||||
Wire body is `{model, messages:[{role,content}], stream?}`. Messages are an `@` row table; headers and optional `stream` use `table.put`. `json` is only for stringify/parse.
|
||||
|
||||
**url = [base_url](self) + [suffix](self)**
|
||||
**auth = [bearer](self) + [api_key](self)**
|
||||
**headers = > table.put in=None at="Authorization" value=`auth`**
|
||||
|
||||
`messages` =
|
||||
|
||||
| @ | role | content |
|
||||
|---|------|---------|
|
||||
| 1 | user | `prompt` |
|
||||
|
||||
`req` =
|
||||
|
||||
| model | messages |
|
||||
|-------|----------|
|
||||
| [model](self) | `messages` |
|
||||
Internal / compatibility primitive. Prefer `ask` / `stream` in new code.
|
||||
|
||||
1. `stream`
|
||||
**req = > table.put in=`req` at="stream" value=True**
|
||||
**body = > json.stringify value=`req`**
|
||||
**resp = > net.http_post_sse url=`url` body=`body` headers=`headers` echo=`echo`**
|
||||
1. [status](resp) == 200
|
||||
*[events](resp)*
|
||||
2. *
|
||||
> print text=ext/ai/llm: HTTP error (stream)
|
||||
> print text=[status](resp)
|
||||
> sys.exit code=1
|
||||
**pack = > self.complete_result prompt=`prompt` stream=True echo=`echo`**
|
||||
*[events](pack)*
|
||||
2. *
|
||||
**body = > json.stringify value=`req`**
|
||||
**resp = > net.http_post url=`url` body=`body` headers=`headers`**
|
||||
1. [status](resp) == 200
|
||||
**data = > json.parse text=[body](resp)**
|
||||
*[content]([message]([1]([choices](data))))*
|
||||
2. *
|
||||
> print text=ext/ai/llm: HTTP error
|
||||
> print text=[status](resp)
|
||||
> print text=[body](resp)
|
||||
> sys.exit code=1
|
||||
|
||||
---
|
||||
**pack = > self.complete_result prompt=`prompt` stream=False echo=`echo`**
|
||||
*[text](pack)*
|
||||
|
||||
## chat
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
|
||||
Alias for `complete`.
|
||||
Alias for `complete` (compat).
|
||||
|
||||
*> self.complete prompt=`prompt` stream=`stream` echo=`echo`*
|
||||
@@ -0,0 +1,55 @@
|
||||
---
|
||||
title: ext/ai/llm/ollama
|
||||
description: >-
|
||||
Ollama backend defaults. Transport reuses openai-compatible /v1 chat completions.
|
||||
import openai:ext/ai/llm/openai.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
---
|
||||
|
||||
## resolve
|
||||
+ `model`=None
|
||||
+ `base_url`=None
|
||||
+ `api_key`=None
|
||||
|
||||
Resolve Ollama handle fields: default host `http://127.0.0.1:11434/v1`, key `ollama`, model `llama3.2`.
|
||||
|
||||
1. `api_key`
|
||||
**key = api_key**
|
||||
2. *
|
||||
**key = "ollama"**
|
||||
|
||||
1. `base_url`
|
||||
**url = base_url**
|
||||
2. *
|
||||
**url = > sys.env_get name="OLLAMA_HOST"**
|
||||
1. not `url`
|
||||
**url = "http://127.0.0.1:11434/v1"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `model`
|
||||
**mdl = model**
|
||||
2. *
|
||||
**mdl = > sys.env_get name="MARQDO_LLM_MODEL"**
|
||||
1. not `mdl`
|
||||
**mdl = "llama3.2"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
`cfg` =
|
||||
|
||||
| backend | api_key | base_url | model |
|
||||
|---------|---------|----------|-------|
|
||||
| ollama | `key` | `url` | `mdl` |
|
||||
|
||||
*cfg*
|
||||
|
||||
## chat_completions
|
||||
+ `base_url`
|
||||
+ `api_key`
|
||||
+ `model`
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
|
||||
*> openai.chat_completions base_url=`base_url` api_key=`api_key` model=`model` prompt=`prompt` stream=`stream` echo=`echo`*
|
||||
@@ -0,0 +1,145 @@
|
||||
---
|
||||
title: ext/ai/llm/openai
|
||||
description: >-
|
||||
OpenAI-compatible chat completions transport (HTTP + SSE).
|
||||
Prefers optional plugins/llm ABI when present; falls back to lib/net.
|
||||
Used by ext/ai/llm; authors should prefer llm.ask / llm.stream.
|
||||
import net:lib/net.mq.md
|
||||
import json:lib/json.mq.md
|
||||
import table:lib/table.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
import plugin:lib/plugin.mq.md
|
||||
---
|
||||
|
||||
## usage_from
|
||||
+ `data`
|
||||
|
||||
Extract `{prompt_tokens,completion_tokens,total_tokens}` from an OpenAI-shaped response map.
|
||||
|
||||
**u = > json.get value=`data` key="usage"**
|
||||
1. `u`
|
||||
**pt = > json.get value=`u` key="prompt_tokens"**
|
||||
**ct = > json.get value=`u` key="completion_tokens"**
|
||||
**tt = > json.get value=`u` key="total_tokens"**
|
||||
1. not `pt`
|
||||
**pt = 0**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. not `ct`
|
||||
**ct = 0**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. not `tt`
|
||||
**tt = pt + ct**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**pt = 0**
|
||||
**ct = 0**
|
||||
**tt = 0**
|
||||
|
||||
`usage` =
|
||||
|
||||
| prompt_tokens | completion_tokens | total_tokens |
|
||||
|---------------|-------------------|--------------|
|
||||
| `pt` | `ct` | `tt` |
|
||||
|
||||
*usage*
|
||||
|
||||
## tool_calls_from
|
||||
+ `data`
|
||||
|
||||
Extract OpenAI-shaped `choices[0].message.tool_calls` as a list (or empty list).
|
||||
|
||||
**choices = > json.get value=`data` key="choices"**
|
||||
1. not `choices`
|
||||
**empty = > json.parse text=[]**
|
||||
*empty*
|
||||
2. *
|
||||
**c0 = [1](choices)**
|
||||
**msg = > json.get value=`c0` key="message"**
|
||||
1. not `msg`
|
||||
**empty = > json.parse text=[]**
|
||||
*empty*
|
||||
2. *
|
||||
**tc = > json.get value=`msg` key="tool_calls"**
|
||||
1. `tc`
|
||||
*tc*
|
||||
2. *
|
||||
**empty = > json.parse text=[]**
|
||||
*empty*
|
||||
|
||||
## chat_completions
|
||||
+ `base_url`
|
||||
+ `api_key`
|
||||
+ `model`
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
+ `suffix`=/chat/completions
|
||||
+ `bearer`="Bearer "
|
||||
|
||||
POST `/chat/completions` (or stream SSE). Prefers `plugins/llm` ABI (`llm_chat_completions`) when the native plugin is resolvable; otherwise `lib/net` HTTP/SSE. Non-stream → `{text,usage,finish,tool_calls}`; stream → `{events,finish}`.
|
||||
|
||||
**p = > plugin.native_path name="llm"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
*> llm_chat_completions base_url=`base_url` api_key=`api_key` model=`model` prompt=`prompt` stream=`stream` echo=`echo` suffix=`suffix` bearer=`bearer`*
|
||||
2. *
|
||||
**url = base_url + suffix**
|
||||
**auth = bearer + api_key**
|
||||
**headers = > table.put in=None at="Authorization" value=`auth`**
|
||||
|
||||
`messages` =
|
||||
|
||||
| @ | role | content |
|
||||
|---|------|---------|
|
||||
| 1 | user | `prompt` |
|
||||
|
||||
`req` =
|
||||
|
||||
| model | messages |
|
||||
|-------|----------|
|
||||
| `model` | `messages` |
|
||||
|
||||
1. `stream`
|
||||
**req = > table.put in=`req` at="stream" value=True**
|
||||
**body = > json.stringify value=`req`**
|
||||
**resp = > net.http_post_sse url=`url` body=`body` headers=`headers` echo=`echo`**
|
||||
1. [status](resp) == 200
|
||||
**pack = > json.parse text={"finish":"stop"}**
|
||||
**pack = > json.set map=`pack` key="events" value=[events](resp)**
|
||||
*pack*
|
||||
2. *
|
||||
> print text=ext/ai/llm/openai: HTTP error (stream)
|
||||
> print text=[status](resp)
|
||||
> sys.exit code=1
|
||||
2. *
|
||||
**body = > json.stringify value=`req`**
|
||||
**resp = > net.http_post url=`url` body=`body` headers=`headers`**
|
||||
1. [status](resp) == 200
|
||||
**data = > json.parse text=[body](resp)**
|
||||
**msg = [message]([1]([choices](data)))**
|
||||
**text = > json.get value=`msg` key="content"**
|
||||
1. not `text`
|
||||
**text = ""**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**usage = > usage_from data=`data`**
|
||||
**fr = > json.get value=[1]([choices](data)) key="finish_reason"**
|
||||
1. not `fr`
|
||||
**fr = "stop"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**tc = > tool_calls_from data=`data`**
|
||||
**pack = > json.parse text={"finish":"stop"}**
|
||||
**pack = > json.set map=`pack` key="text" value=`text`**
|
||||
**pack = > json.set map=`pack` key="usage" value=`usage`**
|
||||
**pack = > json.set map=`pack` key="finish" value=`fr`**
|
||||
**pack = > json.set map=`pack` key="tool_calls" value=`tc`**
|
||||
*pack*
|
||||
2. *
|
||||
> print text=ext/ai/llm/openai: HTTP error
|
||||
> print text=[status](resp)
|
||||
> print text=[body](resp)
|
||||
> sys.exit code=1
|
||||
+238
-72
@@ -1,24 +1,23 @@
|
||||
---
|
||||
title: ext/ai/大模型
|
||||
description: OpenAI 兼容大模型对象(中文)。导入 ext/ai/大模型.mq.md。
|
||||
description: >-
|
||||
Marqdo 大模型智能原语 — 提问 / 流式 / 收集 / 命名句柄。
|
||||
OpenAI 兼容与 Ollama HTTP 藏在语义面之后。
|
||||
导入 系统:lib/系统.mq.md
|
||||
导入 网络:lib/网络.mq.md
|
||||
import json:lib/json.mq.md
|
||||
导入 表:lib/表.mq.md
|
||||
导入 文件:lib/文件.mq.md
|
||||
导入 文本:lib/文本.mq.md
|
||||
import openai:ext/ai/llm/openai.mq.md
|
||||
---
|
||||
|
||||
## 加载环境
|
||||
+ `路径`=None
|
||||
|
||||
从工作目录加载 `.env`(可选命名实参 `路径=`)。不覆盖已有变量。
|
||||
|
||||
*> 系统.加载环境 路径=`路径`*
|
||||
|
||||
## 流式结果
|
||||
## 收集
|
||||
+ `事件`
|
||||
|
||||
把 `运行 流式=真` 的事件列表收成最终 `done.result`(或 `error.message`)。未知类型(如 `reasoning`)忽略。
|
||||
|
||||
**answer = ""**
|
||||
|
||||
- [ev](事件)
|
||||
@@ -28,97 +27,264 @@ import json:lib/json.mq.md
|
||||
2. `t` == "error"
|
||||
**answer = [message](ev)**
|
||||
3. `t` == "delta"
|
||||
4. *
|
||||
4. `t` == "text"
|
||||
**chunk = [text](ev)**
|
||||
1. `chunk`
|
||||
**answer = answer + chunk**
|
||||
2. *
|
||||
**_ = 1**
|
||||
5. *
|
||||
**_ = 1**
|
||||
|
||||
*answer*
|
||||
|
||||
# 大模型
|
||||
## 流式结果
|
||||
+ `事件`
|
||||
|
||||
从 `OPENAI_*` / `MARQDO_LLM_*` 构造句柄。
|
||||
*> 收集 事件=`事件`*
|
||||
|
||||
**api_key = > 系统.取环境 名="OPENAI_API_KEY"**
|
||||
1. 非 `api_key`
|
||||
**api_key = > 系统.取环境 名="MARQDO_LLM_API_KEY"**
|
||||
## 用量
|
||||
+ `数据`
|
||||
|
||||
1. 非 `api_key`
|
||||
> 打印 内容=ext/ai/大模型: 请设置 OPENAI_API_KEY 或 MARQDO_LLM_API_KEY
|
||||
*> openai.usage_from data=`数据`*
|
||||
|
||||
## 创建
|
||||
+ `模型`=None
|
||||
+ `基址`=None
|
||||
+ `密钥`=None
|
||||
+ `后端`=openai-compatible
|
||||
+ `名`=None
|
||||
|
||||
**h = > 大模型 模型=`模型` 基址=`基址` 密钥=`密钥` 后端=`后端` 名=`名`**
|
||||
*h*
|
||||
|
||||
## 快速
|
||||
+ `模型`=None
|
||||
+ `基址`=None
|
||||
+ `密钥`=None
|
||||
|
||||
1. `模型`
|
||||
**mdl = 模型**
|
||||
2. *
|
||||
**mdl = > 系统.取环境 名="MARQDO_LLM_FAST_MODEL"**
|
||||
|
||||
*> 创建 模型=`mdl` 基址=`基址` 密钥=`密钥` 名="fast" 后端="openai-compatible"*
|
||||
|
||||
## 推理
|
||||
+ `模型`=None
|
||||
+ `基址`=None
|
||||
+ `密钥`=None
|
||||
|
||||
1. `模型`
|
||||
**mdl = 模型**
|
||||
2. *
|
||||
**mdl = > 系统.取环境 名="MARQDO_LLM_REASONING_MODEL"**
|
||||
|
||||
*> 创建 模型=`mdl` 基址=`基址` 密钥=`密钥` 名="reasoning" 后端="openai-compatible"*
|
||||
|
||||
## 提示正文
|
||||
+ `源`
|
||||
|
||||
**sw = > 文本.开头是 内容=`源` 前缀="---"**
|
||||
1. 非 `sw`
|
||||
*源*
|
||||
2. *
|
||||
**parts = > split value=`源` sep="\n---\n"**
|
||||
**n = > len value=`parts`**
|
||||
1. `n` < 2
|
||||
*源*
|
||||
2. *
|
||||
**fm = > at value=`parts` index=0**
|
||||
**is_p = > 文本.包含 内容=`fm` 子串="type: prompt"**
|
||||
1. 非 `is_p`
|
||||
**is_p = > 文本.包含 内容=`fm` 子串="类型: 提示"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. `is_p`
|
||||
**body = > at value=`parts` index=1**
|
||||
*> 文本.去空白 s=`body`*
|
||||
2. *
|
||||
*源*
|
||||
|
||||
## 加载提示
|
||||
+ `路径`
|
||||
|
||||
**raw = > 文件.读文本 路径=`路径`**
|
||||
*> 提示正文 源=`raw`*
|
||||
|
||||
## 提示
|
||||
+ `路径`=None
|
||||
+ `文本`=None
|
||||
|
||||
1. `路径`
|
||||
*> 加载提示 路径=`路径`*
|
||||
2. `文本`
|
||||
*> 提示正文 源=`文本`*
|
||||
3. *
|
||||
> 打印 内容=ext/ai/大模型.提示: 请传 路径= 或 文本=
|
||||
> 系统.退出 码=1
|
||||
|
||||
**base_url = > 系统.取环境 名="OPENAI_BASE_URL"**
|
||||
1. 非 `base_url`
|
||||
**base_url = > 系统.取环境 名="MARQDO_LLM_BASE_URL"**
|
||||
1. 非 `base_url`
|
||||
**base_url = "https://api.openai.com/v1"**
|
||||
## 提问
|
||||
+ `提示`=None
|
||||
+ `路径`=None
|
||||
+ `模型`=None
|
||||
+ `基址`=None
|
||||
+ `密钥`=None
|
||||
|
||||
**model = > 系统.取环境 名="OPENAI_MODEL"**
|
||||
1. 非 `model`
|
||||
**model = > 系统.取环境 名="MARQDO_LLM_MODEL"**
|
||||
1. 非 `model`
|
||||
**model = "gpt-4o-mini"**
|
||||
1. `路径`
|
||||
**p = > 加载提示 路径=`路径`**
|
||||
2. `提示`
|
||||
**p = 提示**
|
||||
3. *
|
||||
> 打印 内容=ext/ai/大模型.提问: 请传 提示= 或 路径=
|
||||
> 系统.退出 码=1
|
||||
|
||||
**m = > 创建 模型=`模型` 基址=`基址` 密钥=`密钥`**
|
||||
**r = > m.提问 提示=`p`**
|
||||
*[text](r)*
|
||||
|
||||
# 大模型
|
||||
+ `模型`=None
|
||||
+ `基址`=None
|
||||
+ `密钥`=None
|
||||
+ `后端`=openai-compatible
|
||||
+ `名`=None
|
||||
|
||||
1. `密钥`
|
||||
**key = 密钥**
|
||||
2. *
|
||||
**key = > 系统.取环境 名="OPENAI_API_KEY"**
|
||||
1. 非 `key`
|
||||
**key = > 系统.取环境 名="MARQDO_LLM_API_KEY"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `后端` == ollama
|
||||
1. 非 `key`
|
||||
**key = "ollama"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. 非 `key`
|
||||
> 打印 内容=ext/ai/大模型: 请设置 OPENAI_API_KEY 或 MARQDO_LLM_API_KEY
|
||||
> 系统.退出 码=1
|
||||
3. *
|
||||
**_ = 1**
|
||||
|
||||
1. `基址`
|
||||
**url = 基址**
|
||||
2. `后端` == ollama
|
||||
**url = > 系统.取环境 名="OLLAMA_HOST"**
|
||||
1. 非 `url`
|
||||
**url = "http://127.0.0.1:11434/v1"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
3. *
|
||||
**url = > 系统.取环境 名="OPENAI_BASE_URL"**
|
||||
1. 非 `url`
|
||||
**url = > 系统.取环境 名="MARQDO_LLM_BASE_URL"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. 非 `url`
|
||||
**url = "https://api.openai.com/v1"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `模型`
|
||||
**mdl = 模型**
|
||||
2. *
|
||||
**mdl = > 系统.取环境 名="OPENAI_MODEL"**
|
||||
1. 非 `mdl`
|
||||
**mdl = > 系统.取环境 名="MARQDO_LLM_MODEL"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. 非 `mdl`
|
||||
1. `后端` == ollama
|
||||
**mdl = "llama3.2"**
|
||||
2. *
|
||||
**mdl = "gpt-4o-mini"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `名`
|
||||
**nm = 名**
|
||||
2. *
|
||||
**nm = None**
|
||||
|
||||
`h` =
|
||||
|
||||
| api_key | base_url | model | suffix | bearer |
|
||||
|---------|----------|-------|--------|--------|
|
||||
| `api_key` | `base_url` | `model` | /chat/completions | "Bearer " |
|
||||
| _type | backend | api_key | base_url | model | suffix | bearer | name |
|
||||
|-------|---------|---------|----------|-------|--------|--------|------|
|
||||
| 大模型 | `后端` | `key` | `url` | `mdl` | /chat/completions | "Bearer " | `nm` |
|
||||
|
||||
*h*
|
||||
|
||||
## 提问
|
||||
+ `提示`=None
|
||||
+ `路径`=None
|
||||
|
||||
1. `路径`
|
||||
**p = > 加载提示 路径=`路径`**
|
||||
2. `提示`
|
||||
**p = 提示**
|
||||
3. *
|
||||
> 打印 内容=ext/ai/大模型: 提问需要 提示= 或 路径=
|
||||
> 系统.退出 码=1
|
||||
|
||||
**pack = > 自.完整结果 提示=`p` 流式=False 打印增量=False**
|
||||
**text = > json.get value=`pack` key="text"**
|
||||
**usage = > json.get value=`pack` key="usage"**
|
||||
**finish = > json.get value=`pack` key="finish"**
|
||||
**tc = > json.get value=`pack` key="tool_calls"**
|
||||
1. not `tc`
|
||||
**tc = > json.parse text=[]**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
`result` =
|
||||
|
||||
| text | model | finish | backend | usage | name | llm_calls | tokens | tool_calls |
|
||||
|------|-------|--------|---------|-------|------|-----------|--------|------------|
|
||||
| `text` | [model](自) | `finish` | [backend](自) | `usage` | [name](自) | 1 | [total_tokens](usage) | `tc` |
|
||||
|
||||
*result*
|
||||
|
||||
## 流式
|
||||
+ `提示`=None
|
||||
+ `路径`=None
|
||||
+ `打印增量`=False
|
||||
|
||||
1. `路径`
|
||||
**p = > 加载提示 路径=`路径`**
|
||||
2. `提示`
|
||||
**p = 提示**
|
||||
3. *
|
||||
> 打印 内容=ext/ai/大模型: 流式需要 提示= 或 路径=
|
||||
> 系统.退出 码=1
|
||||
|
||||
*> 自.运行 提示=`p` 流式=True 打印增量=`打印增量`*
|
||||
|
||||
## 完整结果
|
||||
+ `提示`
|
||||
+ `流式`=False
|
||||
+ `打印增量`=False
|
||||
|
||||
*> openai.chat_completions base_url=[base_url](自) api_key=[api_key](自) model=[model](自) prompt=`提示` stream=`流式` echo=`打印增量` suffix=[suffix](自) bearer=[bearer](自)*
|
||||
|
||||
## 运行
|
||||
+ `提示`
|
||||
+ `流式`=False
|
||||
+ `打印增量`=False
|
||||
|
||||
使用 `自` 句柄字段做聊天补全。
|
||||
|
||||
线上 body 为 `{model, messages:[{role,content}], stream?}`。消息用 `@` 行向表;headers 与可选 `stream` 用 `表.改`。`json` 只做 stringify/parse。
|
||||
|
||||
**url = [base_url](自) + [suffix](自)**
|
||||
**auth = [bearer](自) + [api_key](自)**
|
||||
**headers = > 表.改 入=None 于="Authorization" 值=`auth`**
|
||||
|
||||
`messages` =
|
||||
|
||||
| @ | role | content |
|
||||
|---|------|---------|
|
||||
| 1 | user | `提示` |
|
||||
|
||||
`req` =
|
||||
|
||||
| model | messages |
|
||||
|-------|----------|
|
||||
| [model](自) | `messages` |
|
||||
|
||||
1. `流式`
|
||||
**req = > 表.改 入=`req` 于="stream" 值=True**
|
||||
**body = > json.stringify value=`req`**
|
||||
**resp = > 网络.提交流式 地址=`url` 内容=`body` headers=`headers` 打印增量=`打印增量`**
|
||||
1. [status](resp) == 200
|
||||
*[events](resp)*
|
||||
2. *
|
||||
> 打印 内容=ext/ai/大模型: HTTP 错误(流式)
|
||||
> 打印 内容=[status](resp)
|
||||
> 系统.退出 码=1
|
||||
**pack = > 自.完整结果 提示=`提示` 流式=True 打印增量=`打印增量`**
|
||||
*[events](pack)*
|
||||
2. *
|
||||
**body = > json.stringify value=`req`**
|
||||
**resp = > 网络.提交 地址=`url` 内容=`body` headers=`headers`**
|
||||
1. [status](resp) == 200
|
||||
**data = > json.parse text=[body](resp)**
|
||||
*[content]([message]([1]([choices](data))))*
|
||||
2. *
|
||||
> 打印 内容=ext/ai/大模型: HTTP 错误
|
||||
> 打印 内容=[status](resp)
|
||||
> 打印 内容=[body](resp)
|
||||
> 系统.退出 码=1
|
||||
|
||||
---
|
||||
**pack = > 自.完整结果 提示=`提示` 流式=False 打印增量=`打印增量`**
|
||||
*[text](pack)*
|
||||
|
||||
## 聊天
|
||||
+ `提示`
|
||||
+ `流式`=False
|
||||
+ `打印增量`=False
|
||||
|
||||
运行 的别名。
|
||||
|
||||
*> 自.运行 提示=`提示` 流式=`流式` 打印增量=`打印增量`*
|
||||
+110
-1
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: ext/ai/智能体
|
||||
description: 文档驱动智能体 — 单步 / 多步;工具经 lib/子任务;运行时经 ABI v2 agent 插件。
|
||||
description: 文档驱动智能体 v2 — 单步 / 多步 / 运行;技能编译经 ABI v2 agent 插件(情景 → 学习 → llm_free)。
|
||||
导入 大模型:ext/ai/大模型.mq.md
|
||||
import agent:ext/ai/agent.mq.md
|
||||
import json:lib/json.mq.md
|
||||
@@ -212,3 +212,112 @@ import json:lib/json.mq.md
|
||||
**standing = > json.get value=`self` key="standing"**
|
||||
**en = > agent.agent model=`model` tools=`tools` standing=`standing`**
|
||||
*> en.plan goal=`目标` workbook_dir=`工作簿目录` workbook=`工作簿` max_rounds=`最大轮次` confirm=`确认分解` writeback=`写回` skeleton=`骨架` reuse=`复用` optimize=`优化` force=`强制` soft_match=`软命中` near_match=`近义命中` near_threshold=`近义阈值` promote=`晋升` kb_dir=`知识库目录` improve_every=`改进周期` explore_n=`探索次数` stream=`流式` echo=`打印增量` trace=`轨迹`*
|
||||
|
||||
## 解析
|
||||
+ `任务`
|
||||
+ `知识库目录`=.marqdo/agent-kb
|
||||
+ `近义命中`=True
|
||||
+ `近义阈值`=0.78
|
||||
|
||||
技能解析(v2):exact → alias → canonical → near。
|
||||
|
||||
**model = > json.get value=`self` key="model"**
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
**standing = > json.get value=`self` key="standing"**
|
||||
**en = > agent.agent model=`model` tools=`tools` standing=`standing`**
|
||||
*> en.resolve task=`任务` kb_dir=`知识库目录` near_match=`近义命中` near_threshold=`近义阈值`*
|
||||
|
||||
## 记录情景
|
||||
+ `任务`
|
||||
+ `状态`=ok
|
||||
+ `结果`=None
|
||||
+ `模式`=explore
|
||||
+ `技能`=none
|
||||
+ `观察`=None
|
||||
+ `工作簿`=None
|
||||
+ `缓存`=None
|
||||
+ `匹配`=None
|
||||
+ `大模型次数`=0
|
||||
+ `工具次数`=0
|
||||
+ `错误`=None
|
||||
+ `记忆目录`=.marqdo/agent-memory
|
||||
|
||||
将一次执行沉淀为 `.mq.md` 情景记忆。
|
||||
|
||||
**model = > json.get value=`self` key="model"**
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
**standing = > json.get value=`self` key="standing"**
|
||||
**en = > agent.agent model=`model` tools=`tools` standing=`standing`**
|
||||
*> en.record_episode task=`任务` status=`状态` result=`结果` mode=`模式` skill=`技能` observation=`观察` workbook=`工作簿` cache=`缓存` match=`匹配` llm_calls=`大模型次数` tool_calls=`工具次数` error=`错误` memory_dir=`记忆目录`*
|
||||
|
||||
## 尝试学习
|
||||
+ `任务`
|
||||
+ `知识库目录`=.marqdo/agent-kb
|
||||
+ `记忆目录`=.marqdo/agent-memory
|
||||
+ `改进周期`=3
|
||||
+ `晋升`=True
|
||||
|
||||
情景足够时:归纳 → 合成候选 → 编译 → 可选晋升为 `llm_free` 技能。
|
||||
|
||||
**model = > json.get value=`self` key="model"**
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
**standing = > json.get value=`self` key="standing"**
|
||||
**en = > agent.agent model=`model` tools=`tools` standing=`standing`**
|
||||
*> en.maybe_learn task=`任务` kb_dir=`知识库目录` memory_dir=`记忆目录` improve_every=`改进周期` promote=`晋升`*
|
||||
|
||||
## 运行
|
||||
+ `任务`
|
||||
+ `知识库目录`=.marqdo/agent-kb
|
||||
+ `记忆目录`=.marqdo/agent-memory
|
||||
+ `学习`=True
|
||||
+ `校验`=True
|
||||
+ `写回`=True
|
||||
+ `复用`=True
|
||||
+ `近义命中`=True
|
||||
+ `近义阈值`=0.78
|
||||
+ `晋升`=True
|
||||
+ `改进周期`=3
|
||||
+ `探索次数`=3
|
||||
+ `最大轮次`=4
|
||||
+ `流式`=False
|
||||
+ `打印增量`=False
|
||||
+ `轨迹`=False
|
||||
+ `强制`=False
|
||||
+ `路由`=auto
|
||||
+ `路由模型`=None
|
||||
|
||||
v2+P4 主入口:自适应路由 → 执行 → 记录情景 → 尝试学习(技能 + 策略)。`路由`=`auto`|`marqdo`|`small-llm`|`llm`(`jev` 为 `small-llm` 兼容别名);决策器用 `路由模型`(任意 llm 句柄)。
|
||||
|
||||
**model = > json.get value=`self` key="model"**
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
**standing = > json.get value=`self` key="standing"**
|
||||
**en = > agent.agent model=`model` tools=`tools` standing=`standing`**
|
||||
*> en.run task=`任务` kb_dir=`知识库目录` memory_dir=`记忆目录` learn=`学习` verify=`校验` writeback=`写回` reuse=`复用` near_match=`近义命中` near_threshold=`近义阈值` promote=`晋升` improve_every=`改进周期` explore_n=`探索次数` max_rounds=`最大轮次` stream=`流式` echo=`打印增量` trace=`轨迹` force=`强制` router=`路由` router_model=`路由模型`*
|
||||
|
||||
## 路由
|
||||
+ `任务`
|
||||
+ `后端`=auto
|
||||
+ `知识库目录`=.marqdo/agent-kb
|
||||
+ `记忆目录`=.marqdo/agent-memory
|
||||
+ `阈值`=0.78
|
||||
|
||||
自适应路由(阶段 4):`marqdo` / `small-llm` / `llm` / `auto`(`jev`→别名)。
|
||||
|
||||
**model = > json.get value=`self` key="model"**
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
**standing = > json.get value=`self` key="standing"**
|
||||
**en = > agent.agent model=`model` tools=`tools` standing=`standing`**
|
||||
*> en.route task=`任务` backend=`后端` kb_dir=`知识库目录` memory_dir=`记忆目录` threshold=`阈值`*
|
||||
|
||||
## 编译策略
|
||||
+ `记忆目录`=.marqdo/agent-memory
|
||||
+ `最少证据`=3
|
||||
+ `强制`=False
|
||||
|
||||
策略编译:稳定的 task→skill 映射 → `policies/router.mq.md`。
|
||||
|
||||
**model = > json.get value=`self` key="model"**
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
**standing = > json.get value=`self` key="standing"**
|
||||
**en = > agent.agent model=`model` tools=`tools` standing=`standing`**
|
||||
*> en.compile_policy memory_dir=`记忆目录` min_evidence=`最少证据` force=`强制`*
|
||||
+4
-3
@@ -800,10 +800,11 @@ Serve `/`, routed pages, `/_part/{id}` (home) and `{path}/_part/{id}` (routes),
|
||||
+ `room_key`=None
|
||||
+ `on_message`=None
|
||||
+ `presence`=False
|
||||
+ `require_auth`=None
|
||||
|
||||
Register a WebSocket endpoint at `path` (e.g. `/live` or `/chat/{id}`). `mode` is `echo` (default), `broadcast` (fan-out text to all sockets on this path), `room` (named room via `room_key`, supports `{id}` from the path), or `drain`. Legacy `echo=False` maps to `drain`. `on_message` is an optional `lib.member` hook (G-WS2): called before fan-out with `message`/`room`/`user`; return `{skip:true}` to drop or `{message:…}` to rewrite. `presence=True` also joins `room.presence` and emits join/leave JSON. Connect from a client with `web.ws.connect`.
|
||||
Register a WebSocket endpoint at `path` (e.g. `/live` or `/chat/{id}`). `mode` is `echo` (default), `broadcast` (fan-out text to all sockets on this path), `room` (named room via `room_key`, supports `{id}` from the path), or `drain`. Legacy `echo=False` maps to `drain`. `on_message` is an optional `lib.member` hook (G-WS2): called before fan-out with `message`/`room`/`user`; return `{skip:true}` to drop or `{message:…}` to rewrite. `presence=True` also joins `room.presence` and emits join/leave JSON. Stateful `broadcast` and `room` endpoints require a logged-in session by default; set `require_auth=False` only for intentionally public live feeds. Browser handshakes are same-origin. Connect from a client with `web.ws.connect`.
|
||||
|
||||
*> web_app_route_ws app=`self` path=`path` echo=`echo` mode=`mode` room_key=`room_key` on_message=`on_message` presence=`presence`*
|
||||
*> web_app_route_ws app=`self` path=`path` echo=`echo` mode=`mode` room_key=`room_key` on_message=`on_message` presence=`presence` require_auth=`require_auth`*
|
||||
|
||||
## auth
|
||||
+ `users`
|
||||
@@ -1160,7 +1161,7 @@ WebSocket client helper.
|
||||
+ `message`=""
|
||||
+ `headers`=None
|
||||
|
||||
Single request–response: connect to `url`, send `message`, collect all server text replies, close. Returns `{ok, messages}`.
|
||||
Single request–response: connect to `url`, send `message`, collect up to 256 server text replies (each up to 1 MiB), close. Transport errors and timeouts return `{ok:false, error}` instead of looking like an empty successful reply.
|
||||
|
||||
**timeout = [timeout_sec](self)**
|
||||
*> web_ws_connect url=`url` message=`message` headers=`headers` timeout_sec=`timeout`*
|
||||
+4
-3
@@ -768,10 +768,11 @@ SSR 顶栏刊名 Logo、主题切换钮与目录抽屉钮。交互写在 `client
|
||||
+ `房间`=None
|
||||
+ `消息钩子`=None
|
||||
+ `在场`=假
|
||||
+ `需要登录`=None
|
||||
|
||||
在 `路径`(如 `/live` 或 `/chat/{id}`)注册 WebSocket。`模式` 为 `echo`(默认)、`broadcast`、`room`(需 `房间` 键,可含 `{id}`)或 `drain`。旧参数 `回显=假` 等价于 `drain`。`消息钩子` 为可选 `库.成员`(G-WS2):广播前调用,入参含 `message`/`room`/`user`;返回 `{skip:真}` 丢弃或 `{message:…}` 改写。`在场=真` 时另进 `房间.presence` 并推送 join/leave。客户端用 `网页.实时` 的 `连接` 接入。
|
||||
在 `路径`(如 `/live` 或 `/chat/{id}`)注册 WebSocket。`模式` 为 `echo`(默认)、`broadcast`、`room`(需 `房间` 键,可含 `{id}`)或 `drain`。旧参数 `回显=假` 等价于 `drain`。`消息钩子` 为可选 `库.成员`(G-WS2):广播前调用,入参含 `message`/`room`/`user`;返回 `{skip:真}` 丢弃或 `{message:…}` 改写。`在场=真` 时另进 `房间.presence` 并推送 join/leave。有状态的 `broadcast` 与 `room` 端点默认需登录会话;仅对有意公开的直播流设 `需要登录=假`。浏览器握手同源。客户端用 `网页.实时` 的 `连接` 接入。
|
||||
|
||||
*> web_app_route_ws app=`self` path=`路径` echo=`回显` mode=`模式` room_key=`房间` on_message=`消息钩子` presence=`在场`*
|
||||
*> web_app_route_ws app=`self` path=`路径` echo=`回显` mode=`模式` room_key=`房间` on_message=`消息钩子` presence=`在场` require_auth=`需要登录`*
|
||||
|
||||
## 鉴权
|
||||
+ `用户表`
|
||||
@@ -1128,7 +1129,7 @@ WebSocket 客户端工具。
|
||||
+ `消息`=""
|
||||
+ `headers`=None
|
||||
|
||||
单次请求-响应:连接 `地址`,发送 `消息`,收集服务器全部文本回复后关闭。返回 `{ok, messages}`。
|
||||
单次请求-响应:连接 `地址`,发送 `消息`,最多收集 256 条服务器文本回复(每条 ≤1 MiB)后关闭。传输错误与超时返回 `{ok:false, error}`,不再像空的成功回复。
|
||||
|
||||
**timeout = [timeout_sec](self)**
|
||||
*> web_ws_connect url=`地址` message=`消息` headers=`headers` timeout_sec=`timeout`*
|
||||
+2
-2
@@ -9,7 +9,7 @@ English HTTP(S) wrappers. Optional headers / content_type via host aliases.
|
||||
+ `url`
|
||||
+ `headers`=None
|
||||
|
||||
HTTPS supported. Optional named arg `headers=` (map).
|
||||
HTTPS supported. Optional named arg `headers=` (map). Responses are capped at 8 MiB before decoding, so an upstream cannot exhaust the interpreter's memory.
|
||||
|
||||
*> host_http_get url=`url` headers=`headers`*
|
||||
|
||||
@@ -56,7 +56,7 @@ Optional `body=` / `content_type=` / `headers=`.
|
||||
## url_encode
|
||||
+ `text`
|
||||
|
||||
Percent-encode a URL segment.
|
||||
Percent-encode a URL segment. Spaces become `%20` (not form/query `+`), so literal `+` remains unambiguous in paths.
|
||||
|
||||
*> host_url_encode text=`text`*
|
||||
|
||||
|
||||
@@ -39,6 +39,25 @@ Current working directory.
|
||||
|
||||
*> host_cwd*
|
||||
|
||||
## meta
|
||||
|
||||
Bound Artifact Metadata from the entry `.mq.md` (map). See `doc/design/binding.md`.
|
||||
|
||||
*> host_meta*
|
||||
|
||||
## meta_get
|
||||
+ `key`
|
||||
|
||||
Lookup one metadata key; missing → None.
|
||||
|
||||
*> host_meta_get key=`key`*
|
||||
|
||||
## module_source
|
||||
|
||||
Entry `.mq.md` source text (for document-as-prompt). Empty string if unset.
|
||||
|
||||
*> host_module_source*
|
||||
|
||||
## exit
|
||||
+ `code`
|
||||
|
||||
|
||||
@@ -39,6 +39,25 @@ description: 中文系统包装(加载环境 可选 路径=)
|
||||
|
||||
*> host_cwd*
|
||||
|
||||
## 元信息
|
||||
|
||||
入口 `.mq.md` 已绑定的 Artifact Metadata(映射)。见 `doc/design/binding.md`。
|
||||
|
||||
*> host_meta*
|
||||
|
||||
## 取元信息
|
||||
+ `键`
|
||||
|
||||
按键取一条元信息;缺失 → None。
|
||||
|
||||
*> host_meta_get key=`键`*
|
||||
|
||||
## 模块源码
|
||||
|
||||
入口 `.mq.md` 全文(供 document-as-prompt)。未设置时为空文本。
|
||||
|
||||
*> host_module_source*
|
||||
|
||||
## 退出
|
||||
+ `码`
|
||||
|
||||
|
||||
@@ -1269,7 +1269,7 @@ pub fn workbook_solidify(args: &Value) -> Result<Value, String> {
|
||||
};
|
||||
let lit = marqdo_quoted_text(&result);
|
||||
let body = format!(
|
||||
"---\ntitle: agent workbook\n---\n\n# main\n\n*`result` = {lit} *\n**`result`**\n"
|
||||
"---\ntitle: agent workbook\n---\n\n# main\n\n**`result` = {lit}**\n*`result`*\n"
|
||||
);
|
||||
fs::write(&p, body).map_err(|e| format!("solidify write: {e}"))?;
|
||||
Ok(json!({
|
||||
|
||||
+134
-1
@@ -1,8 +1,12 @@
|
||||
//! Agent plugin (C ABI v2): layout helpers + session bag + context via host_query + agent-kb.
|
||||
//! Agent plugin (C ABI v2): layout helpers + session bag + context via host_query + agent-kb + v2 memory + adaptive route.
|
||||
|
||||
mod corpus;
|
||||
mod kb;
|
||||
mod mcp_client;
|
||||
mod mcp_server;
|
||||
mod memory;
|
||||
mod resume;
|
||||
mod route;
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::ffi::{CStr, CString};
|
||||
@@ -902,6 +906,30 @@ kb_ffi!(
|
||||
"agent_workbook_solidify"
|
||||
);
|
||||
kb_ffi!(agent_kb_task_files, kb::kb_task_files, "agent_kb_task_files");
|
||||
kb_ffi!(agent_memory_ensure, memory::memory_ensure, "agent_memory_ensure");
|
||||
kb_ffi!(agent_record_episode, memory::record_episode, "agent_record_episode");
|
||||
kb_ffi!(agent_list_episodes, memory::list_episodes, "agent_list_episodes");
|
||||
kb_ffi!(agent_get_episode, memory::get_episode, "agent_get_episode");
|
||||
kb_ffi!(agent_cluster_episodes, memory::cluster_episodes, "agent_cluster_episodes");
|
||||
kb_ffi!(agent_extract_pattern, memory::extract_pattern, "agent_extract_pattern");
|
||||
kb_ffi!(agent_generalize, memory::generalize, "agent_generalize");
|
||||
kb_ffi!(agent_synthesize_skill, memory::synthesize_skill, "agent_synthesize_skill");
|
||||
kb_ffi!(agent_compile_skill, memory::compile_skill, "agent_compile_skill");
|
||||
kb_ffi!(agent_maybe_learn, memory::maybe_learn, "agent_maybe_learn");
|
||||
kb_ffi!(agent_metrics, memory::metrics, "agent_metrics");
|
||||
kb_ffi!(agent_resolve, memory::resolve_task, "agent_resolve");
|
||||
kb_ffi!(agent_route, route::route, "agent_route");
|
||||
kb_ffi!(agent_route_apply, route::route_apply, "agent_route_apply");
|
||||
kb_ffi!(agent_compile_policy, route::compile_policy, "agent_compile_policy");
|
||||
kb_ffi!(
|
||||
agent_maybe_compile_policy,
|
||||
route::maybe_compile_policy,
|
||||
"agent_maybe_compile_policy"
|
||||
);
|
||||
kb_ffi!(agent_resume_save, resume::resume_save, "agent_resume_save");
|
||||
kb_ffi!(agent_resume_load, resume::resume_load, "agent_resume_load");
|
||||
kb_ffi!(agent_resume_clear, resume::resume_clear, "agent_resume_clear");
|
||||
kb_ffi!(agent_resume_list, resume::resume_list, "agent_resume_list");
|
||||
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn marqdo_plugin_abi_version() -> u32 {
|
||||
@@ -1001,6 +1029,27 @@ pub unsafe extern "C" fn marqdo_plugin_init(host: *const MarqdoHostApi) -> c_int
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
if register(
|
||||
host,
|
||||
"agent_mcp_client",
|
||||
"action,command",
|
||||
mcp_client::agent_mcp_client,
|
||||
) != 0
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_resume_save", "id,goal,status,round", agent_resume_save) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_resume_load", "id,resume_dir", agent_resume_load) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_resume_clear", "id,resume_dir", agent_resume_clear) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_resume_list", "resume_dir", agent_resume_list) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_kb_add_alias", "kb_dir,slug,alias", agent_kb_add_alias) != 0 {
|
||||
return 1;
|
||||
}
|
||||
@@ -1022,6 +1071,90 @@ pub unsafe extern "C" fn marqdo_plugin_init(host: *const MarqdoHostApi) -> c_int
|
||||
if register(host, "agent_kb_task_files", "kb_dir,goal", agent_kb_task_files) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_memory_ensure", "memory_dir", agent_memory_ensure) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(
|
||||
host,
|
||||
"agent_record_episode",
|
||||
"task,status,result,mode",
|
||||
agent_record_episode,
|
||||
) != 0
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_list_episodes", "memory_dir,task", agent_list_episodes) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_get_episode", "id", agent_get_episode) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(
|
||||
host,
|
||||
"agent_cluster_episodes",
|
||||
"memory_dir,task,threshold",
|
||||
agent_cluster_episodes,
|
||||
) != 0
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_extract_pattern", "task", agent_extract_pattern) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_generalize", "task", agent_generalize) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_synthesize_skill", "task", agent_synthesize_skill) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(
|
||||
host,
|
||||
"agent_compile_skill",
|
||||
"task,kb_dir,min_evidence",
|
||||
agent_compile_skill,
|
||||
) != 0
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
if register(
|
||||
host,
|
||||
"agent_maybe_learn",
|
||||
"task,kb_dir,improve_every",
|
||||
agent_maybe_learn,
|
||||
) != 0
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_metrics", "memory_dir", agent_metrics) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_resolve", "task,kb_dir", agent_resolve) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_route", "task,backend,kb_dir,memory_dir", agent_route) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_route_apply", "task,reply,kb_dir", agent_route_apply) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(
|
||||
host,
|
||||
"agent_compile_policy",
|
||||
"memory_dir,min_evidence",
|
||||
agent_compile_policy,
|
||||
) != 0
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
if register(
|
||||
host,
|
||||
"agent_maybe_compile_policy",
|
||||
"memory_dir,min_evidence",
|
||||
agent_maybe_compile_policy,
|
||||
) != 0
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
0
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,327 @@
|
||||
//! MCP Client (stdio JSON-RPC) — spawn a server process, initialize, list/call tools.
|
||||
|
||||
use serde_json::{json, Value};
|
||||
use std::ffi::CStr;
|
||||
use std::io::{BufRead, BufReader, Write};
|
||||
use std::os::raw::{c_char, c_int};
|
||||
use std::process::{Child, Command, Stdio};
|
||||
use std::sync::Mutex;
|
||||
use std::time::Duration;
|
||||
|
||||
use crate::{parse_args, set_err, set_out};
|
||||
|
||||
struct Session {
|
||||
child: Child,
|
||||
stdin: std::process::ChildStdin,
|
||||
reader: BufReader<std::process::ChildStdout>,
|
||||
next_id: i64,
|
||||
server_name: String,
|
||||
}
|
||||
|
||||
static SESSIONS: Mutex<Vec<(String, Session)>> = Mutex::new(Vec::new());
|
||||
|
||||
fn opt_text<'a>(args: &'a Value, key: &str) -> Option<&'a str> {
|
||||
args.get(key).and_then(|v| v.as_str()).filter(|s| !s.is_empty())
|
||||
}
|
||||
|
||||
fn arg_text<'a>(args: &'a Value, key: &str) -> Result<&'a str, String> {
|
||||
opt_text(args, key).ok_or_else(|| format!("missing text `{key}`"))
|
||||
}
|
||||
|
||||
fn write_rpc(stdin: &mut impl Write, msg: &Value) -> Result<(), String> {
|
||||
let s = serde_json::to_string(msg).map_err(|e| e.to_string())?;
|
||||
writeln!(stdin, "{s}").map_err(|e| format!("mcp client write: {e}"))?;
|
||||
stdin.flush().map_err(|e| format!("mcp client flush: {e}"))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn read_rpc(reader: &mut impl BufRead) -> Result<Value, String> {
|
||||
let mut line = String::new();
|
||||
loop {
|
||||
line.clear();
|
||||
let n = reader
|
||||
.read_line(&mut line)
|
||||
.map_err(|e| format!("mcp client read: {e}"))?;
|
||||
if n == 0 {
|
||||
return Err("mcp client: server closed stdout".into());
|
||||
}
|
||||
let t = line.trim();
|
||||
if t.is_empty() {
|
||||
continue;
|
||||
}
|
||||
return serde_json::from_str(t).map_err(|e| format!("mcp client json: {e}"));
|
||||
}
|
||||
}
|
||||
|
||||
fn rpc_call(session: &mut Session, method: &str, params: Value) -> Result<Value, String> {
|
||||
let id = session.next_id;
|
||||
session.next_id += 1;
|
||||
let req = json!({
|
||||
"jsonrpc": "2.0",
|
||||
"id": id,
|
||||
"method": method,
|
||||
"params": params,
|
||||
});
|
||||
write_rpc(&mut session.stdin, &req)?;
|
||||
// Read until matching id (skip notifications).
|
||||
for _ in 0..64 {
|
||||
let resp = read_rpc(&mut session.reader)?;
|
||||
if resp.get("id") == Some(&json!(id)) {
|
||||
if let Some(err) = resp.get("error") {
|
||||
let msg = err
|
||||
.get("message")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("rpc error");
|
||||
return Err(msg.to_string());
|
||||
}
|
||||
return Ok(resp.get("result").cloned().unwrap_or(Value::Null));
|
||||
}
|
||||
}
|
||||
Err("mcp client: no matching response".into())
|
||||
}
|
||||
|
||||
fn spawn_session(command: &str, args: &[String], name: &str) -> Result<Session, String> {
|
||||
let mut cmd = Command::new(command);
|
||||
for a in args {
|
||||
cmd.arg(a);
|
||||
}
|
||||
cmd.stdin(Stdio::piped())
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::inherit());
|
||||
let mut child = cmd
|
||||
.spawn()
|
||||
.map_err(|e| format!("mcp client spawn `{command}`: {e}"))?;
|
||||
let stdin = child
|
||||
.stdin
|
||||
.take()
|
||||
.ok_or_else(|| "mcp client: no stdin".to_string())?;
|
||||
let stdout = child
|
||||
.stdout
|
||||
.take()
|
||||
.ok_or_else(|| "mcp client: no stdout".to_string())?;
|
||||
let mut session = Session {
|
||||
child,
|
||||
stdin,
|
||||
reader: BufReader::new(stdout),
|
||||
next_id: 1,
|
||||
server_name: name.to_string(),
|
||||
};
|
||||
// initialize
|
||||
let _ = rpc_call(
|
||||
&mut session,
|
||||
"initialize",
|
||||
json!({
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": {},
|
||||
"clientInfo": { "name": "marqdo-agent", "version": "0.1.0" }
|
||||
}),
|
||||
)?;
|
||||
// optional initialized notification
|
||||
let _ = write_rpc(
|
||||
&mut session.stdin,
|
||||
&json!({
|
||||
"jsonrpc": "2.0",
|
||||
"method": "notifications/initialized",
|
||||
"params": {}
|
||||
}),
|
||||
);
|
||||
let _ = session.server_name;
|
||||
// Give slow servers a beat (ignore).
|
||||
let _ = Duration::from_millis(1);
|
||||
Ok(session)
|
||||
}
|
||||
|
||||
fn command_args(args: &Value) -> Result<(String, Vec<String>), String> {
|
||||
let command = arg_text(args, "command")?;
|
||||
let mut argv = Vec::new();
|
||||
if let Some(arr) = args.get("args").and_then(|v| v.as_array()) {
|
||||
for a in arr {
|
||||
if let Some(s) = a.as_str() {
|
||||
argv.push(s.to_string());
|
||||
} else {
|
||||
argv.push(a.to_string());
|
||||
}
|
||||
}
|
||||
} else if let Some(s) = opt_text(args, "args") {
|
||||
// space-split simple args
|
||||
for part in s.split_whitespace() {
|
||||
argv.push(part.to_string());
|
||||
}
|
||||
}
|
||||
Ok((command.to_string(), argv))
|
||||
}
|
||||
|
||||
fn session_id_of(args: &Value) -> String {
|
||||
opt_text(args, "session")
|
||||
.or_else(|| opt_text(args, "id"))
|
||||
.unwrap_or("default")
|
||||
.to_string()
|
||||
}
|
||||
|
||||
fn with_session<F>(args: &Value, f: F) -> Result<Value, String>
|
||||
where
|
||||
F: FnOnce(&mut Session) -> Result<Value, String>,
|
||||
{
|
||||
let sid = session_id_of(args);
|
||||
let mut guard = SESSIONS
|
||||
.lock()
|
||||
.map_err(|_| "mcp client: session lock poisoned".to_string())?;
|
||||
let idx = guard
|
||||
.iter()
|
||||
.position(|(id, _)| id == &sid)
|
||||
.ok_or_else(|| format!("mcp client: no session `{sid}` (call connect first)"))?;
|
||||
f(&mut guard[idx].1)
|
||||
}
|
||||
|
||||
fn action_connect(args: &Value) -> Result<Value, String> {
|
||||
let (command, argv) = command_args(args)?;
|
||||
let name = opt_text(args, "name").unwrap_or("mcp");
|
||||
let sid = session_id_of(args);
|
||||
let session = spawn_session(&command, &argv, name)?;
|
||||
let mut guard = SESSIONS
|
||||
.lock()
|
||||
.map_err(|_| "mcp client: session lock poisoned".to_string())?;
|
||||
// Replace existing
|
||||
guard.retain(|(id, _)| id != &sid);
|
||||
guard.push((sid.clone(), session));
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"session": sid,
|
||||
"transport": "stdio",
|
||||
"command": command,
|
||||
}))
|
||||
}
|
||||
|
||||
fn action_list(args: &Value) -> Result<Value, String> {
|
||||
with_session(args, |s| {
|
||||
let result = rpc_call(s, "tools/list", json!({}))?;
|
||||
let mut out = result;
|
||||
if let Some(obj) = out.as_object_mut() {
|
||||
obj.insert("authority".into(), json!("workbook"));
|
||||
obj.insert("session".into(), json!(session_id_of(args)));
|
||||
}
|
||||
Ok(out)
|
||||
})
|
||||
}
|
||||
|
||||
fn action_call(args: &Value) -> Result<Value, String> {
|
||||
let name = arg_text(args, "name")?;
|
||||
let arguments = args.get("arguments").cloned().unwrap_or(json!({}));
|
||||
with_session(args, |s| {
|
||||
let result = rpc_call(
|
||||
s,
|
||||
"tools/call",
|
||||
json!({ "name": name, "arguments": arguments }),
|
||||
)?;
|
||||
let mut out = if result.is_object() {
|
||||
result
|
||||
} else {
|
||||
json!({ "result": result })
|
||||
};
|
||||
if let Some(obj) = out.as_object_mut() {
|
||||
obj.insert("authority".into(), json!("workbook"));
|
||||
obj.insert("session".into(), json!(session_id_of(args)));
|
||||
}
|
||||
Ok(out)
|
||||
})
|
||||
}
|
||||
|
||||
fn action_close(args: &Value) -> Result<Value, String> {
|
||||
let sid = session_id_of(args);
|
||||
let mut guard = SESSIONS
|
||||
.lock()
|
||||
.map_err(|_| "mcp client: session lock poisoned".to_string())?;
|
||||
if let Some(idx) = guard.iter().position(|(id, _)| id == &sid) {
|
||||
let (_, mut session) = guard.remove(idx);
|
||||
let _ = session.child.kill();
|
||||
let _ = session.child.wait();
|
||||
Ok(json!({ "ok": true, "session": sid, "closed": true }))
|
||||
} else {
|
||||
Ok(json!({ "ok": true, "session": sid, "closed": false }))
|
||||
}
|
||||
}
|
||||
|
||||
/// One-shot: connect → list or call → close (no persistent session).
|
||||
fn action_oneshot(args: &Value, kind: &str) -> Result<Value, String> {
|
||||
let (command, argv) = command_args(args)?;
|
||||
let name = opt_text(args, "name").unwrap_or("mcp");
|
||||
let mut session = spawn_session(&command, &argv, name)?;
|
||||
let result = match kind {
|
||||
"list" => {
|
||||
let r = rpc_call(&mut session, "tools/list", json!({}))?;
|
||||
Ok(r)
|
||||
}
|
||||
"call" => {
|
||||
let tname = arg_text(args, "tool").or_else(|_| arg_text(args, "name"))?;
|
||||
// When oneshot call, `name` may be server name — prefer `tool=`.
|
||||
let tname = opt_text(args, "tool").unwrap_or(tname);
|
||||
let arguments = args.get("arguments").cloned().unwrap_or(json!({}));
|
||||
rpc_call(
|
||||
&mut session,
|
||||
"tools/call",
|
||||
json!({ "name": tname, "arguments": arguments }),
|
||||
)
|
||||
}
|
||||
_ => Err(format!("unknown oneshot kind `{kind}`")),
|
||||
};
|
||||
let _ = session.child.kill();
|
||||
let _ = session.child.wait();
|
||||
let mut out = result?;
|
||||
if let Some(obj) = out.as_object_mut() {
|
||||
obj.insert("authority".into(), json!("workbook"));
|
||||
obj.insert("transport".into(), json!("stdio"));
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
pub fn mcp_client(args: &Value) -> Result<Value, String> {
|
||||
let action = opt_text(args, "action").unwrap_or("list");
|
||||
match action {
|
||||
"connect" => action_connect(args),
|
||||
"list" => {
|
||||
if opt_text(args, "command").is_some() && opt_text(args, "session").is_none() {
|
||||
action_oneshot(args, "list")
|
||||
} else {
|
||||
action_list(args)
|
||||
}
|
||||
}
|
||||
"call" => {
|
||||
if opt_text(args, "command").is_some() && opt_text(args, "session").is_none() {
|
||||
action_oneshot(args, "call")
|
||||
} else {
|
||||
action_call(args)
|
||||
}
|
||||
}
|
||||
"close" | "disconnect" => action_close(args),
|
||||
other => Err(format!(
|
||||
"mcp client: unknown action `{other}` (connect|list|call|close)"
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
pub unsafe extern "C" fn agent_mcp_client(
|
||||
args_json: *const c_char,
|
||||
out_json: *mut *mut c_char,
|
||||
err_msg: *mut *mut c_char,
|
||||
) -> c_int {
|
||||
let args = match parse_args(args_json) {
|
||||
Ok(v) => v,
|
||||
Err(e) => {
|
||||
set_err(err_msg, &e);
|
||||
return 1;
|
||||
}
|
||||
};
|
||||
match mcp_client(&args) {
|
||||
Ok(v) => {
|
||||
set_out(out_json, &v.to_string());
|
||||
0
|
||||
}
|
||||
Err(e) => {
|
||||
set_err(err_msg, &e);
|
||||
1
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
fn _unused(_: &CStr) {}
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,181 @@
|
||||
//! Agent plan/run resume checkpoints under `.marqdo/agent-resume/`.
|
||||
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use serde_json::{json, Value};
|
||||
|
||||
const DEFAULT_RESUME_DIR: &str = ".marqdo/agent-resume";
|
||||
|
||||
fn resolve_path(rel: &str) -> PathBuf {
|
||||
let p = PathBuf::from(rel);
|
||||
if p.is_absolute() {
|
||||
return p;
|
||||
}
|
||||
if p.exists() {
|
||||
return p;
|
||||
}
|
||||
let base = (|| {
|
||||
let q = crate::host_query_json("cwd")?;
|
||||
q.as_str()
|
||||
.map(PathBuf::from)
|
||||
.ok_or_else(|| "cwd not text".to_string())
|
||||
})()
|
||||
.or_else(|_| std::env::current_dir().map_err(|e| e.to_string()))
|
||||
.unwrap_or_else(|_| PathBuf::from("."));
|
||||
base.join(p)
|
||||
}
|
||||
|
||||
fn opt_text<'a>(args: &'a Value, key: &str) -> Option<&'a str> {
|
||||
args.get(key).and_then(|v| v.as_str()).filter(|s| !s.is_empty())
|
||||
}
|
||||
|
||||
fn arg_text<'a>(args: &'a Value, key: &str) -> Result<&'a str, String> {
|
||||
opt_text(args, key).ok_or_else(|| format!("missing text `{key}`"))
|
||||
}
|
||||
|
||||
fn opt_i64(args: &Value, key: &str, default: i64) -> i64 {
|
||||
match args.get(key) {
|
||||
None | Some(Value::Null) => default,
|
||||
Some(Value::Number(n)) => n.as_i64().unwrap_or(default),
|
||||
Some(Value::String(s)) => s.parse().unwrap_or(default),
|
||||
_ => default,
|
||||
}
|
||||
}
|
||||
|
||||
fn resume_root(args: &Value) -> PathBuf {
|
||||
let rel = opt_text(args, "resume_dir").unwrap_or(DEFAULT_RESUME_DIR);
|
||||
resolve_path(rel)
|
||||
}
|
||||
|
||||
fn checkpoint_path(root: &Path, id: &str) -> PathBuf {
|
||||
let safe: String = id
|
||||
.chars()
|
||||
.map(|c| {
|
||||
if c.is_ascii_alphanumeric() || c == '-' || c == '_' {
|
||||
c
|
||||
} else {
|
||||
'_'
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
root.join(format!("{safe}.json"))
|
||||
}
|
||||
|
||||
fn now_rfc3339() -> String {
|
||||
chrono::Utc::now().to_rfc3339_opts(chrono::SecondsFormat::Secs, true)
|
||||
}
|
||||
|
||||
fn ensure_dir(root: &Path) -> Result<(), String> {
|
||||
fs::create_dir_all(root).map_err(|e| format!("resume mkdir: {e}"))
|
||||
}
|
||||
|
||||
pub fn resume_save(args: &Value) -> Result<Value, String> {
|
||||
let id = arg_text(args, "id")?;
|
||||
let root = resume_root(args);
|
||||
ensure_dir(&root)?;
|
||||
let path = checkpoint_path(&root, id);
|
||||
let mut doc = json!({
|
||||
"id": id,
|
||||
"updated_at": now_rfc3339(),
|
||||
});
|
||||
if let Some(obj) = doc.as_object_mut() {
|
||||
for key in [
|
||||
"goal", "task", "status", "workbook", "events", "route", "result", "round", "meta",
|
||||
] {
|
||||
if let Some(v) = args.get(key) {
|
||||
if !v.is_null() {
|
||||
obj.insert(key.to_string(), v.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
if !obj.contains_key("round") {
|
||||
obj.insert("round".into(), json!(opt_i64(args, "round", 0)));
|
||||
}
|
||||
if !obj.contains_key("status") {
|
||||
obj.insert("status".into(), json!("running"));
|
||||
}
|
||||
}
|
||||
let text = serde_json::to_string_pretty(&doc).map_err(|e| e.to_string())?;
|
||||
fs::write(&path, text).map_err(|e| format!("resume save: {e}"))?;
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"id": id,
|
||||
"path": path.to_string_lossy(),
|
||||
"updated_at": doc.get("updated_at").cloned().unwrap_or(Value::Null),
|
||||
}))
|
||||
}
|
||||
|
||||
pub fn resume_load(args: &Value) -> Result<Value, String> {
|
||||
let id = arg_text(args, "id")?;
|
||||
let root = resume_root(args);
|
||||
let path = checkpoint_path(&root, id);
|
||||
if !path.is_file() {
|
||||
return Ok(json!({
|
||||
"ok": false,
|
||||
"found": false,
|
||||
"id": id,
|
||||
"path": path.to_string_lossy(),
|
||||
}));
|
||||
}
|
||||
let text = fs::read_to_string(&path).map_err(|e| format!("resume load: {e}"))?;
|
||||
let mut doc: Value = serde_json::from_str(&text).map_err(|e| format!("resume json: {e}"))?;
|
||||
if let Some(obj) = doc.as_object_mut() {
|
||||
obj.insert("ok".into(), json!(true));
|
||||
obj.insert("found".into(), json!(true));
|
||||
obj.insert("path".into(), json!(path.to_string_lossy()));
|
||||
}
|
||||
Ok(doc)
|
||||
}
|
||||
|
||||
pub fn resume_clear(args: &Value) -> Result<Value, String> {
|
||||
let id = arg_text(args, "id")?;
|
||||
let root = resume_root(args);
|
||||
let path = checkpoint_path(&root, id);
|
||||
let existed = path.is_file();
|
||||
if existed {
|
||||
fs::remove_file(&path).map_err(|e| format!("resume clear: {e}"))?;
|
||||
}
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"id": id,
|
||||
"cleared": existed,
|
||||
"path": path.to_string_lossy(),
|
||||
}))
|
||||
}
|
||||
|
||||
pub fn resume_list(args: &Value) -> Result<Value, String> {
|
||||
let root = resume_root(args);
|
||||
if !root.is_dir() {
|
||||
return Ok(json!({ "ok": true, "items": [], "resume_dir": root.to_string_lossy() }));
|
||||
}
|
||||
let mut items = Vec::new();
|
||||
let rd = fs::read_dir(&root).map_err(|e| format!("resume list: {e}"))?;
|
||||
for ent in rd.flatten() {
|
||||
let p = ent.path();
|
||||
if p.extension().and_then(|e| e.to_str()) != Some("json") {
|
||||
continue;
|
||||
}
|
||||
let id = p
|
||||
.file_stem()
|
||||
.and_then(|s| s.to_str())
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
let meta = fs::metadata(&p).ok();
|
||||
items.push(json!({
|
||||
"id": id,
|
||||
"path": p.to_string_lossy(),
|
||||
"len": meta.map(|m| m.len()).unwrap_or(0),
|
||||
}));
|
||||
}
|
||||
items.sort_by(|a, b| {
|
||||
let ai = a.get("id").and_then(|v| v.as_str()).unwrap_or("");
|
||||
let bi = b.get("id").and_then(|v| v.as_str()).unwrap_or("");
|
||||
ai.cmp(bi)
|
||||
});
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"items": items,
|
||||
"resume_dir": root.to_string_lossy(),
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,719 @@
|
||||
//! Adaptive Routing / Policy Compilation (Phase 4).
|
||||
//! Router is abstract: marqdo | small-llm | llm | auto.
|
||||
//! `jev` is accepted as an alias of `small-llm` (compat); not a first-class backend.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use serde_json::{json, Value};
|
||||
|
||||
use crate::kb::{goal_sig_hex, goal_slug_str, normalize_goal};
|
||||
use crate::memory::{self};
|
||||
|
||||
const DEFAULT_MEMORY_DIR: &str = ".marqdo/agent-memory";
|
||||
const DEFAULT_KB_DIR: &str = ".marqdo/agent-kb";
|
||||
const POLICY_NAME: &str = "router.mq.md";
|
||||
|
||||
fn arg_text<'a>(args: &'a Value, key: &str) -> Result<&'a str, String> {
|
||||
args.get(key)
|
||||
.and_then(|v| v.as_str())
|
||||
.ok_or_else(|| format!("missing text `{key}`"))
|
||||
}
|
||||
|
||||
fn opt_text<'a>(args: &'a Value, key: &str) -> Option<&'a str> {
|
||||
args.get(key).and_then(|v| v.as_str()).filter(|s| !s.is_empty())
|
||||
}
|
||||
|
||||
fn opt_i64(args: &Value, key: &str, default: i64) -> i64 {
|
||||
match args.get(key) {
|
||||
None | Some(Value::Null) => default,
|
||||
Some(Value::Number(n)) => n.as_i64().unwrap_or(default),
|
||||
Some(Value::String(s)) => s.parse().unwrap_or(default),
|
||||
_ => default,
|
||||
}
|
||||
}
|
||||
|
||||
fn opt_f64(args: &Value, key: &str, default: f64) -> f64 {
|
||||
match args.get(key) {
|
||||
None | Some(Value::Null) => default,
|
||||
Some(Value::Number(n)) => n.as_f64().unwrap_or(default),
|
||||
Some(Value::String(s)) => s.parse().unwrap_or(default),
|
||||
_ => default,
|
||||
}
|
||||
}
|
||||
|
||||
fn opt_bool(args: &Value, key: &str, default: bool) -> bool {
|
||||
match args.get(key) {
|
||||
None | Some(Value::Null) => default,
|
||||
Some(Value::Bool(b)) => *b,
|
||||
Some(Value::String(s)) => matches!(s.as_str(), "true" | "True" | "1" | "yes"),
|
||||
Some(Value::Number(n)) => n.as_i64().unwrap_or(0) != 0,
|
||||
_ => default,
|
||||
}
|
||||
}
|
||||
|
||||
fn now_rfc3339() -> String {
|
||||
chrono::Utc::now().to_rfc3339_opts(chrono::SecondsFormat::Secs, true)
|
||||
}
|
||||
|
||||
fn yaml_escape(s: &str) -> String {
|
||||
if s.is_empty()
|
||||
|| s.contains(':')
|
||||
|| s.contains('#')
|
||||
|| s.contains('|')
|
||||
|| s.contains('"')
|
||||
|| s.contains('\n')
|
||||
{
|
||||
format!("\"{}\"", s.replace('\\', "\\\\").replace('"', "\\\""))
|
||||
} else {
|
||||
s.to_string()
|
||||
}
|
||||
}
|
||||
|
||||
fn resolve_path(rel: &str) -> PathBuf {
|
||||
let p = PathBuf::from(rel);
|
||||
if p.is_absolute() {
|
||||
return p;
|
||||
}
|
||||
if p.exists() {
|
||||
return p;
|
||||
}
|
||||
let base = (|| {
|
||||
let q = crate::host_query_json("cwd")?;
|
||||
q.as_str()
|
||||
.map(PathBuf::from)
|
||||
.ok_or_else(|| "cwd not text".to_string())
|
||||
})()
|
||||
.or_else(|_| std::env::current_dir().map_err(|e| e.to_string()))
|
||||
.unwrap_or_else(|_| PathBuf::from("."));
|
||||
base.join(p)
|
||||
}
|
||||
|
||||
fn memory_root(args: &Value) -> PathBuf {
|
||||
let rel = opt_text(args, "memory_dir").unwrap_or(DEFAULT_MEMORY_DIR);
|
||||
resolve_path(rel)
|
||||
}
|
||||
|
||||
fn policy_path(root: &Path) -> PathBuf {
|
||||
root.join("policies").join(POLICY_NAME)
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
struct PolicyRule {
|
||||
task_hash: String,
|
||||
slug: String,
|
||||
skill: String,
|
||||
task: String,
|
||||
count: i64,
|
||||
confidence: f64,
|
||||
}
|
||||
|
||||
fn parse_policy(src: &str) -> Vec<PolicyRule> {
|
||||
let mut rules = Vec::new();
|
||||
let mut in_table = false;
|
||||
let mut header_ok = false;
|
||||
for line in src.lines() {
|
||||
let t = line.trim();
|
||||
if t.starts_with("|") && t.contains("task_hash") {
|
||||
in_table = true;
|
||||
header_ok = true;
|
||||
continue;
|
||||
}
|
||||
if in_table && t.starts_with("|") && t.contains("---") {
|
||||
continue;
|
||||
}
|
||||
if in_table && t.starts_with("|") {
|
||||
let cols: Vec<&str> = t
|
||||
.trim_matches('|')
|
||||
.split('|')
|
||||
.map(|c| c.trim().trim_matches('"'))
|
||||
.collect();
|
||||
if cols.len() >= 5 {
|
||||
let count = cols[4].parse().unwrap_or(1);
|
||||
let confidence = if cols.len() >= 6 {
|
||||
cols[5].parse().unwrap_or(0.9)
|
||||
} else {
|
||||
0.9
|
||||
};
|
||||
let task = if cols.len() >= 7 {
|
||||
cols[6].to_string()
|
||||
} else {
|
||||
String::new()
|
||||
};
|
||||
rules.push(PolicyRule {
|
||||
task_hash: cols[0].to_string(),
|
||||
slug: cols[1].to_string(),
|
||||
skill: cols[2].to_string(),
|
||||
count,
|
||||
confidence,
|
||||
task,
|
||||
});
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if in_table && header_ok && !t.starts_with('|') && !t.is_empty() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
rules
|
||||
}
|
||||
|
||||
fn write_policy(root: &Path, rules: &[PolicyRule]) -> Result<PathBuf, String> {
|
||||
fs::create_dir_all(root.join("policies")).map_err(|e| format!("mkdir policies: {e}"))?;
|
||||
let path = policy_path(root);
|
||||
let ts = now_rfc3339();
|
||||
let mut body = format!(
|
||||
"---\ntype: policy\nstatus: compiled\nbackend: marqdo\ngenerated_at: {ts}\nrules: {}\n---\n\n# Adaptive router (compiled Marqdo policy)\n\nDeterministic task → skill routing. No LLM required.\n\n# Rules\n\n| task_hash | slug | skill | resource | count | confidence | task |\n|-----------|------|-------|----------|------:|-----------:|------|\n",
|
||||
rules.len()
|
||||
);
|
||||
for r in rules {
|
||||
body.push_str(&format!(
|
||||
"| {} | {} | {} | {} | {} | {:.2} | {} |\n",
|
||||
r.task_hash,
|
||||
yaml_escape(&r.slug),
|
||||
yaml_escape(&r.skill),
|
||||
yaml_escape(&r.skill), // resource often == skill path for OKF
|
||||
r.count,
|
||||
r.confidence,
|
||||
yaml_escape(&r.task),
|
||||
));
|
||||
}
|
||||
body.push_str("\n# Fallback\n\nllm\n");
|
||||
fs::write(&path, body).map_err(|e| format!("write policy: {e}"))?;
|
||||
Ok(path)
|
||||
}
|
||||
|
||||
fn load_rules(root: &Path) -> Vec<PolicyRule> {
|
||||
let path = policy_path(root);
|
||||
let Ok(src) = fs::read_to_string(&path) else {
|
||||
return Vec::new();
|
||||
};
|
||||
parse_policy(&src)
|
||||
}
|
||||
|
||||
fn match_policy(rules: &[PolicyRule], task: &str, threshold: f64) -> Option<PolicyRule> {
|
||||
let th = goal_sig_hex(&normalize_goal(task));
|
||||
if let Some(r) = rules.iter().find(|r| r.task_hash == th) {
|
||||
return Some(r.clone());
|
||||
}
|
||||
// Near-match on stored task text / slug.
|
||||
let mut best: Option<(f64, PolicyRule)> = None;
|
||||
for r in rules {
|
||||
let score = memory::task_near_score(task, &r.task);
|
||||
if score >= threshold {
|
||||
if best.as_ref().map(|(s, _)| score > *s).unwrap_or(true) {
|
||||
best = Some((score, r.clone()));
|
||||
}
|
||||
}
|
||||
let slug_score = memory::task_near_score(task, &r.slug);
|
||||
if slug_score >= threshold {
|
||||
if best.as_ref().map(|(s, _)| slug_score > *s).unwrap_or(true) {
|
||||
let mut rr = r.clone();
|
||||
rr.confidence = slug_score;
|
||||
best = Some((slug_score, rr));
|
||||
}
|
||||
}
|
||||
}
|
||||
best.map(|(s, mut r)| {
|
||||
r.confidence = s;
|
||||
r
|
||||
})
|
||||
}
|
||||
|
||||
fn build_route_prompt(task: &str, candidates: &[Value]) -> String {
|
||||
let mut s = String::from(
|
||||
"You are a Marqdo Agent router. Pick the best skill slug for the task, or say EXPLORE.\n\nReply with exactly one line:\nSKILL:<slug>\nor\nEXPLORE\n\nTask:\n",
|
||||
);
|
||||
s.push_str(task);
|
||||
s.push_str("\n\nCandidates:\n");
|
||||
for (i, c) in candidates.iter().enumerate() {
|
||||
let slug = c
|
||||
.get("slug")
|
||||
.or_else(|| c.get("title"))
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("?");
|
||||
let title = c
|
||||
.get("title")
|
||||
.or_else(|| c.get("goal"))
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("");
|
||||
s.push_str(&format!("{}. {slug} — {title}\n", i + 1));
|
||||
}
|
||||
s
|
||||
}
|
||||
|
||||
fn candidate_list(args: &Value) -> Result<Vec<Value>, String> {
|
||||
if let Some(Value::Array(items)) = args.get("candidates") {
|
||||
return Ok(items.clone());
|
||||
}
|
||||
let kb = opt_text(args, "kb_dir").unwrap_or(DEFAULT_KB_DIR);
|
||||
let listed = crate::kb::kb_list_tasks(&json!({ "kb_dir": kb }))?;
|
||||
let tasks = listed
|
||||
.get("tasks")
|
||||
.and_then(|v| v.as_array())
|
||||
.cloned()
|
||||
.unwrap_or_default();
|
||||
if !tasks.is_empty() {
|
||||
return Ok(tasks);
|
||||
}
|
||||
let goal = arg_text(args, "task").or_else(|_| arg_text(args, "goal"))?;
|
||||
let near = crate::kb::kb_near_match(&json!({
|
||||
"kb_dir": kb,
|
||||
"goal": goal,
|
||||
}))?;
|
||||
Ok(near
|
||||
.get("candidates")
|
||||
.and_then(|v| v.as_array())
|
||||
.cloned()
|
||||
.unwrap_or_default())
|
||||
}
|
||||
|
||||
fn enrich_with_kb(rule: &PolicyRule, kb: &str, task: &str) -> Result<Value, String> {
|
||||
let hit = crate::kb::kb_lookup(&json!({
|
||||
"kb_dir": kb,
|
||||
"goal": task,
|
||||
"slug": rule.slug,
|
||||
"near_match": true,
|
||||
"near_threshold": 0.5,
|
||||
}))
|
||||
.unwrap_or(Value::Null);
|
||||
let resource = hit
|
||||
.get("resource")
|
||||
.cloned()
|
||||
.unwrap_or_else(|| Value::String(rule.skill.clone()));
|
||||
let skill = hit
|
||||
.get("skill")
|
||||
.cloned()
|
||||
.unwrap_or_else(|| Value::String(rule.skill.clone()));
|
||||
let llm_free = hit
|
||||
.get("llm_free")
|
||||
.and_then(|v| v.as_bool())
|
||||
.unwrap_or(true);
|
||||
Ok(json!({
|
||||
"backend": "marqdo",
|
||||
"mode": "policy",
|
||||
"matched": true,
|
||||
"needs_llm": false,
|
||||
"slug": rule.slug,
|
||||
"skill": skill,
|
||||
"resource": resource,
|
||||
"task_hash": rule.task_hash,
|
||||
"confidence": rule.confidence,
|
||||
"llm_free": llm_free,
|
||||
"count": rule.count,
|
||||
}))
|
||||
}
|
||||
|
||||
fn route_marqdo(args: &Value) -> Result<Value, String> {
|
||||
let root = memory_root(args);
|
||||
let _ = memory::memory_ensure(&json!({
|
||||
"memory_dir": opt_text(args, "memory_dir").unwrap_or(DEFAULT_MEMORY_DIR),
|
||||
}));
|
||||
let task = arg_text(args, "task").or_else(|_| arg_text(args, "goal"))?;
|
||||
let threshold = opt_f64(args, "threshold", 0.78);
|
||||
let kb = opt_text(args, "kb_dir").unwrap_or(DEFAULT_KB_DIR);
|
||||
let rules = load_rules(&root);
|
||||
if let Some(rule) = match_policy(&rules, task, threshold) {
|
||||
if rule.confidence >= threshold {
|
||||
return enrich_with_kb(&rule, kb, task);
|
||||
}
|
||||
}
|
||||
Ok(json!({
|
||||
"backend": "marqdo",
|
||||
"mode": "miss",
|
||||
"matched": false,
|
||||
"needs_llm": false,
|
||||
"confidence": 0.0,
|
||||
}))
|
||||
}
|
||||
|
||||
fn route_resolve_fallback(args: &Value) -> Result<Value, String> {
|
||||
let resolved = memory::resolve_task(args)?;
|
||||
let conf = resolved
|
||||
.get("confidence")
|
||||
.and_then(|v| v.as_f64())
|
||||
.unwrap_or(0.0);
|
||||
let llm_free = resolved
|
||||
.get("llm_free")
|
||||
.and_then(|v| v.as_bool())
|
||||
.unwrap_or(false);
|
||||
if conf >= opt_f64(args, "threshold", 0.78) || llm_free {
|
||||
return Ok(json!({
|
||||
"backend": "kb",
|
||||
"mode": resolved.get("mode").cloned().unwrap_or(Value::String("near".into())),
|
||||
"matched": true,
|
||||
"needs_llm": false,
|
||||
"slug": resolved.get("slug").cloned().unwrap_or(Value::Null),
|
||||
"skill": resolved.get("skill").cloned().unwrap_or(Value::Null),
|
||||
"resource": resolved.get("resource").cloned().unwrap_or(Value::Null),
|
||||
"confidence": conf,
|
||||
"llm_free": llm_free,
|
||||
}));
|
||||
}
|
||||
Ok(json!({
|
||||
"backend": "kb",
|
||||
"mode": "miss",
|
||||
"matched": false,
|
||||
"needs_llm": false,
|
||||
"confidence": conf,
|
||||
"resolve": resolved,
|
||||
}))
|
||||
}
|
||||
|
||||
fn route_model_request(args: &Value, level: &str) -> Result<Value, String> {
|
||||
let task = arg_text(args, "task").or_else(|_| arg_text(args, "goal"))?;
|
||||
let cands = candidate_list(args)?;
|
||||
let prompt = build_route_prompt(task, &cands);
|
||||
Ok(json!({
|
||||
"backend": level,
|
||||
"mode": "needs_llm",
|
||||
"matched": false,
|
||||
"needs_llm": true,
|
||||
"level": level,
|
||||
"prompt": prompt,
|
||||
"candidates": cands,
|
||||
"confidence": 0.0,
|
||||
}))
|
||||
}
|
||||
|
||||
/// Cascade Adaptive Router (Phase 4).
|
||||
///
|
||||
/// `backend`: `auto` | `marqdo` | `small-llm` | `llm` (`jev` → alias of `small-llm`).
|
||||
///
|
||||
/// `auto` order: marqdo policy → kb resolve → small-llm request.
|
||||
pub fn route(args: &Value) -> Result<Value, String> {
|
||||
let backend = opt_text(args, "backend").unwrap_or("auto");
|
||||
let threshold = opt_f64(args, "threshold", 0.78);
|
||||
|
||||
match backend {
|
||||
"marqdo" => {
|
||||
let hit = route_marqdo(args)?;
|
||||
if hit.get("matched").and_then(|v| v.as_bool()) == Some(true) {
|
||||
return Ok(hit);
|
||||
}
|
||||
route_resolve_fallback(args)
|
||||
}
|
||||
"small-llm" | "small_llm" | "small" | "jev" => route_model_request(args, "small-llm"),
|
||||
"llm" | "large" | "large-llm" => route_model_request(args, "llm"),
|
||||
"auto" | _ => {
|
||||
let mut trace: Vec<Value> = Vec::new();
|
||||
let m = route_marqdo(args)?;
|
||||
trace.push(json!({"try": "marqdo", "matched": m.get("matched")}));
|
||||
if m.get("matched").and_then(|v| v.as_bool()) == Some(true)
|
||||
&& m.get("confidence").and_then(|v| v.as_f64()).unwrap_or(0.0) >= threshold
|
||||
{
|
||||
let mut out = m;
|
||||
if let Some(obj) = out.as_object_mut() {
|
||||
obj.insert("trace".into(), Value::Array(trace));
|
||||
obj.insert("backend".into(), Value::String("auto/marqdo".into()));
|
||||
}
|
||||
return Ok(out);
|
||||
}
|
||||
let kb = route_resolve_fallback(args)?;
|
||||
trace.push(json!({"try": "kb", "matched": kb.get("matched")}));
|
||||
if kb.get("matched").and_then(|v| v.as_bool()) == Some(true) {
|
||||
let mut out = kb;
|
||||
if let Some(obj) = out.as_object_mut() {
|
||||
obj.insert("trace".into(), Value::Array(trace));
|
||||
obj.insert("backend".into(), Value::String("auto/kb".into()));
|
||||
}
|
||||
return Ok(out);
|
||||
}
|
||||
// Prefer small-llm request in auto when nothing matched (caller may use large).
|
||||
let mut req = route_model_request(args, "small-llm")?;
|
||||
if let Some(obj) = req.as_object_mut() {
|
||||
obj.insert("trace".into(), Value::Array(trace));
|
||||
obj.insert("backend".into(), Value::String("auto/small-llm".into()));
|
||||
}
|
||||
Ok(req)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Apply a router model reply (`SKILL:<slug>` / `EXPLORE`) to a concrete skill hit.
|
||||
pub fn route_apply(args: &Value) -> Result<Value, String> {
|
||||
let reply = arg_text(args, "reply")?;
|
||||
let kb = opt_text(args, "kb_dir").unwrap_or(DEFAULT_KB_DIR);
|
||||
let task = arg_text(args, "task").or_else(|_| arg_text(args, "goal"))?;
|
||||
let line = reply
|
||||
.lines()
|
||||
.map(|l| l.trim())
|
||||
.find(|l| !l.is_empty())
|
||||
.unwrap_or("");
|
||||
let upper = line.to_ascii_uppercase();
|
||||
if upper.starts_with("EXPLORE") || upper == "LLM" || upper == "NEW" {
|
||||
return Ok(json!({
|
||||
"matched": false,
|
||||
"mode": "explore",
|
||||
"decision": "EXPLORE",
|
||||
"needs_llm": false,
|
||||
}));
|
||||
}
|
||||
let slug = if let Some(rest) = line.strip_prefix("SKILL:") {
|
||||
rest.trim()
|
||||
} else if let Some(rest) = line.strip_prefix("skill:") {
|
||||
rest.trim()
|
||||
} else if let Some(rest) = line.strip_prefix("调用:") {
|
||||
rest.trim()
|
||||
} else {
|
||||
line.trim()
|
||||
};
|
||||
if slug.is_empty() {
|
||||
return Ok(json!({
|
||||
"matched": false,
|
||||
"mode": "explore",
|
||||
"decision": "EXPLORE",
|
||||
"reason": "empty_slug",
|
||||
}));
|
||||
}
|
||||
let hit = crate::kb::kb_lookup(&json!({
|
||||
"kb_dir": kb,
|
||||
"goal": task,
|
||||
"slug": slug,
|
||||
"near_match": true,
|
||||
"near_threshold": 0.5,
|
||||
}))?;
|
||||
if hit.is_null() {
|
||||
return Ok(json!({
|
||||
"matched": false,
|
||||
"mode": "miss",
|
||||
"decision": "SKILL",
|
||||
"slug": slug,
|
||||
"reason": "slug_not_in_kb",
|
||||
}));
|
||||
}
|
||||
Ok(json!({
|
||||
"matched": true,
|
||||
"mode": "routed",
|
||||
"decision": "SKILL",
|
||||
"backend": opt_text(args, "backend").unwrap_or("llm"),
|
||||
"slug": slug,
|
||||
"skill": hit.get("skill").cloned().unwrap_or(Value::Null),
|
||||
"resource": hit.get("resource").cloned().unwrap_or(Value::Null),
|
||||
"llm_free": hit.get("llm_free").cloned().unwrap_or(Value::Bool(false)),
|
||||
"confidence": 0.85,
|
||||
"needs_llm": false,
|
||||
}))
|
||||
}
|
||||
|
||||
/// Compile policy from successful episode skill selections (Policy Compilation).
|
||||
pub fn compile_policy(args: &Value) -> Result<Value, String> {
|
||||
let root = memory_root(args);
|
||||
let mem_arg = opt_text(args, "memory_dir").unwrap_or(DEFAULT_MEMORY_DIR);
|
||||
let _ = memory::memory_ensure(&json!({ "memory_dir": mem_arg }));
|
||||
let min_evidence = opt_i64(args, "min_evidence", opt_i64(args, "improve_every", 3));
|
||||
let listed = memory::list_episodes(&json!({
|
||||
"memory_dir": mem_arg,
|
||||
"status": "success",
|
||||
"limit": 500,
|
||||
}))?;
|
||||
let eps = listed
|
||||
.get("episodes")
|
||||
.and_then(|v| v.as_array())
|
||||
.cloned()
|
||||
.unwrap_or_default();
|
||||
|
||||
// task_hash → (slug → count, sample task, skill path)
|
||||
let mut by_hash: HashMap<String, HashMap<String, (i64, String, String)>> = HashMap::new();
|
||||
for e in &eps {
|
||||
let id = e.get("id").and_then(|v| v.as_str()).unwrap_or("");
|
||||
if id.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let full = memory::get_episode(&json!({
|
||||
"memory_dir": mem_arg,
|
||||
"id": id,
|
||||
}))?;
|
||||
if full.is_null() {
|
||||
continue;
|
||||
}
|
||||
let src = full.get("source").and_then(|v| v.as_str()).unwrap_or("");
|
||||
let skill = extract_fm(src, "skill").unwrap_or_default();
|
||||
if skill.is_empty() || skill == "none" || skill == "null" {
|
||||
continue;
|
||||
}
|
||||
let task_hash = extract_fm(src, "task_hash")
|
||||
.or_else(|| e.get("task_hash").and_then(|v| v.as_str()).map(|s| s.to_string()))
|
||||
.unwrap_or_default();
|
||||
if task_hash.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let task = full
|
||||
.get("task")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
let slug = extract_fm(src, "slug")
|
||||
.filter(|s| !s.is_empty())
|
||||
.unwrap_or_else(|| goal_slug_str(&task));
|
||||
// Prefer skill path as key when it looks like a path; else slug.
|
||||
let key = if skill.contains('/') || skill.ends_with(".md") {
|
||||
skill.clone()
|
||||
} else {
|
||||
slug.clone()
|
||||
};
|
||||
let entry = by_hash.entry(task_hash).or_default();
|
||||
let cell = entry.entry(key).or_insert((0, task, skill));
|
||||
cell.0 += 1;
|
||||
}
|
||||
|
||||
let mut rules: Vec<PolicyRule> = Vec::new();
|
||||
for (task_hash, skills) in by_hash {
|
||||
let Some((skill_key, (count, task, skill_path))) =
|
||||
skills.into_iter().max_by_key(|(_, (c, _, _))| *c)
|
||||
else {
|
||||
continue;
|
||||
};
|
||||
if count < min_evidence {
|
||||
continue;
|
||||
}
|
||||
let slug = if skill_key.contains('/') {
|
||||
Path::new(&skill_key)
|
||||
.file_stem()
|
||||
.and_then(|s| s.to_str())
|
||||
.unwrap_or(&skill_key)
|
||||
.trim_end_matches(".mq")
|
||||
.to_string()
|
||||
} else {
|
||||
skill_key
|
||||
};
|
||||
let confidence = (count as f64 / (count as f64 + 1.0)).min(0.99);
|
||||
rules.push(PolicyRule {
|
||||
task_hash,
|
||||
slug,
|
||||
skill: skill_path,
|
||||
task,
|
||||
count,
|
||||
confidence,
|
||||
});
|
||||
}
|
||||
rules.sort_by(|a, b| b.count.cmp(&a.count));
|
||||
|
||||
if rules.is_empty() && !opt_bool(args, "force", false) {
|
||||
return Ok(json!({
|
||||
"ok": false,
|
||||
"compiled": false,
|
||||
"reason": "insufficient_stable_mappings",
|
||||
"min_evidence": min_evidence,
|
||||
}));
|
||||
}
|
||||
|
||||
let path = write_policy(&root, &rules)?;
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"compiled": true,
|
||||
"path": path.to_string_lossy(),
|
||||
"rules": rules.len() as i64,
|
||||
"min_evidence": min_evidence,
|
||||
}))
|
||||
}
|
||||
|
||||
fn extract_fm(source: &str, key: &str) -> Option<String> {
|
||||
let source = source.strip_prefix('\u{feff}').unwrap_or(source);
|
||||
let lines: Vec<&str> = source.lines().collect();
|
||||
if lines.first().map(|l| l.trim()) != Some("---") {
|
||||
return None;
|
||||
}
|
||||
let prefix = format!("{key}:");
|
||||
for line in lines.iter().skip(1) {
|
||||
let t = line.trim();
|
||||
if t == "---" {
|
||||
break;
|
||||
}
|
||||
if let Some(rest) = t.strip_prefix(&prefix) {
|
||||
return Some(rest.trim().trim_matches('"').to_string());
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Learn gate for policy: compile when enough skill-tagged episodes exist.
|
||||
pub fn maybe_compile_policy(args: &Value) -> Result<Value, String> {
|
||||
let compiled = compile_policy(args)?;
|
||||
Ok(json!({
|
||||
"policy_learned": compiled.get("ok").and_then(|v| v.as_bool()).unwrap_or(false),
|
||||
"compile": compiled,
|
||||
}))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn compile_and_route_marqdo() {
|
||||
let dir = std::env::temp_dir().join(format!("marqdo-route-{}", std::process::id()));
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
fs::create_dir_all(&dir).unwrap();
|
||||
let mem = dir.join("agent-memory");
|
||||
let kb = dir.join("agent-kb");
|
||||
let mem_s = mem.to_string_lossy().into_owned();
|
||||
let kb_s = kb.to_string_lossy().into_owned();
|
||||
let task = "Reply with exactly the word pong and nothing else.";
|
||||
|
||||
// Seed a solidified skill via memory compile path.
|
||||
for _ in 0..3 {
|
||||
memory::record_episode(&json!({
|
||||
"memory_dir": mem_s,
|
||||
"task": task,
|
||||
"status": "ok",
|
||||
"mode": "explore",
|
||||
"result": "pong",
|
||||
"skill": "concepts/skills/reply-with-exactly-the-word-pong-and-nothing-els.md",
|
||||
"llm_calls": 1,
|
||||
}))
|
||||
.unwrap();
|
||||
}
|
||||
let learned = memory::maybe_learn(&json!({
|
||||
"memory_dir": mem_s,
|
||||
"kb_dir": kb_s,
|
||||
"task": task,
|
||||
"improve_every": 3,
|
||||
"promote": true,
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(learned.get("learned").and_then(|v| v.as_bool()), Some(true));
|
||||
|
||||
// Tag episodes with skill path for policy mining (re-record with skill).
|
||||
for _ in 0..3 {
|
||||
memory::record_episode(&json!({
|
||||
"memory_dir": mem_s,
|
||||
"task": task,
|
||||
"status": "ok",
|
||||
"mode": "compiled",
|
||||
"result": "pong",
|
||||
"skill": "concepts/skills/reply-with-exactly-the-word-pong-and-nothing-els.md",
|
||||
"llm_calls": 0,
|
||||
}))
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
let pol = compile_policy(&json!({
|
||||
"memory_dir": mem_s,
|
||||
"min_evidence": 3,
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(pol.get("ok").and_then(|v| v.as_bool()), Some(true), "{pol}");
|
||||
|
||||
let routed = route(&json!({
|
||||
"memory_dir": mem_s,
|
||||
"kb_dir": kb_s,
|
||||
"task": task,
|
||||
"backend": "auto",
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
routed.get("matched").and_then(|v| v.as_bool()),
|
||||
Some(true),
|
||||
"{routed}"
|
||||
);
|
||||
assert_eq!(
|
||||
routed.get("needs_llm").and_then(|v| v.as_bool()),
|
||||
Some(false),
|
||||
"{routed}"
|
||||
);
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
[package]
|
||||
name = "marqdo_plugin_llm"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
license = "Apache-2.0"
|
||||
description = "Marqdo LLM transport plugin (ABI v2) — OpenAI-compatible chat completions"
|
||||
publish = false
|
||||
|
||||
[lib]
|
||||
name = "llm"
|
||||
crate-type = ["cdylib"]
|
||||
|
||||
[dependencies]
|
||||
serde_json = "1"
|
||||
# Pin with host (edition2024-safe for rustc 1.81).
|
||||
ureq = { version = "=2.9.7", default-features = false, features = ["tls", "gzip"] }
|
||||
idna_adapter = { version = "=1.1.0" }
|
||||
zeroize = { version = "=1.8.1" }
|
||||
@@ -0,0 +1,369 @@
|
||||
//! Marqdo LLM transport plugin (C ABI v2): OpenAI-compatible chat completions.
|
||||
|
||||
use std::ffi::{CStr, CString};
|
||||
use std::io::{BufRead, BufReader, Read};
|
||||
use std::os::raw::{c_char, c_int, c_void};
|
||||
use std::ptr;
|
||||
use std::time::Duration;
|
||||
|
||||
use serde_json::{json, Map, Value};
|
||||
|
||||
const ABI_VERSION: u32 = 2;
|
||||
const MAX_BODY: usize = 32 * 1024 * 1024;
|
||||
|
||||
type PluginFn = unsafe extern "C" fn(
|
||||
args_json: *const c_char,
|
||||
out_json: *mut *mut c_char,
|
||||
err_msg: *mut *mut c_char,
|
||||
) -> c_int;
|
||||
|
||||
type HostQueryFn = unsafe extern "C" fn(
|
||||
userdata: *mut c_void,
|
||||
name: *const c_char,
|
||||
args_json: *const c_char,
|
||||
out_json: *mut *mut c_char,
|
||||
err_msg: *mut *mut c_char,
|
||||
) -> c_int;
|
||||
|
||||
#[repr(C)]
|
||||
pub struct MarqdoHostApi {
|
||||
pub userdata: *mut c_void,
|
||||
pub register_fn: Option<
|
||||
unsafe extern "C" fn(
|
||||
userdata: *mut c_void,
|
||||
name: *const c_char,
|
||||
params: *const c_char,
|
||||
fn_ptr: PluginFn,
|
||||
) -> c_int,
|
||||
>,
|
||||
pub alloc: Option<unsafe extern "C" fn(n: usize) -> *mut c_void>,
|
||||
pub free: Option<unsafe extern "C" fn(p: *mut c_void)>,
|
||||
pub host_query: Option<HostQueryFn>,
|
||||
}
|
||||
|
||||
static mut HOST_FREE: Option<unsafe extern "C" fn(*mut c_void)> = None;
|
||||
static mut HOST_ALLOC: Option<unsafe extern "C" fn(usize) -> *mut c_void> = None;
|
||||
|
||||
unsafe fn host_strdup(s: &str) -> *mut c_char {
|
||||
let alloc = HOST_ALLOC.expect("host alloc");
|
||||
let bytes = s.as_bytes();
|
||||
let p = alloc(bytes.len() + 1) as *mut u8;
|
||||
if p.is_null() {
|
||||
return ptr::null_mut();
|
||||
}
|
||||
std::ptr::copy_nonoverlapping(bytes.as_ptr(), p, bytes.len());
|
||||
*p.add(bytes.len()) = 0;
|
||||
p as *mut c_char
|
||||
}
|
||||
|
||||
fn set_out(out: *mut *mut c_char, s: &str) {
|
||||
if out.is_null() {
|
||||
return;
|
||||
}
|
||||
unsafe {
|
||||
*out = host_strdup(s);
|
||||
}
|
||||
}
|
||||
|
||||
fn set_err(err: *mut *mut c_char, s: &str) {
|
||||
if err.is_null() {
|
||||
return;
|
||||
}
|
||||
unsafe {
|
||||
*err = host_strdup(s);
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_args(args_json: *const c_char) -> Result<Value, String> {
|
||||
if args_json.is_null() {
|
||||
return Ok(json!({}));
|
||||
}
|
||||
let s = unsafe { CStr::from_ptr(args_json) }
|
||||
.to_str()
|
||||
.map_err(|_| "args not utf-8".to_string())?;
|
||||
if s.trim().is_empty() {
|
||||
return Ok(json!({}));
|
||||
}
|
||||
serde_json::from_str(s).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn reply(out_json: *mut *mut c_char, err_msg: *mut *mut c_char, r: Result<Value, String>) -> c_int {
|
||||
match r {
|
||||
Ok(v) => {
|
||||
set_out(out_json, &v.to_string());
|
||||
0
|
||||
}
|
||||
Err(e) => {
|
||||
set_err(err_msg, &e);
|
||||
1
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn opt_text<'a>(args: &'a Value, key: &str) -> Option<&'a str> {
|
||||
args.get(key).and_then(|v| v.as_str()).filter(|s| !s.is_empty())
|
||||
}
|
||||
|
||||
fn arg_text<'a>(args: &'a Value, key: &str) -> Result<&'a str, String> {
|
||||
opt_text(args, key).ok_or_else(|| format!("missing text `{key}`"))
|
||||
}
|
||||
|
||||
fn opt_bool(args: &Value, key: &str, default: bool) -> bool {
|
||||
match args.get(key) {
|
||||
None | Some(Value::Null) => default,
|
||||
Some(Value::Bool(b)) => *b,
|
||||
Some(Value::String(s)) => matches!(s.as_str(), "true" | "True" | "1" | "yes"),
|
||||
Some(Value::Number(n)) => n.as_i64().unwrap_or(0) != 0,
|
||||
_ => default,
|
||||
}
|
||||
}
|
||||
|
||||
fn read_limited(mut r: impl Read) -> Result<String, String> {
|
||||
let mut buf = Vec::new();
|
||||
let mut chunk = [0u8; 8192];
|
||||
loop {
|
||||
let n = r.read(&mut chunk).map_err(|e| e.to_string())?;
|
||||
if n == 0 {
|
||||
break;
|
||||
}
|
||||
if buf.len() + n > MAX_BODY {
|
||||
return Err("response body too large".into());
|
||||
}
|
||||
buf.extend_from_slice(&chunk[..n]);
|
||||
}
|
||||
String::from_utf8(buf).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn usage_from(data: &Value) -> Value {
|
||||
let u = data.get("usage").cloned().unwrap_or(Value::Null);
|
||||
let pt = u.get("prompt_tokens").and_then(|v| v.as_i64()).unwrap_or(0);
|
||||
let ct = u
|
||||
.get("completion_tokens")
|
||||
.and_then(|v| v.as_i64())
|
||||
.unwrap_or(0);
|
||||
let tt = u
|
||||
.get("total_tokens")
|
||||
.and_then(|v| v.as_i64())
|
||||
.unwrap_or(pt + ct);
|
||||
json!({
|
||||
"prompt_tokens": pt,
|
||||
"completion_tokens": ct,
|
||||
"total_tokens": tt,
|
||||
})
|
||||
}
|
||||
|
||||
fn tool_calls_from(data: &Value) -> Value {
|
||||
data.get("choices")
|
||||
.and_then(|c| c.as_array())
|
||||
.and_then(|a| a.first())
|
||||
.and_then(|c0| c0.get("message"))
|
||||
.and_then(|m| m.get("tool_calls"))
|
||||
.cloned()
|
||||
.unwrap_or_else(|| json!([]))
|
||||
}
|
||||
|
||||
fn parse_sse_events(reader: impl BufRead, echo: bool) -> Result<Vec<Value>, String> {
|
||||
let mut events = Vec::new();
|
||||
let mut data_lines: Vec<String> = Vec::new();
|
||||
for line in reader.lines() {
|
||||
let line = line.map_err(|e| e.to_string())?;
|
||||
if echo {
|
||||
eprint!("{line}\n");
|
||||
}
|
||||
if line.is_empty() {
|
||||
if !data_lines.is_empty() {
|
||||
let data = data_lines.join("\n");
|
||||
data_lines.clear();
|
||||
if data.trim() == "[DONE]" {
|
||||
events.push(json!({"type": "done", "data": "[DONE]"}));
|
||||
break;
|
||||
}
|
||||
events.push(json!({"type": "data", "data": data}));
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if let Some(rest) = line.strip_prefix("data:") {
|
||||
data_lines.push(rest.trim_start().to_string());
|
||||
}
|
||||
}
|
||||
if !data_lines.is_empty() {
|
||||
let data = data_lines.join("\n");
|
||||
if data.trim() == "[DONE]" {
|
||||
events.push(json!({"type": "done", "data": "[DONE]"}));
|
||||
} else {
|
||||
events.push(json!({"type": "data", "data": data}));
|
||||
}
|
||||
}
|
||||
Ok(events)
|
||||
}
|
||||
|
||||
fn chat_completions(args: &Value) -> Result<Value, String> {
|
||||
let base_url = arg_text(args, "base_url")?;
|
||||
let api_key = arg_text(args, "api_key").unwrap_or("");
|
||||
let model = arg_text(args, "model")?;
|
||||
let prompt = arg_text(args, "prompt")?;
|
||||
let stream = opt_bool(args, "stream", false);
|
||||
let echo = opt_bool(args, "echo", false);
|
||||
let suffix = opt_text(args, "suffix").unwrap_or("/chat/completions");
|
||||
let bearer = opt_text(args, "bearer").unwrap_or("Bearer ");
|
||||
|
||||
let url = format!("{}{}", base_url.trim_end_matches('/'), suffix);
|
||||
if !(url.starts_with("http://") || url.starts_with("https://")) {
|
||||
return Err("url must start with http:// or https://".into());
|
||||
}
|
||||
|
||||
let mut req_body = Map::new();
|
||||
req_body.insert("model".into(), Value::String(model.to_string()));
|
||||
req_body.insert(
|
||||
"messages".into(),
|
||||
json!([{"role": "user", "content": prompt}]),
|
||||
);
|
||||
if stream {
|
||||
req_body.insert("stream".into(), Value::Bool(true));
|
||||
}
|
||||
let body = Value::Object(req_body).to_string();
|
||||
let auth = format!("{bearer}{api_key}");
|
||||
|
||||
let agent = ureq::AgentBuilder::new()
|
||||
.timeout_connect(Duration::from_secs(15))
|
||||
.timeout_read(Duration::from_secs(120))
|
||||
.build();
|
||||
let mut req = agent
|
||||
.post(&url)
|
||||
.set("User-Agent", "marqdo-plugin-llm")
|
||||
.set("Content-Type", "application/json; charset=utf-8");
|
||||
if stream {
|
||||
req = req.set("Accept", "text/event-stream");
|
||||
}
|
||||
if !api_key.is_empty() {
|
||||
req = req.set("Authorization", &auth);
|
||||
}
|
||||
|
||||
if stream {
|
||||
match req.send_string(&body) {
|
||||
Ok(r) => {
|
||||
let status = r.status();
|
||||
if status != 200 {
|
||||
let text = read_limited(r.into_reader())?;
|
||||
return Err(format!("HTTP {status}: {text}"));
|
||||
}
|
||||
let events = parse_sse_events(BufReader::new(r.into_reader()), echo)?;
|
||||
Ok(json!({
|
||||
"events": events,
|
||||
"finish": "stop",
|
||||
"transport": "plugin",
|
||||
}))
|
||||
}
|
||||
Err(ureq::Error::Status(code, r)) => {
|
||||
let text = read_limited(r.into_reader()).unwrap_or_default();
|
||||
Err(format!("HTTP {code}: {text}"))
|
||||
}
|
||||
Err(e) => Err(format!("http POST sse {url}: {e}")),
|
||||
}
|
||||
} else {
|
||||
match req.send_string(&body) {
|
||||
Ok(r) => {
|
||||
let status = r.status();
|
||||
let text = read_limited(r.into_reader())?;
|
||||
if status != 200 {
|
||||
return Err(format!("HTTP {status}: {text}"));
|
||||
}
|
||||
let data: Value = serde_json::from_str(&text).map_err(|e| e.to_string())?;
|
||||
let msg = data
|
||||
.get("choices")
|
||||
.and_then(|c| c.as_array())
|
||||
.and_then(|a| a.first())
|
||||
.and_then(|c0| c0.get("message"))
|
||||
.cloned()
|
||||
.unwrap_or(json!({}));
|
||||
let content = msg
|
||||
.get("content")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
let fr = data
|
||||
.get("choices")
|
||||
.and_then(|c| c.as_array())
|
||||
.and_then(|a| a.first())
|
||||
.and_then(|c0| c0.get("finish_reason"))
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("stop")
|
||||
.to_string();
|
||||
Ok(json!({
|
||||
"text": content,
|
||||
"usage": usage_from(&data),
|
||||
"finish": fr,
|
||||
"tool_calls": tool_calls_from(&data),
|
||||
"transport": "plugin",
|
||||
}))
|
||||
}
|
||||
Err(ureq::Error::Status(code, r)) => {
|
||||
let text = read_limited(r.into_reader()).unwrap_or_default();
|
||||
Err(format!("HTTP {code}: {text}"))
|
||||
}
|
||||
Err(e) => Err(format!("http POST {url}: {e}")),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
unsafe extern "C" fn llm_chat_completions(
|
||||
args_json: *const c_char,
|
||||
out_json: *mut *mut c_char,
|
||||
err_msg: *mut *mut c_char,
|
||||
) -> c_int {
|
||||
let r = (|| {
|
||||
let args = parse_args(args_json)?;
|
||||
chat_completions(&args)
|
||||
})();
|
||||
reply(out_json, err_msg, r)
|
||||
}
|
||||
|
||||
unsafe extern "C" fn llm_ping(
|
||||
_args_json: *const c_char,
|
||||
out_json: *mut *mut c_char,
|
||||
err_msg: *mut *mut c_char,
|
||||
) -> c_int {
|
||||
reply(out_json, err_msg, Ok(json!({"ok": true, "plugin": "llm"})))
|
||||
}
|
||||
|
||||
fn register(host: &MarqdoHostApi, name: &str, params: &str, fn_ptr: PluginFn) -> c_int {
|
||||
let register = match host.register_fn {
|
||||
Some(f) => f,
|
||||
None => return 1,
|
||||
};
|
||||
let n = CString::new(name).unwrap();
|
||||
let p = CString::new(params).unwrap();
|
||||
unsafe { register(host.userdata, n.as_ptr(), p.as_ptr(), fn_ptr) }
|
||||
}
|
||||
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn marqdo_plugin_abi_version() -> u32 {
|
||||
ABI_VERSION
|
||||
}
|
||||
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn marqdo_plugin_shutdown() {}
|
||||
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn marqdo_plugin_init(host: *const MarqdoHostApi) -> c_int {
|
||||
if host.is_null() {
|
||||
return 1;
|
||||
}
|
||||
let host = &*host;
|
||||
HOST_ALLOC = host.alloc;
|
||||
HOST_FREE = host.free;
|
||||
let regs = [
|
||||
("llm_ping", "", llm_ping as PluginFn),
|
||||
(
|
||||
"llm_chat_completions",
|
||||
"base_url,api_key,model,prompt,stream,echo,suffix,bearer",
|
||||
llm_chat_completions as PluginFn,
|
||||
),
|
||||
];
|
||||
for (name, params, f) in regs {
|
||||
if register(host, name, params, f) != 0 {
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
0
|
||||
}
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
|
||||
"github.com/gorilla/websocket"
|
||||
@@ -12,8 +13,19 @@ import (
|
||||
"github.com/marqdo/marqdo/plugins/web/internal/ws"
|
||||
)
|
||||
|
||||
var wsUpgrader = websocket.Upgrader{
|
||||
CheckOrigin: func(r *http.Request) bool { return true },
|
||||
const maxWSMessageBytes = 1 << 20
|
||||
|
||||
var wsUpgrader = websocket.Upgrader{CheckOrigin: sameOrigin}
|
||||
|
||||
// Browsers send Origin for a cross-site WebSocket handshake. Permit clients
|
||||
// without Origin, but reject browser origins that differ from this application.
|
||||
func sameOrigin(r *http.Request) bool {
|
||||
origin := r.Header.Get("Origin")
|
||||
if origin == "" {
|
||||
return true
|
||||
}
|
||||
u, err := url.Parse(origin)
|
||||
return err == nil && strings.EqualFold(u.Host, r.Host)
|
||||
}
|
||||
|
||||
func (st *state) mountWSRoutes(mux *http.ServeMux) {
|
||||
@@ -57,11 +69,16 @@ func pathValuesOf(r *http.Request, routePath string) map[string]string {
|
||||
}
|
||||
|
||||
func (st *state) handleWS(w http.ResponseWriter, r *http.Request, path string, spec ws.RouteSpec) {
|
||||
if spec.RequireAuth && session.UsernameFromCookie(r.Header.Get("Cookie")) == "" {
|
||||
http.Error(w, "WebSocket authentication required", http.StatusUnauthorized)
|
||||
return
|
||||
}
|
||||
conn, err := wsUpgrader.Upgrade(w, r, nil)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
defer conn.Close()
|
||||
conn.SetReadLimit(maxWSMessageBytes)
|
||||
|
||||
vals := pathValuesOf(r, path)
|
||||
switch spec.Mode {
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
package httpx
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
|
||||
"github.com/marqdo/marqdo/plugins/web/internal/ws"
|
||||
)
|
||||
|
||||
func TestHandleWSRejectsUnauthenticatedHandshake(t *testing.T) {
|
||||
st := &state{}
|
||||
r := httptest.NewRequest("GET", "http://app.example/live", nil)
|
||||
w := httptest.NewRecorder()
|
||||
st.handleWS(w, r, "/live", ws.RouteSpec{Mode: ws.ModeBroadcast, RequireAuth: true})
|
||||
if w.Code != http.StatusUnauthorized {
|
||||
t.Fatalf("unauthenticated ws handshake code=%d, want 401", w.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleWSAllowsPublicRouteThroughAuthGate(t *testing.T) {
|
||||
st := &state{}
|
||||
r := httptest.NewRequest("GET", "http://app.example/live", nil)
|
||||
w := httptest.NewRecorder()
|
||||
// Public route: must not 401. Upgrade then fails on the recorder (no
|
||||
// Hijacker), which is fine — only the auth gate is under test.
|
||||
st.handleWS(w, r, "/live", ws.RouteSpec{Mode: ws.ModeBroadcast, RequireAuth: false})
|
||||
if w.Code == http.StatusUnauthorized {
|
||||
t.Fatalf("public ws route unexpectedly requires authentication")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSameOriginWebSocketHandshake(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
origin string
|
||||
want bool
|
||||
}{
|
||||
{name: "non-browser client", want: true},
|
||||
{name: "same origin", origin: "https://app.example", want: true},
|
||||
{name: "different origin", origin: "https://evil.example", want: false},
|
||||
{name: "malformed origin", origin: "://bad", want: false},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
r := httptest.NewRequest("GET", "http://app.example/live", nil)
|
||||
if tc.origin != "" {
|
||||
r.Header.Set("Origin", tc.origin)
|
||||
}
|
||||
if got := sameOrigin(r); got != tc.want {
|
||||
t.Fatalf("sameOrigin(%q)=%v, want %v", tc.origin, got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -9,6 +9,11 @@ import (
|
||||
"github.com/gorilla/websocket"
|
||||
)
|
||||
|
||||
const (
|
||||
maxMessageBytes = 1 << 20
|
||||
maxMessages = 256
|
||||
)
|
||||
|
||||
// Connect dials url, sends one text message, collects text replies until close or timeout.
|
||||
// Mirrors Rust ws::connect; errors are returned as {ok:false, error}.
|
||||
func Connect(url, message string, headers map[string]any, timeoutSec uint64) map[string]any {
|
||||
@@ -29,6 +34,7 @@ func Connect(url, message string, headers map[string]any, timeoutSec uint64) map
|
||||
return map[string]any{"ok": false, "error": fmt.Sprintf("ws connect: %v", err)}
|
||||
}
|
||||
defer conn.Close()
|
||||
conn.SetReadLimit(maxMessageBytes)
|
||||
|
||||
if err := conn.SetWriteDeadline(time.Now().Add(timeout)); err != nil {
|
||||
return map[string]any{"ok": false, "error": fmt.Sprintf("ws send: %v", err)}
|
||||
@@ -45,9 +51,15 @@ func Connect(url, message string, headers map[string]any, timeoutSec uint64) map
|
||||
}
|
||||
mt, data, err := conn.ReadMessage()
|
||||
if err != nil {
|
||||
break
|
||||
if websocket.IsCloseError(err, websocket.CloseNormalClosure, websocket.CloseGoingAway) {
|
||||
break
|
||||
}
|
||||
return map[string]any{"ok": false, "error": fmt.Sprintf("ws receive: %v", err)}
|
||||
}
|
||||
if mt == websocket.TextMessage {
|
||||
if len(messages) == maxMessages {
|
||||
return map[string]any{"ok": false, "error": "ws receive: too many messages"}
|
||||
}
|
||||
messages = append(messages, string(data))
|
||||
}
|
||||
if mt == websocket.CloseMessage {
|
||||
|
||||
@@ -42,7 +42,22 @@ func unsubscribe(path string, ch chan string) {
|
||||
}
|
||||
room.mu.Lock()
|
||||
delete(room.subs, ch)
|
||||
empty := len(room.subs) == 0
|
||||
room.mu.Unlock()
|
||||
if !empty {
|
||||
return
|
||||
}
|
||||
// A room can be derived from a URL path. Remove empty rooms so a stream of
|
||||
// one-off room names cannot leave metadata allocated for the process lifetime.
|
||||
globalHub.mu.Lock()
|
||||
if globalHub.room[path] == room {
|
||||
room.mu.Lock()
|
||||
if len(room.subs) == 0 {
|
||||
delete(globalHub.room, path)
|
||||
}
|
||||
room.mu.Unlock()
|
||||
}
|
||||
globalHub.mu.Unlock()
|
||||
}
|
||||
|
||||
// JoinRoom subscribes to a named room (G-WS1). Same backing store as path broadcast.
|
||||
|
||||
@@ -35,6 +35,38 @@ func TestBroadcastFanout(t *testing.T) {
|
||||
unsubscribe(path, ch2)
|
||||
}
|
||||
|
||||
func TestUnsubscribeRemovesEmptyRoom(t *testing.T) {
|
||||
path := "/t-empty-room-gc"
|
||||
ch := subscribe(path)
|
||||
unsubscribe(path, ch)
|
||||
globalHub.mu.Lock()
|
||||
_, ok := globalHub.room[path]
|
||||
globalHub.mu.Unlock()
|
||||
if ok {
|
||||
t.Fatalf("empty room %q was not garbage-collected", path)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResubscribeDuringCleanupKeepsRoom(t *testing.T) {
|
||||
path := "/t-room-rejoin"
|
||||
ch := subscribe(path)
|
||||
ch2 := subscribe(path)
|
||||
unsubscribe(path, ch)
|
||||
unsubscribe(path, ch2)
|
||||
ch3 := subscribe(path)
|
||||
Publish(path, "still-alive")
|
||||
if got := readOne(ch3); got != "still-alive" {
|
||||
t.Fatalf("re-subscribed chan got %q", got)
|
||||
}
|
||||
unsubscribe(path, ch3)
|
||||
globalHub.mu.Lock()
|
||||
_, ok := globalHub.room[path]
|
||||
globalHub.mu.Unlock()
|
||||
if ok {
|
||||
t.Fatalf("empty room %q was not garbage-collected after rejoin", path)
|
||||
}
|
||||
}
|
||||
|
||||
func readOne(ch chan string) string {
|
||||
select {
|
||||
case s := <-ch:
|
||||
|
||||
@@ -9,10 +9,11 @@ import (
|
||||
|
||||
// RouteSpec is one ws_routes entry (mode + optional room_key / on_message / presence).
|
||||
type RouteSpec struct {
|
||||
Mode Mode
|
||||
RoomKey string
|
||||
OnMessage string
|
||||
Presence bool
|
||||
Mode Mode
|
||||
RoomKey string
|
||||
OnMessage string
|
||||
Presence bool
|
||||
RequireAuth bool
|
||||
}
|
||||
|
||||
// RouteWS registers path → {mode, room_key?, on_message?, presence?} on app.ws_routes.
|
||||
@@ -28,6 +29,12 @@ func RouteWS(appBag map[string]any, path string, args map[string]any) (map[strin
|
||||
}
|
||||
onMsg := OnMessageFromArgs(args)
|
||||
presence := PresenceFromArgs(args)
|
||||
// Stateful fan-out routes are private by default. Echo and drain remain
|
||||
// useful for public health checks and demos; authors can opt out explicitly.
|
||||
requireAuth := mode == ModeBroadcast || mode == ModeRoom
|
||||
if v, ok := first(args, "require_auth", "需要登录"); ok && v != nil {
|
||||
requireAuth = boolish(v)
|
||||
}
|
||||
out := clone(appBag)
|
||||
wsRoutes := map[string]any{}
|
||||
if r, ok := out["ws_routes"].(map[string]any); ok {
|
||||
@@ -43,6 +50,9 @@ func RouteWS(appBag map[string]any, path string, args map[string]any) (map[strin
|
||||
if presence {
|
||||
spec["presence"] = true
|
||||
}
|
||||
if requireAuth {
|
||||
spec["require_auth"] = true
|
||||
}
|
||||
wsRoutes[path] = spec
|
||||
out["ws_routes"] = wsRoutes
|
||||
return out, nil
|
||||
@@ -165,6 +175,11 @@ func parseRouteSpec(spec any) RouteSpec {
|
||||
} else if p, ok := t["在场"]; ok {
|
||||
out.Presence = boolish(p)
|
||||
}
|
||||
if a, ok := t["require_auth"]; ok {
|
||||
out.RequireAuth = boolish(a)
|
||||
} else if a, ok := t["需要登录"]; ok {
|
||||
out.RequireAuth = boolish(a)
|
||||
}
|
||||
case string:
|
||||
out.Mode = ParseMode(t)
|
||||
default:
|
||||
|
||||
@@ -26,6 +26,27 @@ func TestRouteWSBroadcastMode(t *testing.T) {
|
||||
if room["mode"] != "broadcast" {
|
||||
t.Fatalf("bag mode=%v", room["mode"])
|
||||
}
|
||||
if room["require_auth"] != true {
|
||||
t.Fatalf("broadcast must require authentication by default: %v", room)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRouteWSPublicBroadcastOptOut(t *testing.T) {
|
||||
a := app.New(nil)
|
||||
out, err := ws.RouteWS(a, "/live", map[string]any{
|
||||
"mode": "broadcast",
|
||||
"require_auth": false,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
spec, err := ws.RouteSpecOf(out, "/live")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if spec.RequireAuth {
|
||||
t.Fatalf("explicit public route unexpectedly requires authentication: %+v", spec)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRouteWSEchoDefault(t *testing.T) {
|
||||
|
||||
@@ -258,7 +258,7 @@ static int register_core(void) {
|
||||
if (host_register((char *)"web_app_sitemap", (char *)"app,path,base,table,loc,limit,items", web_app_sitemap) != 0) return 1;
|
||||
if (host_register((char *)"web_app_robots", (char *)"app,body,sitemap", web_app_robots) != 0) return 1;
|
||||
if (host_register((char *)"web_sitemap_build", (char *)"base,items", web_sitemap_build) != 0) return 1;
|
||||
if (host_register((char *)"web_app_route_ws", (char *)"app,path,echo,mode,room_key,on_message,presence", web_app_route_ws) != 0) return 1;
|
||||
if (host_register((char *)"web_app_route_ws", (char *)"app,path,echo,mode,room_key,on_message,presence,require_auth", web_app_route_ws) != 0) return 1;
|
||||
if (host_register((char *)"web_ws_connect", (char *)"url,message,headers,timeout_sec", web_ws_connect) != 0) return 1;
|
||||
if (host_register((char *)"web_cache_new", (char *)"url", web_cache_new) != 0) return 1;
|
||||
if (host_register((char *)"web_cache_get", (char *)"url,key", web_cache_get) != 0) return 1;
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
title: AI 原生闭环(契约 + MLSP)
|
||||
description: 渐进式契约、结构化诊断与面向 AI 的 Marqdo 语言服务
|
||||
---
|
||||
|
||||
# main
|
||||
|
||||
里程碑(v1.1.0):AI 原生闭环——让 AI 不仅能写 Marqdo,还能验证 Marqdo。
|
||||
|
||||
三件套。核心表面守卫(`doc/design/core-surface.md`)钉住 19 个核心构造,任一侧(清单 / 文档 / 金样)单边改动,CI 必红。渐进式契约是可选的 `参数` / `返回` / `字段` 表,直接嵌在叙述里,四边界校验:调用实参、返回值、表绑定、键访问。MLSP for AI(`marqdo mlsp`)是行分隔 JSON 的语言服务,AI 按需查询语法,而不是背诵整部语法。
|
||||
|
||||
静态契约检查:
|
||||
|
||||
```bash
|
||||
marqdo check tests/contracts/ok_contract.mq.md
|
||||
```
|
||||
|
||||
结构化诊断——错误即数据:
|
||||
|
||||
```bash
|
||||
marqdo run tests/structure/hello.mq.md --json
|
||||
```
|
||||
|
||||
按需查询语法(MLSP):
|
||||
|
||||
```bash
|
||||
printf '%s\n' '{"id":1,"method":"syntax","params":{"query":"返回"}}' | marqdo mlsp
|
||||
```
|
||||
|
||||
MLSP 方法:`locate`(查符号 / 构造卡片)、`syntax`(戒律原文 + 样例 + 文档锚点)、`validate`(结构化诊断)、`repair_targets`(有界靶点)、`repair_apply`(有界行编辑,越界必拒)、`schema`(单元契约)。
|
||||
|
||||
> print text=契约是可选的:不写契约表,文件保持全动态。
|
||||
|
||||
> print text=诊断自带 code / severity / suggestion / doc_anchor / doc_quote。
|
||||
|
||||
> print text=MLSP 语法卡片内嵌戒律原文与样例,AI 无需预学语法。
|
||||
|
||||
验证结果见:`doc/roadmap/perf-validation-report-2026-09-23.md`。
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
title: AI-native loop (contracts + MLSP)
|
||||
description: Progressive contracts, structured diagnostics, and the Marqdo Language Server for AI
|
||||
---
|
||||
|
||||
# main
|
||||
|
||||
Milestone (v1.1.0): the AI-native loop — so an AI can verify Marqdo, not just write it.
|
||||
|
||||
Three pieces. A core surface guard (`doc/design/core-surface.md`) pins the 19 core constructs, so a single-sided change (list / doc / golden sample) turns CI red. Progressive contracts are optional `参数` / `返回` / `字段` tables embedded in the narrative, checked at four boundaries: call argument, return, table bind, key access. MLSP for AI (`marqdo mlsp`) is a line-delimited JSON language service; the AI queries syntax on demand instead of memorizing the whole grammar.
|
||||
|
||||
Static contract check:
|
||||
|
||||
```bash
|
||||
marqdo check tests/contracts/ok_contract.mq.md
|
||||
```
|
||||
|
||||
Structured diagnostics — errors as data:
|
||||
|
||||
```bash
|
||||
marqdo run tests/structure/hello.mq.md --json
|
||||
```
|
||||
|
||||
Query syntax on demand (MLSP):
|
||||
|
||||
```bash
|
||||
printf '%s\n' '{"id":1,"method":"syntax","params":{"query":"返回"}}' | marqdo mlsp
|
||||
```
|
||||
|
||||
MLSP methods: `locate` (find a symbol / construct card), `syntax` (rule text + sample + doc anchor), `validate` (structured diagnostics), `repair_targets` (bounded span), `repair_apply` (bounded line edits, out-of-scope refused), `schema` (contract of one unit).
|
||||
|
||||
> print text=Contracts are optional: with no contract table, a file stays fully dynamic.
|
||||
|
||||
> print text=Diagnostics carry code, severity, suggestion, doc_anchor and doc_quote.
|
||||
|
||||
> print text=MLSP syntax cards embed the rule text and a sample, so AI need not pre-learn the grammar.
|
||||
|
||||
See validation results: `doc/roadmap/perf-validation-report-2026-09-23.md`.
|
||||
@@ -4,8 +4,10 @@ set -euo pipefail
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$ROOT"
|
||||
MQ="${MARQDO_BIN:-$ROOT/target/release/marqdo}"
|
||||
export MARQDO_EXT="${MARQDO_EXT:-$ROOT/.marqdo-ext}"
|
||||
export MARQDO_AGENT_PLUGIN="${MARQDO_AGENT_PLUGIN:-$ROOT/.marqdo-ext/native/libagent.so}"
|
||||
export MARQDO_EXT="${MARQDO_EXT:-$ROOT/ext}"
|
||||
export MARQDO_AGENT_PLUGIN="${MARQDO_AGENT_PLUGIN:-$ROOT/target/release/libagent.so}"
|
||||
# Offline golds that construct an llm handle need a dummy key (no network).
|
||||
export OPENAI_API_KEY="${OPENAI_API_KEY:-sk-test}"
|
||||
|
||||
if [[ ! -x "$MQ" ]]; then
|
||||
echo "missing $MQ — run: cargo build --release -p marqdo -p marqdo_plugin_agent" >&2
|
||||
@@ -28,6 +30,13 @@ run tests/ext/agent-writeback-shadow.mq.md
|
||||
run tests/ext/agent-kb-plan-hit.mq.md
|
||||
run tests/ext/agent-context-budget-a3.mq.md
|
||||
run tests/ext/agent-workbook-patch-a0.mq.md
|
||||
run tests/ext/agent-v2-learn.mq.md
|
||||
run tests/ext/agent-p4-route.mq.md
|
||||
run tests/ext/agent-small-llm-route.mq.md
|
||||
run tests/ext/agent-mcp-client.mq.md
|
||||
run tests/ext/agent-resume.mq.md
|
||||
run tests/ext/agent-zero-llm.mq.md
|
||||
run tests/ext/agent-plan-preserve.mq.md
|
||||
run examples/agent-okf-flywheel/index.mq.md
|
||||
run examples/agent-pong/index.mq.md
|
||||
|
||||
|
||||
@@ -200,17 +200,25 @@ PY
|
||||
rm -rf third_party/rust
|
||||
mkdir -p third_party
|
||||
RUST_EXTRACT="$(mktemp -d "${TMPDIR:-/tmp}/marqdo-rust.XXXXXX")"
|
||||
# Always drop the ~1.5G extract dir, even on failure (tmpfs fills up otherwise).
|
||||
trap 'rm -rf "$RUST_EXTRACT"' EXIT
|
||||
tar -xzf "$RUST_TARBALL" -C "$RUST_EXTRACT"
|
||||
INSTALL_SH="$(find "$RUST_EXTRACT" -maxdepth 2 -name install.sh | head -1)"
|
||||
if [ -z "$INSTALL_SH" ]; then
|
||||
echo "error: install.sh not found in ${RUST_TARBALL}" >&2
|
||||
exit 1
|
||||
fi
|
||||
# Launchpad builds need only rustc + cargo + rust-std (debian/rules PATH).
|
||||
# Skip docs/json-docs/rustfmt/rls — ~1G of dead weight in the orig tarball.
|
||||
bash "$INSTALL_SH" \
|
||||
--prefix="$PKG_ROOT/third_party/rust" \
|
||||
--without=rust-docs \
|
||||
--without=rust-docs-json-preview \
|
||||
--without=rustfmt-preview \
|
||||
--without=rls-preview \
|
||||
--disable-ldconfig
|
||||
rm -rf "$RUST_EXTRACT"
|
||||
trap - EXIT
|
||||
"$PKG_ROOT/third_party/rust/bin/rustc" --version
|
||||
"$PKG_ROOT/third_party/rust/bin/cargo" --version
|
||||
|
||||
|
||||
@@ -85,6 +85,7 @@ pub fn canonical_param(builtin: &str, param: &str) -> String {
|
||||
"path".into()
|
||||
}
|
||||
("host_dotenv_load" | "load_dotenv", "path" | "路径") => "path".into(),
|
||||
("host_meta_get" | "meta_get", "key" | "键") => "key".into(),
|
||||
("host_exec" | "exec", "cmd" | "命令") => "cmd".into(),
|
||||
("host_exec" | "exec", "args" | "参数") => "args".into(),
|
||||
("host_exec" | "exec", "capture" | "捕获") => "capture".into(),
|
||||
|
||||
+10
@@ -27,6 +27,10 @@ pub struct Module {
|
||||
pub functions: Vec<Function>,
|
||||
/// Populated by the loader: bind name → imported module (not flattened).
|
||||
pub import_modules: HashMap<String, Module>,
|
||||
/// Unresolved Artifact Metadata (from frontmatter); filled by parse.
|
||||
pub metadata_raw: Vec<(String, crate::binding::MetaValue)>,
|
||||
/// Bound metadata after load-time Binding resolve (insertion order).
|
||||
pub metadata: Vec<(String, crate::value::Value)>,
|
||||
}
|
||||
|
||||
impl Default for Module {
|
||||
@@ -36,6 +40,8 @@ impl Default for Module {
|
||||
uses: Vec::new(),
|
||||
functions: Vec::new(),
|
||||
import_modules: HashMap::new(),
|
||||
metadata_raw: Vec::new(),
|
||||
metadata: Vec::new(),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -61,6 +67,10 @@ pub struct Function {
|
||||
pub base: Option<String>,
|
||||
/// Prose `` `名` `` never read/assigned on the executable surface (not params).
|
||||
pub dead_binds: Vec<String>,
|
||||
/// 渐进式契约(T2.1):形参/返回/对象字段契约(附着在文档表上,升参推断前提取)。
|
||||
pub contract: Option<crate::contract::Contract>,
|
||||
/// 集合契约(T2.1):变量名 → 字段契约(紧邻 `` `名` = `` 绑定之前的文档表)。
|
||||
pub var_contracts: HashMap<String, crate::contract::Contract>,
|
||||
}
|
||||
|
||||
impl Function {
|
||||
|
||||
@@ -0,0 +1,531 @@
|
||||
//! Artifact Metadata Binding (`${ns.name}`) — Phase 1: frontmatter only.
|
||||
//!
|
||||
//! Spec: `doc/design/binding.md` · ADR 0006.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use anyhow::{bail, Result};
|
||||
|
||||
use crate::diagnostics::{bail_at, Span};
|
||||
use crate::value::Value;
|
||||
|
||||
/// Unresolved frontmatter value (literal, sole binding, or interpolated text).
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum MetaValue {
|
||||
/// Already typed literal (no `${}`).
|
||||
Literal(Value),
|
||||
/// Whole value is one `${…}` expression.
|
||||
Binding(BindingExpr),
|
||||
/// Quoted/bare string containing `${…}` interpolations → Text after resolve.
|
||||
Interpolated(String),
|
||||
}
|
||||
|
||||
/// Parsed binding expression (inside `${…}`).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct BindingExpr {
|
||||
pub cast: Option<CastKind>,
|
||||
pub primary: BindingRef,
|
||||
pub default: Option<DefaultVal>,
|
||||
pub required: bool,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum CastKind {
|
||||
Int,
|
||||
Float,
|
||||
Bool,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct BindingRef {
|
||||
pub ns: String,
|
||||
pub name: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum DefaultVal {
|
||||
Text(String),
|
||||
/// Nested `${…}` not supported in Phase 1 defaults; only quoted/bare literals.
|
||||
Binding(Box<BindingExpr>),
|
||||
}
|
||||
|
||||
/// Sources available while resolving metadata at load time.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct BindingContext {
|
||||
/// `--bind KEY=VALUE` → `arg.KEY` and post-resolve overlay.
|
||||
pub args: HashMap<String, String>,
|
||||
pub cwd: PathBuf,
|
||||
}
|
||||
|
||||
impl BindingContext {
|
||||
pub fn from_binds(binds: &[(String, String)], cwd: PathBuf) -> Self {
|
||||
let mut args = HashMap::new();
|
||||
for (k, v) in binds {
|
||||
args.insert(k.clone(), v.clone());
|
||||
}
|
||||
Self { args, cwd }
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse a frontmatter RHS into a [`MetaValue`].
|
||||
pub fn parse_meta_value(raw: &str) -> Result<MetaValue> {
|
||||
let t = raw.trim();
|
||||
if t.is_empty() {
|
||||
return Ok(MetaValue::Literal(Value::Text(String::new())));
|
||||
}
|
||||
|
||||
// Sole binding: entire value is `${…}`
|
||||
if let Some(inner) = sole_binding(t) {
|
||||
let expr = parse_binding_expr(inner)?;
|
||||
return Ok(MetaValue::Binding(expr));
|
||||
}
|
||||
|
||||
// Quoted string (may interpolate)
|
||||
if (t.starts_with('"') && t.ends_with('"') && t.len() >= 2)
|
||||
|| (t.starts_with('\'') && t.ends_with('\'') && t.len() >= 2)
|
||||
{
|
||||
let inner = &t[1..t.len() - 1];
|
||||
if inner.contains("${") {
|
||||
return Ok(MetaValue::Interpolated(unquote_escapes(inner)));
|
||||
}
|
||||
return Ok(MetaValue::Literal(Value::Text(unquote_escapes(inner))));
|
||||
}
|
||||
|
||||
// Bare with interpolation
|
||||
if t.contains("${") {
|
||||
return Ok(MetaValue::Interpolated(t.to_string()));
|
||||
}
|
||||
|
||||
Ok(MetaValue::Literal(parse_scalar_literal(t)))
|
||||
}
|
||||
|
||||
fn sole_binding(t: &str) -> Option<&str> {
|
||||
let t = t.trim();
|
||||
if !t.starts_with("${") || !t.ends_with('}') {
|
||||
return None;
|
||||
}
|
||||
// Ensure matching: first `${` closes at last `}` and nothing outside
|
||||
let inner = &t[2..t.len() - 1];
|
||||
// Reject if another `${` appears that would mean concatenation (not sole)
|
||||
// Allow nested parens inside cast: ${int(env.X ?? "1")}
|
||||
Some(inner)
|
||||
}
|
||||
|
||||
fn unquote_escapes(s: &str) -> String {
|
||||
let mut out = String::with_capacity(s.len());
|
||||
let mut chars = s.chars().peekable();
|
||||
while let Some(c) = chars.next() {
|
||||
if c == '\\' {
|
||||
match chars.next() {
|
||||
Some('n') => out.push('\n'),
|
||||
Some('t') => out.push('\t'),
|
||||
Some('\\') => out.push('\\'),
|
||||
Some('"') => out.push('"'),
|
||||
Some('\'') => out.push('\''),
|
||||
Some(o) => {
|
||||
out.push('\\');
|
||||
out.push(o);
|
||||
}
|
||||
None => out.push('\\'),
|
||||
}
|
||||
} else {
|
||||
out.push(c);
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn parse_scalar_literal(t: &str) -> Value {
|
||||
match t {
|
||||
"true" | "True" | "TRUE" => Value::Bool(true),
|
||||
"false" | "False" | "FALSE" => Value::Bool(false),
|
||||
"null" | "Null" | "None" | "~" => Value::None,
|
||||
_ => {
|
||||
if let Ok(n) = t.parse::<i64>() {
|
||||
return Value::Int(n);
|
||||
}
|
||||
if let Ok(n) = t.parse::<f64>() {
|
||||
if !n.is_nan() && !n.is_infinite() {
|
||||
return Value::Num(n);
|
||||
}
|
||||
}
|
||||
Value::Text(t.to_string())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse inside `${…}` (without braces).
|
||||
pub fn parse_binding_expr(inner: &str) -> Result<BindingExpr> {
|
||||
let s = inner.trim();
|
||||
// cast: int(…) / float(…) / bool(…)
|
||||
if let Some((cast, rest)) = parse_cast_wrap(s)? {
|
||||
let mut expr = parse_binding_expr(rest)?;
|
||||
if expr.cast.is_some() {
|
||||
bail!("nested casts are not allowed in binding expressions");
|
||||
}
|
||||
expr.cast = Some(cast);
|
||||
return Ok(expr);
|
||||
}
|
||||
|
||||
let (body, required) = if let Some(b) = s.strip_suffix('!') {
|
||||
(b.trim(), true)
|
||||
} else {
|
||||
(s, false)
|
||||
};
|
||||
|
||||
let (primary_s, default) = if let Some((left, right)) = split_default(body) {
|
||||
(left.trim(), Some(parse_default_val(right.trim())?))
|
||||
} else {
|
||||
(body, None)
|
||||
};
|
||||
|
||||
if required && default.is_some() {
|
||||
bail!("binding cannot combine `!` with `??` default");
|
||||
}
|
||||
|
||||
let primary = parse_binding_ref(primary_s)?;
|
||||
Ok(BindingExpr {
|
||||
cast: None,
|
||||
primary,
|
||||
default,
|
||||
required,
|
||||
})
|
||||
}
|
||||
|
||||
fn parse_cast_wrap(s: &str) -> Result<Option<(CastKind, &str)>> {
|
||||
for (name, kind) in [
|
||||
("int", CastKind::Int),
|
||||
("float", CastKind::Float),
|
||||
("bool", CastKind::Bool),
|
||||
] {
|
||||
let prefix = format!("{name}(");
|
||||
if let Some(rest) = s.strip_prefix(&prefix) {
|
||||
if let Some(inner) = strip_trailing_paren(rest) {
|
||||
return Ok(Some((kind, inner)));
|
||||
}
|
||||
bail!("unclosed `{name}(` in binding expression");
|
||||
}
|
||||
}
|
||||
Ok(None)
|
||||
}
|
||||
|
||||
fn strip_trailing_paren(s: &str) -> Option<&str> {
|
||||
let s = s.trim_end();
|
||||
if !s.ends_with(')') {
|
||||
return None;
|
||||
}
|
||||
Some(s[..s.len() - 1].trim())
|
||||
}
|
||||
|
||||
fn split_default(s: &str) -> Option<(&str, &str)> {
|
||||
// Split on top-level `??`
|
||||
let bytes = s.as_bytes();
|
||||
let mut i = 0;
|
||||
let mut depth = 0i32;
|
||||
while i + 1 < bytes.len() {
|
||||
match bytes[i] {
|
||||
b'(' => depth += 1,
|
||||
b')' => depth -= 1,
|
||||
b'?' if depth == 0 && bytes[i + 1] == b'?' => {
|
||||
return Some((&s[..i], &s[i + 2..]));
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
i += 1;
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
fn parse_default_val(s: &str) -> Result<DefaultVal> {
|
||||
let s = s.trim();
|
||||
if let Some(inner) = sole_binding(s) {
|
||||
return Ok(DefaultVal::Binding(Box::new(parse_binding_expr(inner)?)));
|
||||
}
|
||||
if (s.starts_with('"') && s.ends_with('"') && s.len() >= 2)
|
||||
|| (s.starts_with('\'') && s.ends_with('\'') && s.len() >= 2)
|
||||
{
|
||||
return Ok(DefaultVal::Text(unquote_escapes(&s[1..s.len() - 1])));
|
||||
}
|
||||
// bare default token
|
||||
Ok(DefaultVal::Text(s.to_string()))
|
||||
}
|
||||
|
||||
fn parse_binding_ref(s: &str) -> Result<BindingRef> {
|
||||
let s = s.trim();
|
||||
let Some((ns, name)) = s.split_once('.') else {
|
||||
bail!("binding path must be `namespace.name` (got `{s}`)");
|
||||
};
|
||||
let ns = ns.trim();
|
||||
let name = name.trim();
|
||||
if ns.is_empty() || name.is_empty() {
|
||||
bail!("binding path must be `namespace.name` (got `{s}`)");
|
||||
}
|
||||
match ns {
|
||||
"env" | "arg" | "sys" | "secret" => {}
|
||||
"config" | "file" => {
|
||||
bail!("binding namespace `{ns}` is not available in Phase 1 (Metadata-only)")
|
||||
}
|
||||
other => bail!("unknown binding namespace `{other}`"),
|
||||
}
|
||||
if name.contains('.') {
|
||||
// Phase 1: single segment name only
|
||||
bail!("binding name must be a single segment (got `{s}`)");
|
||||
}
|
||||
Ok(BindingRef {
|
||||
ns: ns.to_string(),
|
||||
name: name.to_string(),
|
||||
})
|
||||
}
|
||||
|
||||
/// Resolve raw metadata entries; then overlay `--bind` keys.
|
||||
pub fn resolve_metadata(
|
||||
raw: &[(String, MetaValue)],
|
||||
ctx: &BindingContext,
|
||||
span: Span,
|
||||
) -> Result<Vec<(String, Value)>> {
|
||||
let mut out: Vec<(String, Value)> = Vec::with_capacity(raw.len());
|
||||
for (k, mv) in raw {
|
||||
let v = resolve_meta_value(mv, ctx, span)
|
||||
.map_err(|e| bail_at(None, span, e.to_string()))?;
|
||||
out.push((k.clone(), v));
|
||||
}
|
||||
// CLI overlay
|
||||
for (k, v) in &ctx.args {
|
||||
if let Some(slot) = out.iter_mut().find(|(mk, _)| mk == k) {
|
||||
slot.1 = Value::Text(v.clone());
|
||||
} else {
|
||||
out.push((k.clone(), Value::Text(v.clone())));
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn resolve_meta_value(mv: &MetaValue, ctx: &BindingContext, span: Span) -> Result<Value> {
|
||||
match mv {
|
||||
MetaValue::Literal(v) => Ok(v.clone()),
|
||||
MetaValue::Binding(expr) => resolve_expr(expr, ctx, span),
|
||||
MetaValue::Interpolated(s) => {
|
||||
let text = interpolate(s, ctx, span)?;
|
||||
Ok(Value::Text(text))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn interpolate(s: &str, ctx: &BindingContext, span: Span) -> Result<String> {
|
||||
let mut out = String::new();
|
||||
let mut rest = s;
|
||||
while let Some(start) = rest.find("${") {
|
||||
out.push_str(&rest[..start]);
|
||||
let after = &rest[start + 2..];
|
||||
let end = find_binding_close(after).ok_or_else(|| {
|
||||
anyhow::anyhow!("{}:{}: unclosed `${{` in metadata interpolation", span.line, span.col)
|
||||
})?;
|
||||
let inner = &after[..end];
|
||||
let expr = parse_binding_expr(inner)?;
|
||||
let v = resolve_expr(&expr, ctx, span)?;
|
||||
out.push_str(&v.as_display());
|
||||
rest = &after[end + 1..];
|
||||
}
|
||||
out.push_str(rest);
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn find_binding_close(s: &str) -> Option<usize> {
|
||||
let mut depth = 0i32;
|
||||
for (i, c) in s.char_indices() {
|
||||
match c {
|
||||
'(' => depth += 1,
|
||||
')' => depth -= 1,
|
||||
'}' if depth == 0 => return Some(i),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
fn resolve_expr(expr: &BindingExpr, ctx: &BindingContext, span: Span) -> Result<Value> {
|
||||
let got = lookup(&expr.primary, ctx)?;
|
||||
let value = match got {
|
||||
Some(v) => v,
|
||||
None if expr.required => {
|
||||
bail!(
|
||||
"required binding `{}.{}` is missing",
|
||||
expr.primary.ns,
|
||||
expr.primary.name
|
||||
)
|
||||
}
|
||||
None => match &expr.default {
|
||||
Some(DefaultVal::Text(t)) => Value::Text(t.clone()),
|
||||
Some(DefaultVal::Binding(inner)) => resolve_expr(inner, ctx, span)?,
|
||||
None => Value::None,
|
||||
},
|
||||
};
|
||||
apply_cast(expr.cast, value)
|
||||
}
|
||||
|
||||
fn lookup(r: &BindingRef, ctx: &BindingContext) -> Result<Option<Value>> {
|
||||
match r.ns.as_str() {
|
||||
"env" => Ok(std::env::var(&r.name).ok().map(Value::Text)),
|
||||
"arg" => Ok(ctx.args.get(&r.name).cloned().map(Value::Text)),
|
||||
"secret" => Ok(std::env::var(&r.name).ok().map(Value::Secret)),
|
||||
"sys" => Ok(lookup_sys(&r.name, ctx)),
|
||||
other => bail!("unknown binding namespace `{other}`"),
|
||||
}
|
||||
}
|
||||
|
||||
fn lookup_sys(name: &str, ctx: &BindingContext) -> Option<Value> {
|
||||
match name {
|
||||
"cwd" => Some(Value::Text(ctx.cwd.display().to_string())),
|
||||
"platform" => Some(Value::Text(std::env::consts::OS.to_string())),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
fn apply_cast(cast: Option<CastKind>, v: Value) -> Result<Value> {
|
||||
let Some(c) = cast else {
|
||||
return Ok(v);
|
||||
};
|
||||
let text = match &v {
|
||||
Value::None => {
|
||||
bail!("cannot cast None to {:?}", c);
|
||||
}
|
||||
Value::Text(s) => s.clone(),
|
||||
Value::Secret(s) => s.clone(),
|
||||
Value::Int(n) => n.to_string(),
|
||||
Value::Num(n) => n.to_string(),
|
||||
Value::Bool(b) => {
|
||||
if *b {
|
||||
"true".into()
|
||||
} else {
|
||||
"false".into()
|
||||
}
|
||||
}
|
||||
other => bail!("cannot cast {} to {:?}", other.as_display(), c),
|
||||
};
|
||||
match c {
|
||||
CastKind::Int => {
|
||||
let n: i64 = text
|
||||
.trim()
|
||||
.parse()
|
||||
.map_err(|_| anyhow::anyhow!("cannot cast `{text}` to int"))?;
|
||||
Ok(Value::Int(n))
|
||||
}
|
||||
CastKind::Float => {
|
||||
let n: f64 = text
|
||||
.trim()
|
||||
.parse()
|
||||
.map_err(|_| anyhow::anyhow!("cannot cast `{text}` to float"))?;
|
||||
Ok(Value::Num(n))
|
||||
}
|
||||
CastKind::Bool => {
|
||||
let t = text.trim().to_ascii_lowercase();
|
||||
match t.as_str() {
|
||||
"1" | "true" | "yes" | "on" => Ok(Value::Bool(true)),
|
||||
"0" | "false" | "no" | "off" => Ok(Value::Bool(false)),
|
||||
_ => bail!("cannot cast `{text}` to bool"),
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Split `KEY=VALUE` from CLI `--bind`.
|
||||
pub fn parse_bind_kv(s: &str) -> Result<(String, String)> {
|
||||
let Some((k, v)) = s.split_once('=') else {
|
||||
bail!("--bind expects KEY=VALUE (got `{s}`)");
|
||||
};
|
||||
let k = k.trim();
|
||||
if k.is_empty() {
|
||||
bail!("--bind key must be non-empty");
|
||||
}
|
||||
Ok((k.to_string(), v.to_string()))
|
||||
}
|
||||
|
||||
pub fn cwd_for_path(path: Option<&Path>) -> PathBuf {
|
||||
path.and_then(|p| p.parent())
|
||||
.map(|p| {
|
||||
if p.as_os_str().is_empty() {
|
||||
PathBuf::from(".")
|
||||
} else {
|
||||
p.to_path_buf()
|
||||
}
|
||||
})
|
||||
.unwrap_or_else(|| std::env::current_dir().unwrap_or_else(|_| PathBuf::from(".")))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn sole_env_default() {
|
||||
let mv = parse_meta_value(r#"${env.NO_SUCH_MARQDO_VAR_XYZ ?? "gpt"}"#).unwrap();
|
||||
let ctx = BindingContext::default();
|
||||
let v = resolve_meta_value(&mv, &ctx, Span::new(1, 1)).unwrap();
|
||||
assert_eq!(v, Value::Text("gpt".into()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn required_missing_errors() {
|
||||
let mv = parse_meta_value("${env.NO_SUCH_MARQDO_REQ_XYZ!}").unwrap();
|
||||
let ctx = BindingContext::default();
|
||||
let err = resolve_meta_value(&mv, &ctx, Span::new(1, 1)).unwrap_err();
|
||||
assert!(err.to_string().contains("required binding"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn secret_masks_display() {
|
||||
std::env::set_var("MARQDO_TEST_SECRET_BIND", "s3cr3t");
|
||||
let mv = parse_meta_value("${secret.MARQDO_TEST_SECRET_BIND}").unwrap();
|
||||
let ctx = BindingContext::default();
|
||||
let v = resolve_meta_value(&mv, &ctx, Span::new(1, 1)).unwrap();
|
||||
assert_eq!(v.as_display(), "<secret>");
|
||||
match v {
|
||||
Value::Secret(s) => assert_eq!(s, "s3cr3t"),
|
||||
other => panic!("expected Secret, got {other:?}"),
|
||||
}
|
||||
std::env::remove_var("MARQDO_TEST_SECRET_BIND");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn literal_not_overridden_by_env() {
|
||||
std::env::set_var("OPENAI_MODEL", "from-env");
|
||||
let raw = vec![(
|
||||
"model".into(),
|
||||
MetaValue::Literal(Value::Text("gpt-literal".into())),
|
||||
)];
|
||||
let ctx = BindingContext::default();
|
||||
let meta = resolve_metadata(&raw, &ctx, Span::new(1, 1)).unwrap();
|
||||
assert_eq!(meta[0].1, Value::Text("gpt-literal".into()));
|
||||
std::env::remove_var("OPENAI_MODEL");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bind_overlay_wins() {
|
||||
let raw = vec![(
|
||||
"model".into(),
|
||||
MetaValue::Literal(Value::Text("gpt-literal".into())),
|
||||
)];
|
||||
let ctx = BindingContext::from_binds(&[("model".into(), "from-cli".into())], PathBuf::from("."));
|
||||
let meta = resolve_metadata(&raw, &ctx, Span::new(1, 1)).unwrap();
|
||||
assert_eq!(meta[0].1, Value::Text("from-cli".into()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn int_cast() {
|
||||
std::env::set_var("MARQDO_TEST_MAX_TOKENS", "2048");
|
||||
let mv = parse_meta_value("${int(env.MARQDO_TEST_MAX_TOKENS)}").unwrap();
|
||||
let v = resolve_meta_value(&mv, &BindingContext::default(), Span::new(1, 1)).unwrap();
|
||||
assert_eq!(v, Value::Int(2048));
|
||||
std::env::remove_var("MARQDO_TEST_MAX_TOKENS");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn interpolate_string() {
|
||||
std::env::set_var("MARQDO_TEST_USER", "Ada");
|
||||
let mv = parse_meta_value(r#""Hello ${env.MARQDO_TEST_USER}""#).unwrap();
|
||||
let v = resolve_meta_value(&mv, &BindingContext::default(), Span::new(1, 1)).unwrap();
|
||||
assert_eq!(v, Value::Text("Hello Ada".into()));
|
||||
std::env::remove_var("MARQDO_TEST_USER");
|
||||
}
|
||||
}
|
||||
@@ -41,6 +41,7 @@ pub fn builtin_type(v: &Value) -> Value {
|
||||
Value::Int(_) => "int",
|
||||
Value::Num(_) => "num",
|
||||
Value::Text(_) => "text",
|
||||
Value::Secret(_) => "secret",
|
||||
Value::List(_) => "list",
|
||||
Value::Map(_) => "map",
|
||||
Value::Formula(_) => "formula",
|
||||
|
||||
+31
-2
@@ -79,7 +79,12 @@ pub fn compile_module(path: Option<&Path>, module: &Module) -> Result<Program> {
|
||||
let path_buf = path.map(|p| p.to_path_buf());
|
||||
let mut functions = Vec::new();
|
||||
for i in 0..flat.len() {
|
||||
functions.push(compile_function(i, &flat, path_buf.as_deref())?);
|
||||
let meta = if i == entry {
|
||||
Some(module.metadata.as_slice())
|
||||
} else {
|
||||
None
|
||||
};
|
||||
functions.push(compile_function(i, &flat, path_buf.as_deref(), meta)?);
|
||||
}
|
||||
|
||||
let parents: Vec<Option<usize>> = flat.iter().map(|f| f.parent).collect();
|
||||
@@ -175,7 +180,12 @@ struct FnCompiler<'a> {
|
||||
stmt_span: Span,
|
||||
}
|
||||
|
||||
fn compile_function(fn_id: usize, flat: &[FlatFun], path: Option<&Path>) -> Result<FnChunk> {
|
||||
fn compile_function(
|
||||
fn_id: usize,
|
||||
flat: &[FlatFun],
|
||||
path: Option<&Path>,
|
||||
metadata: Option<&[(String, Value)]>,
|
||||
) -> Result<FnChunk> {
|
||||
let fun = &flat[fn_id];
|
||||
let mut locals = HashMap::new();
|
||||
for (i, p) in fun.params.iter().enumerate() {
|
||||
@@ -193,6 +203,15 @@ fn compile_function(fn_id: usize, flat: &[FlatFun], path: Option<&Path>) -> Resu
|
||||
}
|
||||
}
|
||||
}
|
||||
// Phase 2: reserve locals for entry metadata keys before body compile.
|
||||
if let Some(meta) = metadata {
|
||||
for (k, _) in meta {
|
||||
if !locals.contains_key(k) {
|
||||
let s = locals.len() as u8;
|
||||
locals.insert(k.clone(), s);
|
||||
}
|
||||
}
|
||||
}
|
||||
let mut c = FnCompiler {
|
||||
path,
|
||||
flat,
|
||||
@@ -208,6 +227,16 @@ fn compile_function(fn_id: usize, flat: &[FlatFun], path: Option<&Path>) -> Resu
|
||||
locals,
|
||||
stmt_span: fun.span,
|
||||
};
|
||||
// Prologue: seed entry locals from bound metadata.
|
||||
if let Some(meta) = metadata {
|
||||
for (k, v) in meta {
|
||||
let slot = *c.locals.get(k).expect("metadata local reserved");
|
||||
let ci = c.add_const(v.clone());
|
||||
c.emit(Op::Constant(ci));
|
||||
c.emit(Op::SetLocal(slot));
|
||||
c.emit(Op::Pop);
|
||||
}
|
||||
}
|
||||
for stmt in &fun.body {
|
||||
c.compile_stmt(stmt)?;
|
||||
}
|
||||
|
||||
@@ -698,6 +698,9 @@ fn add(a: Value, b: Value) -> Result<Value, String> {
|
||||
(Value::Int(x), Value::Num(y)) => Ok(Value::Num(x as f64 + y)),
|
||||
(Value::Num(x), Value::Int(y)) => Ok(Value::Num(x + y as f64)),
|
||||
(Value::Text(x), Value::Text(y)) => Ok(Value::Text(format!("{x}{y}"))),
|
||||
(Value::Text(x), Value::Secret(y)) => Ok(Value::Text(format!("{x}{y}"))),
|
||||
(Value::Secret(x), Value::Text(y)) => Ok(Value::Text(format!("{x}{y}"))),
|
||||
(Value::Secret(x), Value::Secret(y)) => Ok(Value::Text(format!("{x}{y}"))),
|
||||
(Value::Text(x), y) => Ok(Value::Text(format!("{x}{}", y.as_display()))),
|
||||
(x, Value::Text(y)) => Ok(Value::Text(format!("{}{y}", x.as_display()))),
|
||||
(Value::Int(x), y) => Ok(Value::Text(format!("{}{}", x, y.as_display()))),
|
||||
|
||||
+213
@@ -0,0 +1,213 @@
|
||||
//! `marqdo check` —— 静态契约互查(T2.3 + T2.5)。
|
||||
//!
|
||||
//! 只读静态检查,不执行程序。三类必产诊断:
|
||||
//!
|
||||
//! 1. **防漂移(T2.5)**:契约行 vs 升参推断形参**双向互查**——契约表一经出现
|
||||
//! 就是该单元形参的完整声明:契约多出的名字(错行)与未被覆盖的推断形参
|
||||
//! (新升参)都是 `contract.param_drift`。
|
||||
//! 2. **未知类型名**:`Ty::Object` 必须是已知对象类型(防 `nubmer` 拼错,
|
||||
//! 附最近建议)⇒ `contract.unknown_type`。
|
||||
//! 3. **错位契约**:契约形状(含 `类型` 列)的未绑定表出现在不允许位置
|
||||
//! (提取阶段原样留在 body 里的就是)⇒ `contract.misplaced`。
|
||||
//!
|
||||
//! 任一 Error ⇒ 进程以非零码退出。诊断全部带 `doc_anchor` 锚点(T2.4)。
|
||||
|
||||
use std::path::Path;
|
||||
|
||||
use anyhow::Result;
|
||||
|
||||
use crate::ast::{Expr, Function, Module, Stmt};
|
||||
use crate::contract::{self, Contract, Ty};
|
||||
use crate::diagnostics::{Diagnostic, Severity, Span};
|
||||
use crate::load;
|
||||
|
||||
/// 读取文件 + 装载(含 import)后做全量契约互查。
|
||||
pub fn check_path(path: &Path) -> Result<Vec<Diagnostic>> {
|
||||
let module = load::load_module(path)?;
|
||||
Ok(check_module(&module, Some(path)))
|
||||
}
|
||||
|
||||
/// 递归检查模块全部函数(含子函数)的契约。
|
||||
pub fn check_module(module: &Module, path: Option<&Path>) -> Vec<Diagnostic> {
|
||||
let known = known_type_names(module);
|
||||
let mut out = Vec::new();
|
||||
walk(module, &module.functions, path, &known, &mut out);
|
||||
out
|
||||
}
|
||||
|
||||
/// 已知对象类型名:本文件 `#` 对象 + import/use 绑定 + 导入库的 `#` 对象。
|
||||
fn known_type_names(module: &Module) -> Vec<String> {
|
||||
let mut names = Vec::new();
|
||||
for f in &module.functions {
|
||||
if f.is_object() {
|
||||
names.push(f.name.clone());
|
||||
}
|
||||
}
|
||||
for i in &module.imports {
|
||||
names.push(i.bind.clone());
|
||||
}
|
||||
for u in &module.uses {
|
||||
names.push(u.bind.clone());
|
||||
}
|
||||
for lib in module.import_modules.values() {
|
||||
for f in &lib.functions {
|
||||
if f.is_object() {
|
||||
names.push(f.name.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
names
|
||||
}
|
||||
|
||||
fn walk(
|
||||
module: &Module,
|
||||
funs: &[Function],
|
||||
path: Option<&Path>,
|
||||
known: &[String],
|
||||
out: &mut Vec<Diagnostic>,
|
||||
) {
|
||||
for fun in funs {
|
||||
check_one(module, fun, path, known, out);
|
||||
walk(module, &fun.children, path, known, out);
|
||||
}
|
||||
}
|
||||
|
||||
fn check_one(
|
||||
module: &Module,
|
||||
fun: &Function,
|
||||
path: Option<&Path>,
|
||||
known: &[String],
|
||||
out: &mut Vec<Diagnostic>,
|
||||
) {
|
||||
if let Some(c) = &fun.contract {
|
||||
for p in &c.params {
|
||||
type_name_check(&p.ty, c, &fun.name, path, fun.span, known, out);
|
||||
}
|
||||
if let Some(r) = &c.returns {
|
||||
type_name_check(&r.ty, c, &fun.name, path, fun.span, known, out);
|
||||
}
|
||||
for f in &c.fields {
|
||||
type_name_check(&f.ty, c, &fun.name, path, fun.span, known, out);
|
||||
}
|
||||
// ---- T2.5 防漂移:契约行 vs 升参推断形参(双向) ----
|
||||
let inferred: Vec<String> = fun.params.iter().map(|p| p.name.clone()).collect();
|
||||
for p in &c.params {
|
||||
if !inferred.contains(&p.name) {
|
||||
out.push(c.diagnostic(
|
||||
"contract.param_drift",
|
||||
&fun.name,
|
||||
format!(
|
||||
"契约参数 `{}` 不是 `{}` 的形参(当前形参:[{}])",
|
||||
p.name,
|
||||
fun.name,
|
||||
inferred.join(", ")
|
||||
),
|
||||
"删除契约表该行,或让该名在函数体被读取(升参)/ 用 `+` 行显式声明".to_string(),
|
||||
path,
|
||||
fun.span,
|
||||
));
|
||||
}
|
||||
}
|
||||
for name in &inferred {
|
||||
if !c.params.iter().any(|p| p.name == *name) {
|
||||
let d = c
|
||||
.diagnostic(
|
||||
"contract.param_drift",
|
||||
&fun.name,
|
||||
format!(
|
||||
"形参 `{name}` 未被契约覆盖(可能是新推断的升参;契约表一经出现即为完整声明)"
|
||||
),
|
||||
"在契约表补一行 `| {name} | 类型 | 说明 |`,或删掉契约表回到纯动态".to_string(),
|
||||
path,
|
||||
fun.span,
|
||||
)
|
||||
.with_severity(Severity::Warning);
|
||||
out.push(d);
|
||||
}
|
||||
}
|
||||
}
|
||||
for (name, c) in &fun.var_contracts {
|
||||
for f in &c.fields {
|
||||
type_name_check(&f.ty, c, name, path, fun.span, known, out);
|
||||
}
|
||||
}
|
||||
// ---- 错位契约:契约形状的未绑定表被提取器原样留在 body ----
|
||||
for stmt in &fun.body {
|
||||
let Stmt::Expr { value: Expr::Map(pairs), span } = stmt else {
|
||||
continue;
|
||||
};
|
||||
if let Some(Ok(c)) = contract::parse_contract_table(pairs, span.line, span.line, Vec::new())
|
||||
{
|
||||
out.push(c.diagnostic(
|
||||
"contract.misplaced",
|
||||
&fun.name,
|
||||
"契约表位置不允许(形参/返回表只能在函数体首个可执行行之前;字段表只能紧邻绑定或对象体首成员之前),已按普通表处理".to_string(),
|
||||
"移动契约表到允许位置,或改表头避开契约形状(含 `类型` 列)".to_string(),
|
||||
path,
|
||||
*span,
|
||||
));
|
||||
}
|
||||
}
|
||||
let _ = module;
|
||||
}
|
||||
|
||||
/// 对象类型名必须已知;拼错(`nubmer`)⇒ `contract.unknown_type` + 最近建议。
|
||||
fn type_name_check(
|
||||
ty: &Ty,
|
||||
c: &Contract,
|
||||
unit: &str,
|
||||
path: Option<&Path>,
|
||||
span: Span,
|
||||
known: &[String],
|
||||
out: &mut Vec<Diagnostic>,
|
||||
) {
|
||||
let Ty::Object(name) = ty else { return };
|
||||
if known.iter().any(|k| k == name) {
|
||||
return;
|
||||
}
|
||||
let suggestion = match suggest(name, known) {
|
||||
Some(near) => format!("是不是想写 `{near}`?已知类型:{}", known.join(" / ")),
|
||||
None => format!("已知类型:{}", known.join(" / ")),
|
||||
};
|
||||
out.push(c.diagnostic(
|
||||
"contract.unknown_type",
|
||||
unit,
|
||||
format!("未知类型名 `{name}`(不是原语,也没有对应的 `#` 对象)"),
|
||||
suggestion,
|
||||
path,
|
||||
span,
|
||||
));
|
||||
}
|
||||
|
||||
const PRIMITIVES: &[&str] = &["text", "number", "bool", "list", "map", "any"];
|
||||
|
||||
fn edit_distance(a: &str, b: &str) -> usize {
|
||||
let a: Vec<char> = a.chars().collect();
|
||||
let b: Vec<char> = b.chars().collect();
|
||||
let mut prev: Vec<usize> = (0..=b.len()).collect();
|
||||
let mut cur = vec![0usize; b.len() + 1];
|
||||
for i in 1..=a.len() {
|
||||
cur[0] = i;
|
||||
for j in 1..=b.len() {
|
||||
let cost = usize::from(a[i - 1] != b[j - 1]);
|
||||
cur[j] = (prev[j] + 1).min(cur[j - 1] + 1).min(prev[j - 1] + cost);
|
||||
}
|
||||
std::mem::swap(&mut prev, &mut cur);
|
||||
}
|
||||
prev[b.len()]
|
||||
}
|
||||
|
||||
fn suggest(name: &str, known: &[String]) -> Option<String> {
|
||||
let lower = name.to_ascii_lowercase();
|
||||
let mut best: Option<(usize, String)> = None;
|
||||
let candidates = PRIMITIVES.iter().map(|s| s.to_string()).chain(
|
||||
known.iter().cloned(),
|
||||
);
|
||||
for cand in candidates {
|
||||
let d = edit_distance(&lower, &cand.to_ascii_lowercase());
|
||||
if d <= 2 && best.as_ref().map(|(bd, _)| d < *bd).unwrap_or(true) {
|
||||
best = Some((d, cand));
|
||||
}
|
||||
}
|
||||
best.map(|(_, c)| c)
|
||||
}
|
||||
+633
@@ -0,0 +1,633 @@
|
||||
//! Progressive contracts (渐进式契约 / 标注类型) —— schema 长在表格/说明段落里。
|
||||
//!
|
||||
//! 动态默认、按需校验、零契约文件零回归;TS/TS 的全程序静态类型被明确**拒绝**。
|
||||
//! 契约唯一来源是**未绑定文档表**(绑定表是数据,永不视为契约):
|
||||
//!
|
||||
//! | 形状(含 `类型` 列) | 允许位置 | 附着对象 |
|
||||
//! |----------------------|----------|----------|
|
||||
//! | `参数`/`返回` 表 | 函数体首个可执行行之前 | 函数(形参/返回契约) |
|
||||
//! | `字段` 表 | 紧邻 `` `名` = `` 绑定之前 | 变量(集合契约) |
|
||||
//! | `字段` 表 | 对象(`#`)体首成员之前 | 对象类型(字段契约) |
|
||||
//!
|
||||
//! 提取在**升参推断之前**执行(见 `parse`),契约表的单元不会污染推断。
|
||||
//! 校验只发生在四个运行时边界(调用实参 / 字段访问 / 表绑定 / 返回)与
|
||||
//! `marqdo check` 静态互查;契约表本身永不执行、永不进入值空间。
|
||||
//! 锚点诊断见 [`Contract::anchor`] / [`Contract::quote_text`](T2.4)。
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::path::Path;
|
||||
|
||||
use anyhow::Result;
|
||||
|
||||
use crate::ast::Expr;
|
||||
use crate::diagnostics::{ContractRef, Diagnostic, Severity, Span};
|
||||
use crate::value::Value;
|
||||
|
||||
/// 契约类型词汇(`doc/design/core-surface.md` 的文档层扩展,不进语言核心)。
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum Ty {
|
||||
/// 任意值(类型列留空)。
|
||||
Any,
|
||||
Text,
|
||||
/// `int` / `num`(`Value::Int` / `Value::Num` / 符号公式)。
|
||||
Number,
|
||||
Bool,
|
||||
List,
|
||||
Map,
|
||||
/// 对象类型名(`#` 对象;`check` 阶段校验存在性,防 `nubmer` 拼错)。
|
||||
Object(String),
|
||||
}
|
||||
|
||||
impl Ty {
|
||||
/// 解析类型列文本(大小写不敏感;未知名 → `Object(name)`,由 check 验证)。
|
||||
pub fn parse(raw: &str) -> Ty {
|
||||
let name = raw.trim().trim_matches('`').to_ascii_lowercase();
|
||||
match name.as_str() {
|
||||
"" | "any" | "任意" => Ty::Any,
|
||||
"text" | "str" | "string" | "文本" | "字串" | "字符串" => Ty::Text,
|
||||
"number" | "int" | "float" | "num" | "numeric" | "数字" | "数值" => Ty::Number,
|
||||
"bool" | "boolean" | "布尔" => Ty::Bool,
|
||||
"list" | "列表" | "表" => Ty::List,
|
||||
"map" | "dict" | "object" | "字典" | "映射" => Ty::Map,
|
||||
_ => Ty::Object(raw.trim().trim_matches('`').to_string()),
|
||||
}
|
||||
}
|
||||
|
||||
/// 契约类型名(诊断消息显示用;与 `type` 内置的运行时名对照)。
|
||||
pub fn name(&self) -> String {
|
||||
match self {
|
||||
Ty::Any => "any".into(),
|
||||
Ty::Text => "text".into(),
|
||||
Ty::Number => "number".into(),
|
||||
Ty::Bool => "bool".into(),
|
||||
Ty::List => "list".into(),
|
||||
Ty::Map => "map".into(),
|
||||
Ty::Object(n) => n.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
/// 值是否满足契约类型(对象类型看 `_type` 实例标签)。
|
||||
pub fn matches_value(&self, v: &Value) -> bool {
|
||||
match self {
|
||||
Ty::Any => true,
|
||||
Ty::Text => matches!(v, Value::Text(_)),
|
||||
// 符号公式($$…$$)是数值表达式,按 number 收。
|
||||
Ty::Number => matches!(v, Value::Int(_) | Value::Num(_) | Value::Formula(_)),
|
||||
Ty::Bool => matches!(v, Value::Bool(_)),
|
||||
Ty::List => matches!(v, Value::List(_)),
|
||||
Ty::Map => matches!(v, Value::Map(_)),
|
||||
Ty::Object(n) => instance_type_name(v).as_deref() == Some(n.as_str()),
|
||||
}
|
||||
}
|
||||
|
||||
/// 列语义(几何即类型):标量按本类型,或**元素全匹配**的列向量。
|
||||
pub fn matches_column(&self, v: &Value) -> bool {
|
||||
if self.matches_value(v) {
|
||||
return true;
|
||||
}
|
||||
match v {
|
||||
Value::List(items) => items.iter().all(|it| self.matches_value(it)),
|
||||
Value::Map(pairs) => pairs
|
||||
.iter()
|
||||
.all(|(_, it)| match it {
|
||||
// map-of-lists 的列向量
|
||||
Value::List(xs) => xs.iter().all(|x| self.matches_value(x)),
|
||||
scalar => self.matches_value(scalar),
|
||||
})
|
||||
&& !pairs.is_empty(),
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 运行时 `type` 名(诊断消息里的「实际」侧)。
|
||||
pub fn type_of(v: &Value) -> &'static str {
|
||||
match v {
|
||||
Value::None => "none",
|
||||
Value::Bool(_) => "bool",
|
||||
Value::Int(_) => "int",
|
||||
Value::Num(_) => "num",
|
||||
Value::Text(_) => "text",
|
||||
Value::Secret(_) => "secret",
|
||||
Value::List(_) => "list",
|
||||
Value::Map(_) => "map",
|
||||
Value::Formula(_) => "formula",
|
||||
Value::Code(_) => "code",
|
||||
}
|
||||
}
|
||||
|
||||
fn instance_type_name(v: &Value) -> Option<String> {
|
||||
let Value::Map(pairs) = v else { return None };
|
||||
pairs
|
||||
.iter()
|
||||
.find(|(k, _)| k == "_type")
|
||||
.and_then(|(_, v)| match v {
|
||||
Value::Text(t) => Some(t.clone()),
|
||||
_ => None,
|
||||
})
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ParamContract {
|
||||
pub name: String,
|
||||
pub ty: Ty,
|
||||
pub desc: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ReturnContract {
|
||||
pub ty: Ty,
|
||||
pub desc: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct FieldContract {
|
||||
pub name: String,
|
||||
pub ty: Ty,
|
||||
pub nullable: bool,
|
||||
pub desc: String,
|
||||
}
|
||||
|
||||
/// 一份契约(一个契约表块;同单元多表由 [`merge_contract`] 合并)。
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Contract {
|
||||
pub params: Vec<ParamContract>,
|
||||
pub returns: Option<ReturnContract>,
|
||||
pub fields: Vec<FieldContract>,
|
||||
pub start_line: u32,
|
||||
pub end_line: u32,
|
||||
/// 契约表原文行(诊断 `doc_quote`)。
|
||||
pub quote: Vec<String>,
|
||||
}
|
||||
|
||||
impl Contract {
|
||||
/// 文档锚点(T2.4):`file.mq.md#L4-L7`。
|
||||
pub fn anchor(&self, path: Option<&Path>) -> String {
|
||||
match path {
|
||||
Some(p) => format!("{}#L{}-L{}", p.display(), self.start_line, self.end_line),
|
||||
None => format!("#L{}-L{}", self.start_line, self.end_line),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn quote_text(&self) -> String {
|
||||
self.quote.join("\n")
|
||||
}
|
||||
|
||||
/// 契约声明摘要(诊断 `contract_ref.declared`)。
|
||||
pub fn describe(&self) -> String {
|
||||
let mut parts = Vec::new();
|
||||
if !self.params.is_empty() {
|
||||
let list = self
|
||||
.params
|
||||
.iter()
|
||||
.map(|p| format!("{}: {}", p.name, p.ty.name()))
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ");
|
||||
parts.push(format!("参数[{list}]"));
|
||||
}
|
||||
if let Some(r) = &self.returns {
|
||||
parts.push(format!("返回 {}", r.ty.name()));
|
||||
}
|
||||
if !self.fields.is_empty() {
|
||||
let list = self
|
||||
.fields
|
||||
.iter()
|
||||
.map(|f| format!("{}: {}", f.name, f.ty.name()))
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ");
|
||||
parts.push(format!("字段[{list}]"));
|
||||
}
|
||||
parts.join("; ")
|
||||
}
|
||||
|
||||
/// 装配带契约上下文的诊断(code + suggestion + doc_anchor + doc_quote + contract_ref)。
|
||||
pub fn diagnostic(
|
||||
&self,
|
||||
code: &str,
|
||||
unit: &str,
|
||||
message: String,
|
||||
suggestion: String,
|
||||
path: Option<&Path>,
|
||||
span: Span,
|
||||
) -> Diagnostic {
|
||||
Diagnostic::new(path, span, message)
|
||||
.with_code(code)
|
||||
.with_severity(Severity::Error)
|
||||
.with_suggestion(suggestion)
|
||||
.with_doc_anchor(self.anchor(path))
|
||||
.with_doc_quote(self.quote_text())
|
||||
.with_contract_ref(ContractRef {
|
||||
unit: unit.to_string(),
|
||||
kind: "contract".to_string(),
|
||||
name: unit.to_string(),
|
||||
declared: self.describe(),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// 把第二个契约表合并进第一个(多表块;重复行 → Err 防两份契约打架)。
|
||||
pub fn merge_contract(dst: &mut Contract, src: Contract) -> std::result::Result<(), String> {
|
||||
for p in src.params {
|
||||
if dst.params.iter().any(|q| q.name == p.name) {
|
||||
return Err(format!("契约参数 `{}` 重复(防两份契约打架)", p.name));
|
||||
}
|
||||
dst.params.push(p);
|
||||
}
|
||||
if let Some(r) = src.returns {
|
||||
if dst.returns.is_some() {
|
||||
return Err("返回契约重复(防两份契约打架)".to_string());
|
||||
}
|
||||
dst.returns = Some(r);
|
||||
}
|
||||
for f in src.fields {
|
||||
if dst.fields.iter().any(|g| g.name == f.name) {
|
||||
return Err(format!("契约字段 `{}` 重复(防两份契约打架)", f.name));
|
||||
}
|
||||
dst.fields.push(f);
|
||||
}
|
||||
dst.start_line = dst.start_line.min(src.start_line);
|
||||
dst.end_line = dst.end_line.max(src.end_line);
|
||||
for q in src.quote {
|
||||
if !dst.quote.contains(&q) {
|
||||
dst.quote.push(q);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// ---------- 契约表识别与解析 ----------
|
||||
|
||||
const PARAM_HEADERS: &[&str] = &["参数", "形参", "param", "params", "arg", "args"];
|
||||
const RETURN_HEADERS: &[&str] = &["返回", "返回值", "return", "returns", "result"];
|
||||
const FIELD_HEADERS: &[&str] = &["字段", "列", "键", "field", "fields", "key", "keys", "column", "columns"];
|
||||
const TYPE_HEADERS: &[&str] = &["类型", "type"];
|
||||
const DESC_HEADERS: &[&str] = &["说明", "描述", "备注", "注释", "desc", "description"];
|
||||
const NULLABLE_HEADERS: &[&str] = &["可空", "nullable", "optional"];
|
||||
|
||||
fn norm_header(h: &str) -> String {
|
||||
h.trim().trim_matches('`').to_ascii_lowercase()
|
||||
}
|
||||
|
||||
fn header_is(h: &str, vocab: &[&str]) -> bool {
|
||||
let n = norm_header(h);
|
||||
vocab.iter().any(|v| *v == n)
|
||||
}
|
||||
|
||||
fn find_col(headers: &[String], vocab: &[&str]) -> Option<usize> {
|
||||
headers.iter().position(|h| header_is(h, vocab))
|
||||
}
|
||||
|
||||
/// 单元格 → 文本(text / `` `ident` `` / 数字);嵌套表 → None(那是数据表)。
|
||||
fn cell_to_text(e: &Expr) -> Option<String> {
|
||||
match e {
|
||||
Expr::Literal(crate::ast::Literal::Text(s)) => Some(s.clone()),
|
||||
Expr::Var(name) => Some(name.clone()),
|
||||
Expr::Literal(crate::ast::Literal::Int(i)) => Some(i.to_string()),
|
||||
Expr::Literal(crate::ast::Literal::Num(n)) => Some(n.to_string()),
|
||||
Expr::Literal(crate::ast::Literal::Bool(b)) => Some(b.to_string()),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// 列值 → 行文本(标量 = 单行;列向量 = 多行;其它 = None)。
|
||||
fn col_rows(e: &Expr) -> Option<Vec<String>> {
|
||||
match e {
|
||||
Expr::List(items) => items.iter().map(cell_to_text).collect(),
|
||||
scalar => Some(vec![cell_to_text(scalar)?]),
|
||||
}
|
||||
}
|
||||
|
||||
fn is_nullable(raw: &str) -> bool {
|
||||
matches!(
|
||||
norm_header(raw).as_str(),
|
||||
"是" | "true" | "yes" | "y" | "可空" | "✓"
|
||||
)
|
||||
}
|
||||
|
||||
/// 未绑定文档表(`Expr::Map`)→ 契约块。
|
||||
///
|
||||
/// - `None`:非契约形状(普通数据/说明表,原样保留)。
|
||||
/// - `Some(Err)`:契约形状但畸形(缺参数名、列不齐、多行返回表…)——必产诊断。
|
||||
/// - `Some(Ok)`:合法契约(`参数`/`返回`/`字段` 三形状之一,须含 `类型` 列)。
|
||||
pub fn parse_contract_table(
|
||||
pairs: &[(String, Expr)],
|
||||
start_line: u32,
|
||||
end_line: u32,
|
||||
quote: Vec<String>,
|
||||
) -> Option<Result<Contract>> {
|
||||
let headers: Vec<String> = pairs.iter().map(|(k, _)| k.clone()).collect();
|
||||
let kind = if find_col(&headers, PARAM_HEADERS).is_some() {
|
||||
1
|
||||
} else if find_col(&headers, RETURN_HEADERS).is_some() {
|
||||
2
|
||||
} else if find_col(&headers, FIELD_HEADERS).is_some() {
|
||||
3
|
||||
} else {
|
||||
return None;
|
||||
};
|
||||
// 契约表必须有 `类型` 列(否则是普通说明表——如 `| 参数 | 说明 |`)。
|
||||
let Some(type_col) = find_col(&headers, TYPE_HEADERS) else {
|
||||
return None;
|
||||
};
|
||||
|
||||
let meta = || (start_line, end_line, quote.clone());
|
||||
Some(match kind {
|
||||
1 => parse_params_table(pairs, &headers, type_col, meta()),
|
||||
2 => parse_return_table(pairs, &headers, type_col, meta()),
|
||||
_ => parse_fields_table(pairs, &headers, type_col, meta()),
|
||||
})
|
||||
}
|
||||
|
||||
fn contract(
|
||||
meta: (u32, u32, Vec<String>),
|
||||
) -> Contract {
|
||||
Contract {
|
||||
params: Vec::new(),
|
||||
returns: None,
|
||||
fields: Vec::new(),
|
||||
start_line: meta.0,
|
||||
end_line: meta.1,
|
||||
quote: meta.2,
|
||||
}
|
||||
}
|
||||
|
||||
fn malformed(meta: (u32, u32, Vec<String>), span_line: u32, msg: String) -> Result<Contract> {
|
||||
let (start, end, quote) = meta;
|
||||
let c = Contract {
|
||||
params: Vec::new(),
|
||||
returns: None,
|
||||
fields: Vec::new(),
|
||||
start_line: start,
|
||||
end_line: end,
|
||||
quote,
|
||||
};
|
||||
Err(c
|
||||
.diagnostic(
|
||||
"contract.malformed",
|
||||
"(契约表)",
|
||||
msg,
|
||||
"按 `参数|类型|说明` / `返回|类型|说明` / `字段|类型|可空|说明` 形状修正契约表".into(),
|
||||
None,
|
||||
Span { line: span_line, col: 1 },
|
||||
)
|
||||
.into())
|
||||
}
|
||||
|
||||
fn parse_params_table(
|
||||
pairs: &[(String, Expr)],
|
||||
headers: &[String],
|
||||
type_col: usize,
|
||||
meta: (u32, u32, Vec<String>),
|
||||
) -> Result<Contract> {
|
||||
let start_line = meta.0;
|
||||
let name_col = find_col(headers, PARAM_HEADERS).unwrap();
|
||||
let desc_col = find_col(headers, DESC_HEADERS);
|
||||
let names = col_rows(&pairs[name_col].1).unwrap_or_default();
|
||||
let types = col_rows(&pairs[type_col].1).unwrap_or_default();
|
||||
let descs = desc_col
|
||||
.and_then(|c| col_rows(&pairs[c].1))
|
||||
.unwrap_or_default();
|
||||
if names.len() != types.len() || (!descs.is_empty() && descs.len() != names.len()) {
|
||||
return malformed(meta, start_line, "契约表列不齐(参数/类型/说明 行数不一致)".into());
|
||||
}
|
||||
let mut c = contract(meta.clone());
|
||||
for (i, name) in names.iter().enumerate() {
|
||||
let name = name.trim();
|
||||
if name.is_empty() {
|
||||
return malformed(meta, start_line, format!("契约表第 {} 行缺参数名", i + 1));
|
||||
}
|
||||
c.params.push(ParamContract {
|
||||
name: name.to_string(),
|
||||
ty: Ty::parse(&types[i]),
|
||||
desc: descs.get(i).cloned().unwrap_or_default(),
|
||||
});
|
||||
}
|
||||
Ok(c)
|
||||
}
|
||||
|
||||
fn parse_return_table(
|
||||
pairs: &[(String, Expr)],
|
||||
headers: &[String],
|
||||
type_col: usize,
|
||||
meta: (u32, u32, Vec<String>),
|
||||
) -> Result<Contract> {
|
||||
let start_line = meta.0;
|
||||
let desc_col = find_col(headers, DESC_HEADERS);
|
||||
let types = col_rows(&pairs[type_col].1).unwrap_or_default();
|
||||
let descs = desc_col
|
||||
.and_then(|c| col_rows(&pairs[c].1))
|
||||
.unwrap_or_default();
|
||||
if types.len() != 1 {
|
||||
return malformed(meta, start_line, "返回契约只能一行(`返回|类型|说明`)".into());
|
||||
}
|
||||
let mut c = contract(meta.clone());
|
||||
c.returns = Some(ReturnContract {
|
||||
ty: Ty::parse(&types[0]),
|
||||
desc: descs.first().cloned().unwrap_or_default(),
|
||||
});
|
||||
Ok(c)
|
||||
}
|
||||
|
||||
fn parse_fields_table(
|
||||
pairs: &[(String, Expr)],
|
||||
headers: &[String],
|
||||
type_col: usize,
|
||||
meta: (u32, u32, Vec<String>),
|
||||
) -> Result<Contract> {
|
||||
let start_line = meta.0;
|
||||
let name_col = find_col(headers, FIELD_HEADERS).unwrap();
|
||||
let desc_col = find_col(headers, DESC_HEADERS);
|
||||
let null_col = find_col(headers, NULLABLE_HEADERS);
|
||||
let names = col_rows(&pairs[name_col].1).unwrap_or_default();
|
||||
let types = col_rows(&pairs[type_col].1).unwrap_or_default();
|
||||
if names.len() != types.len() {
|
||||
return malformed(meta, start_line, "契约表列不齐(字段/类型 行数不一致)".into());
|
||||
}
|
||||
let mut c = contract(meta.clone());
|
||||
for (i, name) in names.iter().enumerate() {
|
||||
let name = name.trim();
|
||||
if name.is_empty() {
|
||||
return malformed(meta, start_line, format!("契约表第 {} 行缺字段名", i + 1));
|
||||
}
|
||||
c.fields.push(FieldContract {
|
||||
name: name.to_string(),
|
||||
ty: Ty::parse(&types[i]),
|
||||
nullable: null_col
|
||||
.and_then(|c| col_rows(&pairs[c].1))
|
||||
.and_then(|rows| rows.get(i).cloned())
|
||||
.map(|s| is_nullable(&s))
|
||||
.unwrap_or(false),
|
||||
desc: desc_col
|
||||
.and_then(|c| col_rows(&pairs[c].1))
|
||||
.and_then(|rows| rows.get(i).cloned())
|
||||
.unwrap_or_default(),
|
||||
});
|
||||
}
|
||||
Ok(c)
|
||||
}
|
||||
|
||||
// ---------- 运行时边界校验(T2.2,四个边界) ----------
|
||||
|
||||
/// 边界 1:调用实参 vs 形参契约。
|
||||
pub fn call_arg_diag(
|
||||
c: &Contract,
|
||||
unit: &str,
|
||||
bound: &HashMap<String, Value>,
|
||||
path: Option<&Path>,
|
||||
span: Span,
|
||||
) -> Option<Diagnostic> {
|
||||
for p in &c.params {
|
||||
let Some(v) = bound.get(&p.name) else { continue };
|
||||
if v == &Value::None {
|
||||
continue; // 显式 None = 未填,交给缺参/默认值语义
|
||||
}
|
||||
if !p.ty.matches_value(v) {
|
||||
return Some(c.diagnostic(
|
||||
"contract.arg_mismatch",
|
||||
unit,
|
||||
format!(
|
||||
"参数 `{}` 期望 {},实际 {}(契约校验)",
|
||||
p.name,
|
||||
p.ty.name(),
|
||||
type_of(v)
|
||||
),
|
||||
format!(
|
||||
"把实参改为 {},或修正契约表 `参数/类型` 列(锚点 {})",
|
||||
p.ty.name(),
|
||||
c.anchor(path)
|
||||
),
|
||||
path,
|
||||
span,
|
||||
));
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// 边界 2:返回值 vs 返回契约。
|
||||
pub fn return_diag(
|
||||
c: &Contract,
|
||||
unit: &str,
|
||||
v: &Value,
|
||||
path: Option<&Path>,
|
||||
span: Span,
|
||||
) -> Option<Diagnostic> {
|
||||
let r = c.returns.as_ref()?;
|
||||
if !r.ty.matches_value(v) {
|
||||
return Some(c.diagnostic(
|
||||
"contract.return_mismatch",
|
||||
unit,
|
||||
format!(
|
||||
"返回值期望 {},实际 {}(契约校验)",
|
||||
r.ty.name(),
|
||||
type_of(v)
|
||||
),
|
||||
format!(
|
||||
"修正返回值,或调整契约表 `返回|类型`(锚点 {})",
|
||||
c.anchor(path)
|
||||
),
|
||||
path,
|
||||
span,
|
||||
));
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// 边界 3:表绑定 vs 字段契约(集合契约;几何即类型——标量或列向量皆可)。
|
||||
pub fn fields_diag(
|
||||
c: &Contract,
|
||||
unit: &str,
|
||||
v: &Value,
|
||||
path: Option<&Path>,
|
||||
span: Span,
|
||||
) -> Option<Diagnostic> {
|
||||
if c.fields.is_empty() {
|
||||
return None;
|
||||
}
|
||||
let Value::Map(pairs) = v else {
|
||||
return Some(c.diagnostic(
|
||||
"contract.field_mismatch",
|
||||
unit,
|
||||
format!("绑定值期望表(map),实际 {}(契约校验)", type_of(v)),
|
||||
"绑定一个表值,或删除该字段契约表".into(),
|
||||
path,
|
||||
span,
|
||||
));
|
||||
};
|
||||
for f in &c.fields {
|
||||
match pairs.iter().find(|(k, _)| k == &f.name) {
|
||||
Some((_, val)) => {
|
||||
if !f.ty.matches_column(val) {
|
||||
return Some(c.diagnostic(
|
||||
"contract.field_mismatch",
|
||||
unit,
|
||||
format!(
|
||||
"字段 `{}` 期望 {},实际 {}(契约校验)",
|
||||
f.name,
|
||||
f.ty.name(),
|
||||
type_of(val)
|
||||
),
|
||||
format!("修正数据,或调整契约表 `字段/类型`(锚点 {})", c.anchor(path)),
|
||||
path,
|
||||
span,
|
||||
));
|
||||
}
|
||||
}
|
||||
None if f.nullable => {}
|
||||
None => {
|
||||
return Some(c.diagnostic(
|
||||
"contract.field_mismatch",
|
||||
unit,
|
||||
format!("缺少字段 `{}`(契约要求,不可空)", f.name),
|
||||
format!("补上字段 `{}`,或在契约表 `可空` 列标 `是`(锚点 {})", f.name, c.anchor(path)),
|
||||
path,
|
||||
span,
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// 边界 4:字段访问 `[键](集合)` vs 字段契约(键存在性 + 取值类型)。
|
||||
pub fn index_diag(
|
||||
c: &Contract,
|
||||
unit: &str,
|
||||
key: &str,
|
||||
v: &Value,
|
||||
path: Option<&Path>,
|
||||
span: Span,
|
||||
) -> Option<Diagnostic> {
|
||||
let Some(f) = c.fields.iter().find(|f| f.name == key) else {
|
||||
return Some(c.diagnostic(
|
||||
"contract.unknown_key",
|
||||
unit,
|
||||
format!("字段 `{key}` 不在契约中(契约校验)"),
|
||||
format!(
|
||||
"使用契约内的字段({}),或在契约表补一行",
|
||||
c.fields
|
||||
.iter()
|
||||
.map(|f| f.name.as_str())
|
||||
.collect::<Vec<_>>()
|
||||
.join(" / ")
|
||||
),
|
||||
path,
|
||||
span,
|
||||
));
|
||||
};
|
||||
if v != &Value::None && !f.ty.matches_column(v) {
|
||||
return Some(c.diagnostic(
|
||||
"contract.field_mismatch",
|
||||
unit,
|
||||
format!(
|
||||
"字段 `{key}` 期望 {},实际 {}(契约校验)",
|
||||
f.ty.name(),
|
||||
type_of(v)
|
||||
),
|
||||
format!("修正数据,或调整契约表 `字段/类型`(锚点 {})", c.anchor(path)),
|
||||
path,
|
||||
span,
|
||||
));
|
||||
}
|
||||
None
|
||||
}
|
||||
@@ -41,13 +41,40 @@ pub fn display_path(path: &Path) -> String {
|
||||
}
|
||||
}
|
||||
|
||||
/// Diagnostic severity.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum Severity {
|
||||
Error,
|
||||
Warning,
|
||||
}
|
||||
|
||||
/// Contract reference (Phase 2 填值;schema 源头 = 契约表 / 说明段落).
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct ContractRef {
|
||||
pub unit: String,
|
||||
pub kind: String,
|
||||
pub name: String,
|
||||
pub declared: String,
|
||||
}
|
||||
|
||||
/// Unified diagnostic: `path:line:col: message` (path omitted if unknown).
|
||||
///
|
||||
/// 人类文本(`Display`)与机器面([`Diagnostic::to_json`])同源:凡可定位必有
|
||||
/// `span`;凡契约错必带 `doc_anchor` / `doc_quote` / `contract_ref`
|
||||
/// (见 `doc/roadmap/three-problems.md` §3.3 诊断协议)。
|
||||
#[derive(Debug, Error, Clone)]
|
||||
#[error("{}", self.format_message())]
|
||||
pub struct Diagnostic {
|
||||
pub path: Option<PathBuf>,
|
||||
pub span: Span,
|
||||
pub message: String,
|
||||
/// Stable machine code (`parse.*` / `contract.*` / `runtime.*` …).
|
||||
pub code: String,
|
||||
pub severity: Severity,
|
||||
pub suggestion: Option<String>,
|
||||
pub doc_anchor: Option<String>,
|
||||
pub doc_quote: Option<String>,
|
||||
pub contract_ref: Option<ContractRef>,
|
||||
}
|
||||
|
||||
impl Diagnostic {
|
||||
@@ -56,6 +83,12 @@ impl Diagnostic {
|
||||
path: path.map(|p| p.to_path_buf()),
|
||||
span,
|
||||
message: message.into(),
|
||||
code: "marqdo.error".to_string(),
|
||||
severity: Severity::Error,
|
||||
suggestion: None,
|
||||
doc_anchor: None,
|
||||
doc_quote: None,
|
||||
contract_ref: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -63,12 +96,74 @@ impl Diagnostic {
|
||||
Self::new(Some(path), span, message)
|
||||
}
|
||||
|
||||
pub fn with_code(mut self, code: impl Into<String>) -> Self {
|
||||
self.code = code.into();
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_severity(mut self, severity: Severity) -> Self {
|
||||
self.severity = severity;
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_suggestion(mut self, suggestion: impl Into<String>) -> Self {
|
||||
self.suggestion = Some(suggestion.into());
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_doc_anchor(mut self, anchor: impl Into<String>) -> Self {
|
||||
self.doc_anchor = Some(anchor.into());
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_doc_quote(mut self, quote: impl Into<String>) -> Self {
|
||||
self.doc_quote = Some(quote.into());
|
||||
self
|
||||
}
|
||||
|
||||
pub fn with_contract_ref(mut self, contract_ref: ContractRef) -> Self {
|
||||
self.contract_ref = Some(contract_ref);
|
||||
self
|
||||
}
|
||||
|
||||
/// 从 anyhow 错误链中找回结构化诊断(load/parse 包了 context 也不丢;MLSP 面)。
|
||||
pub fn find(err: &anyhow::Error) -> Option<&Diagnostic> {
|
||||
err.chain().find_map(|e| e.downcast_ref::<Diagnostic>())
|
||||
}
|
||||
|
||||
pub fn format_message(&self) -> String {
|
||||
match &self.path {
|
||||
Some(p) => format!("{}:{}: {}", display_path(p), self.span, self.message),
|
||||
None => format!("{}: {}", self.span, self.message),
|
||||
}
|
||||
}
|
||||
|
||||
/// Machine-readable form(AI/MLSP 面;`run --json` 出口)。
|
||||
pub fn to_json(&self) -> serde_json::Value {
|
||||
use serde_json::json;
|
||||
json!({
|
||||
"code": self.code,
|
||||
"severity": match self.severity {
|
||||
Severity::Error => "error",
|
||||
Severity::Warning => "warning",
|
||||
},
|
||||
"message": self.message,
|
||||
"span": {
|
||||
"file": self.path.as_ref().map(|p| display_path(p)),
|
||||
"line": self.span.line,
|
||||
"col": self.span.col,
|
||||
},
|
||||
"doc_anchor": self.doc_anchor,
|
||||
"doc_quote": self.doc_quote,
|
||||
"contract_ref": self.contract_ref.as_ref().map(|c| json!({
|
||||
"unit": c.unit,
|
||||
"kind": c.kind,
|
||||
"name": c.name,
|
||||
"declared": c.declared,
|
||||
})),
|
||||
"suggestion": self.suggestion,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Convenience for `anyhow` sites that already have a span.
|
||||
@@ -106,4 +201,41 @@ mod tests {
|
||||
assert!(!msg.contains(r"\\?\"));
|
||||
assert!(msg.contains(r"E:\marqdo\x.mq.md:7:1:"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn json_shape_defaults_nonempty_code_and_span() {
|
||||
let d = Diagnostic::at(Path::new("x.mq.md"), Span::new(3, 5), "boom");
|
||||
let v = d.to_json();
|
||||
assert_eq!(v["code"], "marqdo.error");
|
||||
assert_eq!(v["severity"], "error");
|
||||
assert_eq!(v["span"]["line"], 3);
|
||||
assert_eq!(v["span"]["col"], 5);
|
||||
assert_eq!(v["span"]["file"], "x.mq.md");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn json_carries_doc_anchor_and_contract_ref() {
|
||||
let d = Diagnostic::at(
|
||||
Path::new("demo.mq.md"),
|
||||
Span::new(12, 3),
|
||||
"参数 `n` 期望 number,实际 text",
|
||||
)
|
||||
.with_code("contract.arg_mismatch")
|
||||
.with_doc_anchor("demo.mq.md#L4-L7")
|
||||
.with_doc_quote("| `n` | number | 输入值 |")
|
||||
.with_suggestion("把实参改为数字")
|
||||
.with_contract_ref(ContractRef {
|
||||
unit: "加一".to_string(),
|
||||
kind: "param".to_string(),
|
||||
name: "n".to_string(),
|
||||
declared: "number".to_string(),
|
||||
});
|
||||
let v = d.to_json();
|
||||
assert_eq!(v["code"], "contract.arg_mismatch");
|
||||
assert_eq!(v["doc_anchor"], "demo.mq.md#L4-L7");
|
||||
assert_eq!(v["contract_ref"]["unit"], "加一");
|
||||
assert_eq!(v["suggestion"], "把实参改为数字");
|
||||
// Display 零回归
|
||||
assert!(d.format_message().contains("demo.mq.md:12:3:"));
|
||||
}
|
||||
}
|
||||
+12
-4
@@ -21,8 +21,13 @@ pub const CATALOG: &[ExtPackage] = &[
|
||||
ExtPackage {
|
||||
id: "llm",
|
||||
description: "OpenAI-compatible chat object (ext/ai/llm)",
|
||||
mq_files: &["ai/llm.mq.md", "ai/大模型.mq.md"],
|
||||
native_crate: None,
|
||||
mq_files: &[
|
||||
"ai/llm.mq.md",
|
||||
"ai/大模型.mq.md",
|
||||
"ai/llm/openai.mq.md",
|
||||
"ai/llm/ollama.mq.md",
|
||||
],
|
||||
native_crate: Some("marqdo_plugin_llm"),
|
||||
},
|
||||
ExtPackage {
|
||||
id: "agent",
|
||||
@@ -98,6 +103,7 @@ pub fn native_short_name(crate_or_id: &str) -> &str {
|
||||
match crate_or_id {
|
||||
"marqdo_plugin_agent" | "agent" => "agent",
|
||||
"marqdo_plugin_web" | "web" => "web",
|
||||
"marqdo_plugin_llm" | "llm" => "llm",
|
||||
"marqdo_plugin_quantum" | "quantum" => "quantum",
|
||||
"marqdo_plugin_linalg" | "linalg" => "linalg",
|
||||
other => other,
|
||||
@@ -118,6 +124,7 @@ pub fn native_lib_filename(short: &str) -> String {
|
||||
pub fn native_env_var(short: &str) -> &'static str {
|
||||
match native_short_name(short) {
|
||||
"web" => "MARQDO_WEB_PLUGIN",
|
||||
"llm" => "MARQDO_LLM_PLUGIN",
|
||||
"quantum" => "MARQDO_QUANTUM_PLUGIN",
|
||||
"linalg" => "MARQDO_LINALG_PLUGIN",
|
||||
_ => "MARQDO_AGENT_PLUGIN",
|
||||
@@ -507,7 +514,7 @@ pub fn remove_ext(id: &str) -> Result<()> {
|
||||
/// point at a freshly built plugin path.
|
||||
pub fn installed_native_path(name: &str) -> Option<PathBuf> {
|
||||
let short = native_short_name(name);
|
||||
if !matches!(short, "agent" | "web" | "quantum" | "linalg") {
|
||||
if !matches!(short, "agent" | "web" | "llm" | "quantum" | "linalg") {
|
||||
return None;
|
||||
}
|
||||
let lib_name = native_lib_filename(short);
|
||||
@@ -549,6 +556,7 @@ pub fn path_is_trusted_plugin(path: &Path) -> bool {
|
||||
for key in [
|
||||
"MARQDO_AGENT_PLUGIN",
|
||||
"MARQDO_WEB_PLUGIN",
|
||||
"MARQDO_LLM_PLUGIN",
|
||||
"MARQDO_QUANTUM_PLUGIN",
|
||||
"MARQDO_LINALG_PLUGIN",
|
||||
] {
|
||||
@@ -560,7 +568,7 @@ pub fn path_is_trusted_plugin(path: &Path) -> bool {
|
||||
}
|
||||
}
|
||||
}
|
||||
for short in ["agent", "web", "quantum", "linalg"] {
|
||||
for short in ["agent", "web", "llm", "quantum", "linalg"] {
|
||||
if let Ok(found) = find_native_plugin(short) {
|
||||
let found = found.canonicalize().unwrap_or(found);
|
||||
if found == path {
|
||||
|
||||
+19
-1
@@ -198,6 +198,9 @@ pub enum HostFn {
|
||||
StatsMean = 189,
|
||||
StatsMedian = 190,
|
||||
StatsStdev = 191,
|
||||
Meta = 192,
|
||||
MetaGet = 193,
|
||||
ModuleSource = 194,
|
||||
}
|
||||
|
||||
|
||||
@@ -386,6 +389,9 @@ impl HostFn {
|
||||
189 => Self::StatsMean,
|
||||
190 => Self::StatsMedian,
|
||||
191 => Self::StatsStdev,
|
||||
192 => Self::Meta,
|
||||
193 => Self::MetaGet,
|
||||
194 => Self::ModuleSource,
|
||||
_ => return None,
|
||||
})
|
||||
}
|
||||
@@ -577,6 +583,9 @@ impl HostFn {
|
||||
"host_stats_mean" | "stats_mean" => Self::StatsMean,
|
||||
"host_stats_median" | "stats_median" => Self::StatsMedian,
|
||||
"host_stats_stdev" | "stats_stdev" => Self::StatsStdev,
|
||||
"host_meta" | "meta" => Self::Meta,
|
||||
"host_meta_get" | "meta_get" => Self::MetaGet,
|
||||
"host_module_source" | "module_source" => Self::ModuleSource,
|
||||
_ => return None,
|
||||
})
|
||||
}
|
||||
@@ -755,6 +764,9 @@ impl HostFn {
|
||||
Self::StatsMean => "host_stats_mean",
|
||||
Self::StatsMedian => "host_stats_median",
|
||||
Self::StatsStdev => "host_stats_stdev",
|
||||
Self::Meta => "host_meta",
|
||||
Self::MetaGet => "host_meta_get",
|
||||
Self::ModuleSource => "host_module_source",
|
||||
Self::WritebackRecord => "host_writeback_record",
|
||||
Self::WritebackGet => "host_writeback_get",
|
||||
Self::WritebackClear => "host_writeback_clear",
|
||||
@@ -778,11 +790,14 @@ impl HostFn {
|
||||
&["path"]
|
||||
}
|
||||
Self::WriteText | Self::AppendText => &["path", "text"],
|
||||
Self::NowUnix | Self::NowMs | Self::Args | Self::Cwd => &[],
|
||||
Self::NowUnix | Self::NowMs | Self::Args | Self::Cwd | Self::Meta | Self::ModuleSource => {
|
||||
&[]
|
||||
}
|
||||
Self::FormatTime => &["unix", "pattern"],
|
||||
Self::ParseTime => &["text", "pattern"],
|
||||
Self::SleepMs => &["ms"],
|
||||
Self::EnvGet => &["name"],
|
||||
Self::MetaGet => &["key"],
|
||||
Self::EnvSet => &["name", "value"],
|
||||
Self::Exit => &["code"],
|
||||
Self::Exec => &["cmd"],
|
||||
@@ -980,6 +995,9 @@ pub fn call_host(
|
||||
HostFn::DotenvLoad => sys::dotenv_load(ctx, bound.get("path")),
|
||||
HostFn::Args => sys::args(ctx),
|
||||
HostFn::Cwd => sys::cwd(ctx),
|
||||
HostFn::Meta => sys::meta(ctx),
|
||||
HostFn::MetaGet => sys::meta_get(ctx, require(bound, "key")?),
|
||||
HostFn::ModuleSource => crate::host::agent_rt::module_source(ctx),
|
||||
HostFn::Exit => sys::exit(ctx, require(bound, "code")?),
|
||||
HostFn::Exec => sys::exec(
|
||||
ctx,
|
||||
|
||||
@@ -146,6 +146,7 @@ pub(crate) fn value_to_json(v: &Value) -> Result<serde_json::Value, String> {
|
||||
}
|
||||
},
|
||||
Value::Code(c) => serde_json::Value::String(c.source.clone()),
|
||||
Value::Secret(_) => serde_json::Value::String("<secret>".into()),
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -103,6 +103,8 @@ pub struct HostContext {
|
||||
pub current_line: u32,
|
||||
/// Entry-file line of each active user call (for writeback anchoring).
|
||||
pub call_site_lines: Vec<u32>,
|
||||
/// Bound Artifact Metadata from the entry `.mq.md` (Phase 1 Binding).
|
||||
pub entry_metadata: Vec<(String, crate::value::Value)>,
|
||||
/// Concurrent subtasks (`lib/subtask`): file / function / foreign.
|
||||
#[cfg_attr(not(feature = "exec-host"), allow(dead_code))]
|
||||
pub(crate) subtasks: HashMap<u64, subtask::Handle>,
|
||||
@@ -129,6 +131,7 @@ impl Default for HostContext {
|
||||
call_stack: Vec::new(),
|
||||
current_line: 1,
|
||||
call_site_lines: Vec::new(),
|
||||
entry_metadata: Vec::new(),
|
||||
subtasks: HashMap::new(),
|
||||
subtask_seq: 0,
|
||||
log_min_level: 20, // Info
|
||||
|
||||
+41
-10
@@ -1,6 +1,6 @@
|
||||
//! HTTP host primitives (HTTP and HTTPS via ureq).
|
||||
|
||||
use std::io::Write;
|
||||
use std::io::{Read, Write};
|
||||
#[cfg(feature = "net-host")]
|
||||
use std::io::{BufRead, BufReader};
|
||||
#[cfg(feature = "net-host")]
|
||||
@@ -32,6 +32,26 @@ fn headers_from_value(headers: Option<&Value>) -> Result<Vec<(String, String)>,
|
||||
}
|
||||
}
|
||||
|
||||
/// Maximum response body accepted by the text HTTP helpers.
|
||||
///
|
||||
/// Read the limit from the stream, rather than after converting it to a `String`:
|
||||
/// an untrusted peer must never be able to allocate an arbitrary response in memory.
|
||||
#[cfg(feature = "net-host")]
|
||||
const MAX_HTTP_BODY_BYTES: usize = 8 * 1024 * 1024;
|
||||
|
||||
#[cfg(feature = "net-host")]
|
||||
fn read_response_text<R: Read>(reader: R) -> Result<String, String> {
|
||||
let mut bytes = Vec::new();
|
||||
reader
|
||||
.take(MAX_HTTP_BODY_BYTES as u64 + 1)
|
||||
.read_to_end(&mut bytes)
|
||||
.map_err(|e| format!("http read body: {e}"))?;
|
||||
if bytes.len() > MAX_HTTP_BODY_BYTES {
|
||||
return Err("http response exceeds 8MiB limit".into());
|
||||
}
|
||||
Ok(String::from_utf8_lossy(&bytes).into_owned())
|
||||
}
|
||||
|
||||
#[cfg(feature = "net-host")]
|
||||
fn http_exchange(
|
||||
method: &str,
|
||||
@@ -74,16 +94,11 @@ fn http_exchange(
|
||||
match resp {
|
||||
Ok(r) => {
|
||||
let status = r.status();
|
||||
let text = r
|
||||
.into_string()
|
||||
.map_err(|e| format!("http read body: {e}"))?;
|
||||
if text.len() > 8 * 1024 * 1024 {
|
||||
return Err("http response exceeds 8MiB limit".into());
|
||||
}
|
||||
let text = read_response_text(r.into_reader())?;
|
||||
Ok((status, text))
|
||||
}
|
||||
Err(ureq::Error::Status(code, r)) => {
|
||||
let text = r.into_string().unwrap_or_default();
|
||||
let text = read_response_text(r.into_reader())?;
|
||||
Ok((code, text))
|
||||
}
|
||||
Err(e) => Err(format!("http {method} {url}: {e}")),
|
||||
@@ -177,7 +192,9 @@ pub fn url_encode(text: &Value) -> Result<Value, String> {
|
||||
b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'_' | b'.' | b'~' => {
|
||||
(b as char).to_string()
|
||||
}
|
||||
b' ' => "+".into(),
|
||||
// `url_encode` documents a URL *segment*. `+` is form/query
|
||||
// encoding, where it is ambiguous with a literal plus in a path.
|
||||
b' ' => "%20".into(),
|
||||
_ => format!("%{b:02X}"),
|
||||
})
|
||||
.collect(),
|
||||
@@ -809,7 +826,7 @@ pub fn http_post_sse(
|
||||
]))
|
||||
}
|
||||
Err(ureq::Error::Status(code, r)) => {
|
||||
let text = r.into_string().unwrap_or_default();
|
||||
let text = read_response_text(r.into_reader())?;
|
||||
Ok(Value::Map(vec![
|
||||
("status".into(), Value::Int(code as i64)),
|
||||
(
|
||||
@@ -842,6 +859,20 @@ mod tests {
|
||||
.any(|(k, v)| k == "Authorization" && v == "Bearer x"));
|
||||
}
|
||||
|
||||
#[cfg(feature = "net-host")]
|
||||
#[test]
|
||||
fn response_limit_stops_before_unbounded_allocation() {
|
||||
let body = vec![b'x'; MAX_HTTP_BODY_BYTES + 1];
|
||||
let err = read_response_text(std::io::Cursor::new(body)).unwrap_err();
|
||||
assert_eq!(err, "http response exceeds 8MiB limit");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_segment_encoding_keeps_space_and_plus_distinct() {
|
||||
let encoded = url_encode(&Value::Text("a b+c".into())).unwrap();
|
||||
assert!(matches!(encoded, Value::Text(text) if text == "a%20b%2Bc"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn openai_sse_fixture_to_events() {
|
||||
let fixture = "\
|
||||
|
||||
@@ -408,6 +408,7 @@ fn spawn_fn(
|
||||
host.argv = argv;
|
||||
host.foreign_cmds = foreign_cmds;
|
||||
host.set_entry_source(Some(&module_path), &source);
|
||||
host.entry_metadata = module.metadata.clone();
|
||||
let mut interp = Interpreter::with_capture(Some(&module_path), false).with_host(host);
|
||||
interp
|
||||
.invoke_function(&module, &name, &call_args)
|
||||
|
||||
@@ -93,6 +93,22 @@ pub fn cwd(ctx: &HostContext) -> Result<Value, String> {
|
||||
Ok(Value::Text(ctx.cwd.display().to_string()))
|
||||
}
|
||||
|
||||
/// Bound entry Artifact Metadata (map).
|
||||
pub fn meta(ctx: &HostContext) -> Result<Value, String> {
|
||||
Ok(Value::Map(ctx.entry_metadata.clone()))
|
||||
}
|
||||
|
||||
/// Lookup one metadata key; missing → None.
|
||||
pub fn meta_get(ctx: &HostContext, key: &Value) -> Result<Value, String> {
|
||||
let k = as_text(key, "key")?;
|
||||
Ok(ctx
|
||||
.entry_metadata
|
||||
.iter()
|
||||
.find(|(name, _)| name == k)
|
||||
.map(|(_, v)| v.clone())
|
||||
.unwrap_or(Value::None))
|
||||
}
|
||||
|
||||
pub fn exit(ctx: &HostContext, code: &Value) -> Result<Value, String> {
|
||||
let c = as_i64(code, "code")?;
|
||||
if ctx.soft_side_effects {
|
||||
|
||||
+106
-4
@@ -46,6 +46,16 @@ impl Env {
|
||||
}
|
||||
}
|
||||
|
||||
/// Phase 2: lift resolved Artifact Metadata into the entry environment.
|
||||
/// Existing keys are left unchanged (`or_insert` semantics).
|
||||
fn with_metadata(module: &Module) -> Self {
|
||||
let mut env = Self::new();
|
||||
for (k, v) in &module.metadata {
|
||||
env.vars.entry(k.clone()).or_insert_with(|| v.clone());
|
||||
}
|
||||
env
|
||||
}
|
||||
|
||||
fn get(&self, name: &str) -> Option<&Value> {
|
||||
self.vars.get(name)
|
||||
}
|
||||
@@ -162,7 +172,7 @@ impl Interpreter {
|
||||
) -> Result<Value> {
|
||||
let fun = find_function_anywhere(module, name)
|
||||
.ok_or_else(|| self.err(format!("unknown function `{name}`")))?;
|
||||
let mut env = Env::new();
|
||||
let mut env = Env::with_metadata(module);
|
||||
for (k, v) in &self.entry_bindings {
|
||||
env.set(k.clone(), v.clone());
|
||||
}
|
||||
@@ -178,7 +188,7 @@ impl Interpreter {
|
||||
self.site_module = Some(module as *const Module);
|
||||
let result = (|| {
|
||||
if let Some(main) = find_top(module, "main") {
|
||||
return self.run_function(module, main, Env::new(), &[], true);
|
||||
return self.run_function(module, main, Env::with_metadata(module), &[], true);
|
||||
}
|
||||
// Document-as-entry: sole top-level `#` object with no params (e.g. `# Hello World`).
|
||||
let entries: Vec<&Function> = module
|
||||
@@ -187,7 +197,13 @@ impl Interpreter {
|
||||
.filter(|f| f.is_object() && f.params.is_empty())
|
||||
.collect();
|
||||
if entries.len() == 1 {
|
||||
return self.run_function(module, entries[0], Env::new(), &[], true);
|
||||
return self.run_function(
|
||||
module,
|
||||
entries[0],
|
||||
Env::with_metadata(module),
|
||||
&[],
|
||||
true,
|
||||
);
|
||||
}
|
||||
bail!("no `# main` object to run");
|
||||
})();
|
||||
@@ -324,6 +340,14 @@ impl Interpreter {
|
||||
Stmt::Assign { name, value, span, .. } => {
|
||||
self.current_span = *span;
|
||||
let v = self.eval_expr(module, fun, env, value)?;
|
||||
// 契约边界 3(T2.2):表绑定 vs 字段契约。
|
||||
if let Some(c) = fun.var_contracts.get(name) {
|
||||
if let Some(d) =
|
||||
crate::contract::fields_diag(c, name, &v, self.path.as_deref(), *span)
|
||||
{
|
||||
return Err(d.into());
|
||||
}
|
||||
}
|
||||
if self.trace {
|
||||
let display = v.as_display();
|
||||
emit_trace(
|
||||
@@ -343,6 +367,14 @@ impl Interpreter {
|
||||
Stmt::Return { value, span } => {
|
||||
self.current_span = *span;
|
||||
let v = self.eval_expr(module, fun, env, value)?;
|
||||
// 契约边界 2(T2.2):返回值 vs 返回契约。
|
||||
if let Some(c) = &fun.contract {
|
||||
if let Some(d) =
|
||||
crate::contract::return_diag(c, &fun.name, &v, self.path.as_deref(), *span)
|
||||
{
|
||||
return Err(d.into());
|
||||
}
|
||||
}
|
||||
if self.trace {
|
||||
let display = v.as_display();
|
||||
emit_trace(
|
||||
@@ -590,7 +622,38 @@ impl Interpreter {
|
||||
Expr::Index { base, label } => {
|
||||
let v = self.eval_expr(module, fun, env, base)?;
|
||||
let key = self.resolve_index_key(module, fun, env, label)?;
|
||||
builtin_footnote_get(&v, &key).map_err(|m| self.err(m))
|
||||
// 契约边界 4(T2.2):字段访问 vs 字段契约——键存在性**先于**取值
|
||||
// (契约诊断优先于运行时 `missing map key`)。
|
||||
if let Expr::Var(name) = base.as_ref() {
|
||||
if let Some(c) = fun.var_contracts.get(name) {
|
||||
if let Some(d) = crate::contract::index_diag(
|
||||
c,
|
||||
name,
|
||||
&key,
|
||||
&Value::None,
|
||||
self.path.as_deref(),
|
||||
self.current_span,
|
||||
) {
|
||||
return Err(d.into());
|
||||
}
|
||||
}
|
||||
}
|
||||
let out = builtin_footnote_get(&v, &key).map_err(|m| self.err(m))?;
|
||||
if let Expr::Var(name) = base.as_ref() {
|
||||
if let Some(c) = fun.var_contracts.get(name) {
|
||||
if let Some(d) = crate::contract::index_diag(
|
||||
c,
|
||||
name,
|
||||
&key,
|
||||
&out,
|
||||
self.path.as_deref(),
|
||||
self.current_span,
|
||||
) {
|
||||
return Err(d.into());
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
Expr::Formula(e) => Ok(Value::Formula(e.clone())),
|
||||
Expr::Code(c) => Ok(Value::Code(c.clone())),
|
||||
@@ -820,6 +883,18 @@ impl Interpreter {
|
||||
|
||||
let bound = bind_function_args(self, module, fun, env, &target.params, &ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
// 契约边界 1(T2.2):调用实参 vs 形参契约。
|
||||
if let Some(c) = &target.contract {
|
||||
if let Some(d) = crate::contract::call_arg_diag(
|
||||
c,
|
||||
&target.name,
|
||||
&bound,
|
||||
self.path.as_deref(),
|
||||
self.current_span,
|
||||
) {
|
||||
return Err(d.into());
|
||||
}
|
||||
}
|
||||
let mut call_env = Env::new();
|
||||
for (k, v) in bound {
|
||||
call_env.set(k, v);
|
||||
@@ -887,6 +962,18 @@ impl Interpreter {
|
||||
)
|
||||
.map_err(|m| self.err(m))?
|
||||
};
|
||||
// 契约边界 1(T2.2):库路径调用的实参 vs 形参契约。
|
||||
if let Some(c) = &node.contract {
|
||||
if let Some(d) = crate::contract::call_arg_diag(
|
||||
c,
|
||||
&display,
|
||||
&bound,
|
||||
self.path.as_deref(),
|
||||
self.current_span,
|
||||
) {
|
||||
return Err(d.into());
|
||||
}
|
||||
}
|
||||
let mut call_env = Env::new();
|
||||
for (k, v) in bound {
|
||||
call_env.set(k, v);
|
||||
@@ -933,6 +1020,18 @@ impl Interpreter {
|
||||
})?;
|
||||
let bound = bind_function_args(self, module, fun, env, &target.params, ev_args, false)
|
||||
.map_err(|m| self.err(m))?;
|
||||
// 契约边界 1(T2.2):方法调用的实参 vs 形参契约。
|
||||
if let Some(c) = &target.contract {
|
||||
if let Some(d) = crate::contract::call_arg_diag(
|
||||
c,
|
||||
&target.name,
|
||||
&bound,
|
||||
self.path.as_deref(),
|
||||
self.current_span,
|
||||
) {
|
||||
return Err(d.into());
|
||||
}
|
||||
}
|
||||
let mut call_env = Env::new();
|
||||
for (k, v) in bound {
|
||||
call_env.set(k, v);
|
||||
@@ -1202,6 +1301,9 @@ fn eval_binary(op: BinaryOp, l: &Value, r: &Value) -> Result<Value> {
|
||||
(Value::Int(a), Value::Num(b)) => Ok(Value::Num(*a as f64 + b)),
|
||||
(Value::Num(a), Value::Int(b)) => Ok(Value::Num(a + *b as f64)),
|
||||
(Value::Text(a), Value::Text(b)) => Ok(Value::Text(format!("{a}{b}"))),
|
||||
(Value::Text(a), Value::Secret(b)) => Ok(Value::Text(format!("{a}{b}"))),
|
||||
(Value::Secret(a), Value::Text(b)) => Ok(Value::Text(format!("{a}{b}"))),
|
||||
(Value::Secret(a), Value::Secret(b)) => Ok(Value::Text(format!("{a}{b}"))),
|
||||
(Value::Text(a), b) => Ok(Value::Text(format!("{a}{}", b.as_display()))),
|
||||
(a, Value::Text(b)) => Ok(Value::Text(format!("{}{b}", a.as_display()))),
|
||||
_ => bail!("`+` needs ints, nums, or text"),
|
||||
|
||||
+21
-3
@@ -5,12 +5,15 @@
|
||||
|
||||
pub mod aliases;
|
||||
pub mod ast;
|
||||
pub mod binding;
|
||||
pub mod browser_session;
|
||||
pub mod builtin;
|
||||
pub mod bytecode;
|
||||
pub mod capture;
|
||||
#[cfg(feature = "cli")]
|
||||
pub mod catalog;
|
||||
pub mod check;
|
||||
pub mod contract;
|
||||
pub mod debug;
|
||||
pub mod diagnostics;
|
||||
pub mod embedded_lib;
|
||||
@@ -27,7 +30,9 @@ pub mod interp;
|
||||
pub mod lex;
|
||||
pub mod linalg_ops;
|
||||
pub mod load;
|
||||
pub mod mlsp;
|
||||
pub mod parse;
|
||||
pub mod repair;
|
||||
pub mod value;
|
||||
#[cfg(feature = "net-host")]
|
||||
pub mod version_check;
|
||||
@@ -47,7 +52,8 @@ use crate::bytecode::{compile_module, Vm};
|
||||
use crate::capture::RunCapture;
|
||||
use crate::interp::Interpreter;
|
||||
use crate::lex::classify_source;
|
||||
use crate::load::{load_module, load_module_from_source};
|
||||
use crate::load::{load_module_from_source, load_module_with_ctx};
|
||||
use crate::binding::{cwd_for_path, BindingContext};
|
||||
|
||||
pub use browser_session::{value_as_json, BrowserSession};
|
||||
|
||||
@@ -91,6 +97,8 @@ pub struct RunOptions {
|
||||
pub sleep_limit_ms: Option<u64>,
|
||||
/// When set, write `# main` return value as JSON to this path (file subtasks).
|
||||
pub emit_result: Option<std::path::PathBuf>,
|
||||
/// CLI `--bind KEY=VALUE` pairs for Artifact Metadata Binding (`arg` + overlay).
|
||||
pub binds: Vec<(String, String)>,
|
||||
}
|
||||
|
||||
impl Default for RunOptions {
|
||||
@@ -111,6 +119,7 @@ impl Default for RunOptions {
|
||||
fs_root: None,
|
||||
sleep_limit_ms: None,
|
||||
emit_result: None,
|
||||
binds: Vec::new(),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -169,7 +178,8 @@ pub fn run_file(path: &Path, opts: &RunOptions) -> Result<i32> {
|
||||
println!("=== marqdo: end tokens ===");
|
||||
}
|
||||
|
||||
let module = load_module(path)?;
|
||||
let bind_ctx = BindingContext::from_binds(&opts.binds, cwd_for_path(Some(path)));
|
||||
let module = load_module_with_ctx(path, &bind_ctx)?;
|
||||
|
||||
if opts.dump_ast {
|
||||
print!("{}", format_ast_dump(&path_label, &module));
|
||||
@@ -185,6 +195,7 @@ pub fn run_file(path: &Path, opts: &RunOptions) -> Result<i32> {
|
||||
.collect::<Vec<_>>()
|
||||
);
|
||||
println!("imports: {:?}", module.imports);
|
||||
println!("metadata: {:?}", module.metadata.iter().map(|(k, v)| (k, v.as_display())).collect::<Vec<_>>());
|
||||
println!("=== marqdo: end sema ===");
|
||||
}
|
||||
|
||||
@@ -192,6 +203,7 @@ pub fn run_file(path: &Path, opts: &RunOptions) -> Result<i32> {
|
||||
Backend::Tree => {
|
||||
let mut host = HostContext::for_run(Some(path), opts.host_caps(), opts.argv.clone());
|
||||
host.set_entry_source(Some(path), &source);
|
||||
host.entry_metadata = module.metadata.clone();
|
||||
let mut interp = Interpreter::new(Some(path), opts.trace_eval)
|
||||
.with_stdin(stdin_lines.clone())
|
||||
.with_host(host);
|
||||
@@ -213,6 +225,7 @@ pub fn run_file(path: &Path, opts: &RunOptions) -> Result<i32> {
|
||||
}
|
||||
let mut host = HostContext::for_run(Some(path), opts.host_caps(), opts.argv.clone());
|
||||
host.set_entry_source(Some(path), &source);
|
||||
host.entry_metadata = module.metadata.clone();
|
||||
let mut vm = Vm::new(Some(path))
|
||||
.with_stdin(stdin_lines)
|
||||
.with_trace(opts.trace_eval)
|
||||
@@ -257,12 +270,14 @@ pub fn run_file_capture(path: &Path, opts: &RunOptions) -> Result<RunCapture> {
|
||||
let source = std::fs::read_to_string(path)
|
||||
.map_err(|e| anyhow::anyhow!("failed to read {}: {e}", path.display()))?;
|
||||
let stdin_lines = crate::input_feed::effective_stdin(&source, &opts.stdin_lines);
|
||||
let module = load_module(path)?;
|
||||
let bind_ctx = BindingContext::from_binds(&opts.binds, cwd_for_path(Some(path)));
|
||||
let module = load_module_with_ctx(path, &bind_ctx)?;
|
||||
match opts.backend {
|
||||
Backend::Tree => {
|
||||
let mut host = HostContext::for_capture(Some(path), opts.host_caps());
|
||||
host.set_entry_source(Some(path), &source);
|
||||
host.argv = opts.argv.clone();
|
||||
host.entry_metadata = module.metadata.clone();
|
||||
if let Some(root) = &opts.fs_root {
|
||||
host.fs_root = Some(root.clone());
|
||||
}
|
||||
@@ -286,6 +301,7 @@ pub fn run_file_capture(path: &Path, opts: &RunOptions) -> Result<RunCapture> {
|
||||
let mut host = HostContext::for_capture(Some(path), opts.host_caps());
|
||||
host.set_entry_source(Some(path), &source);
|
||||
host.argv = opts.argv.clone();
|
||||
host.entry_metadata = module.metadata.clone();
|
||||
if let Some(root) = &opts.fs_root {
|
||||
host.fs_root = Some(root.clone());
|
||||
}
|
||||
@@ -319,6 +335,7 @@ pub fn run_source(source: &str, opts: &RunOptions) -> Result<RunCapture> {
|
||||
let mut host = HostContext::for_capture(None, opts.host_caps());
|
||||
host.set_entry_source(None, source);
|
||||
host.argv = opts.argv.clone();
|
||||
host.entry_metadata = module.metadata.clone();
|
||||
if let Some(root) = &opts.fs_root {
|
||||
host.fs_root = Some(root.clone());
|
||||
}
|
||||
@@ -342,6 +359,7 @@ pub fn run_source(source: &str, opts: &RunOptions) -> Result<RunCapture> {
|
||||
let mut host = HostContext::for_capture(None, opts.host_caps());
|
||||
host.set_entry_source(None, source);
|
||||
host.argv = opts.argv.clone();
|
||||
host.entry_metadata = module.metadata.clone();
|
||||
if let Some(root) = &opts.fs_root {
|
||||
host.fs_root = Some(root.clone());
|
||||
}
|
||||
|
||||
+51
-12
@@ -7,17 +7,27 @@ use std::path::{Path, PathBuf};
|
||||
use anyhow::{bail, Context, Result};
|
||||
|
||||
use crate::ast::Module;
|
||||
use crate::diagnostics::Diagnostic;
|
||||
use crate::binding::{cwd_for_path, resolve_metadata, BindingContext};
|
||||
use crate::diagnostics::{Diagnostic, Span};
|
||||
use crate::embedded_lib;
|
||||
use crate::parse::parse_source;
|
||||
|
||||
/// Load `path` and recursively bind imported modules (no flat merge).
|
||||
pub fn load_module(path: &Path) -> Result<Module> {
|
||||
let mut visited = HashSet::new();
|
||||
load_module_inner(path, &mut visited)
|
||||
load_module_with_ctx(path, &BindingContext::from_binds(&[], cwd_for_path(Some(path))))
|
||||
}
|
||||
|
||||
fn load_module_inner(path: &Path, visited: &mut HashSet<PathBuf>) -> Result<Module> {
|
||||
/// Load with BindingContext (CLI `--bind`, cwd for `sys.cwd`).
|
||||
pub fn load_module_with_ctx(path: &Path, ctx: &BindingContext) -> Result<Module> {
|
||||
let mut visited = HashSet::new();
|
||||
load_module_inner(path, &mut visited, ctx)
|
||||
}
|
||||
|
||||
fn load_module_inner(
|
||||
path: &Path,
|
||||
visited: &mut HashSet<PathBuf>,
|
||||
ctx: &BindingContext,
|
||||
) -> Result<Module> {
|
||||
let canon = path
|
||||
.canonicalize()
|
||||
.unwrap_or_else(|_| path.to_path_buf());
|
||||
@@ -30,12 +40,20 @@ fn load_module_inner(path: &Path, visited: &mut HashSet<PathBuf>) -> Result<Modu
|
||||
.with_context(|| format!("failed to read {}", path.display()))?;
|
||||
let mut module = parse_source(&source).map_err(|e| attach_path(path, e))?;
|
||||
|
||||
// Resolve Artifact Metadata bindings (entry and imports share the same ctx).
|
||||
let file_ctx = BindingContext {
|
||||
args: ctx.args.clone(),
|
||||
cwd: cwd_for_path(Some(path)),
|
||||
};
|
||||
module.metadata = resolve_metadata(&module.metadata_raw, &file_ctx, Span::new(1, 1))
|
||||
.map_err(|e| attach_path(path, e))?;
|
||||
|
||||
let base = path.parent().unwrap_or_else(|| Path::new("."));
|
||||
let imports = module.imports.clone();
|
||||
let mut import_modules = HashMap::new();
|
||||
for imp in imports {
|
||||
let dep_path = resolve_import(base, &imp.path)?;
|
||||
let dep = load_module_inner(&dep_path, visited)?;
|
||||
let dep = load_module_inner(&dep_path, visited, ctx)?;
|
||||
if import_modules.contains_key(&imp.bind) {
|
||||
bail!(
|
||||
"duplicate import bind `{}` while loading {}",
|
||||
@@ -249,13 +267,29 @@ fn attach_path(path: &Path, err: anyhow::Error) -> anyhow::Error {
|
||||
/// `lib/…` stdlib (no filesystem / `ext/` for this MVP path).
|
||||
pub fn load_module_from_source(source: &str) -> Result<Module> {
|
||||
let mut visited = HashSet::new();
|
||||
load_module_from_source_inner(source, "<memory>", &mut visited)
|
||||
let ctx = BindingContext::default();
|
||||
load_module_from_source_inner(source, "<memory>", &mut visited, &ctx)
|
||||
}
|
||||
|
||||
/// 给错误打上文件标签,**保留**结构化诊断(错误出口唯一;AI/MLSP 面不许拍平成字符串)。
|
||||
/// 无结构的错误退回 `{label}: {e}` 文本。
|
||||
fn label_error(label: &str, e: anyhow::Error) -> anyhow::Error {
|
||||
match crate::diagnostics::Diagnostic::find(&e).cloned() {
|
||||
Some(mut d) => {
|
||||
if d.path.is_none() {
|
||||
d.path = Some(PathBuf::from(label));
|
||||
}
|
||||
d.into()
|
||||
}
|
||||
None => anyhow::anyhow!("{label}: {e}"),
|
||||
}
|
||||
}
|
||||
|
||||
fn load_module_from_source_inner(
|
||||
source: &str,
|
||||
label: &str,
|
||||
visited: &mut HashSet<PathBuf>,
|
||||
ctx: &BindingContext,
|
||||
) -> Result<Module> {
|
||||
let key = PathBuf::from(label);
|
||||
if !visited.insert(key.clone()) {
|
||||
@@ -263,19 +297,20 @@ fn load_module_from_source_inner(
|
||||
}
|
||||
|
||||
let result = (|| {
|
||||
let mut module = parse_source(source).map_err(|e| anyhow::anyhow!("{label}: {e}"))?;
|
||||
let mut module = parse_source(source).map_err(|e| label_error(label, e))?;
|
||||
module.metadata = resolve_metadata(&module.metadata_raw, ctx, Span::new(1, 1))
|
||||
.map_err(|e| label_error(label, e))?;
|
||||
let imports = module.imports.clone();
|
||||
let mut import_modules = HashMap::new();
|
||||
for imp in imports {
|
||||
let dep = load_embedded_import(&imp.path, visited)?;
|
||||
let dep = load_embedded_import(&imp.path, visited, ctx)?;
|
||||
if import_modules.contains_key(&imp.bind) {
|
||||
bail!("duplicate import bind `{}` while loading {label}", imp.bind);
|
||||
}
|
||||
import_modules.insert(imp.bind, dep);
|
||||
}
|
||||
module.import_modules = import_modules;
|
||||
crate::inherit::validate_inheritance(&module)
|
||||
.map_err(|e| anyhow::anyhow!("{label}: {e}"))?;
|
||||
crate::inherit::validate_inheritance(&module).map_err(|e| label_error(label, e))?;
|
||||
Ok(module)
|
||||
})();
|
||||
|
||||
@@ -283,7 +318,11 @@ fn load_module_from_source_inner(
|
||||
result
|
||||
}
|
||||
|
||||
fn load_embedded_import(rel: &str, visited: &mut HashSet<PathBuf>) -> Result<Module> {
|
||||
fn load_embedded_import(
|
||||
rel: &str,
|
||||
visited: &mut HashSet<PathBuf>,
|
||||
ctx: &BindingContext,
|
||||
) -> Result<Module> {
|
||||
let rel = rel.replace('\\', "/");
|
||||
let rest = if let Some(r) = rel.strip_prefix("lib/") {
|
||||
r.to_string()
|
||||
@@ -301,7 +340,7 @@ fn load_embedded_import(rel: &str, visited: &mut HashSet<PathBuf>) -> Result<Mod
|
||||
bail!("embedded lib not found: lib/{rest}");
|
||||
};
|
||||
let label = format!("lib/{rest}");
|
||||
load_module_from_source_inner(&dep_source, &label, visited)
|
||||
load_module_from_source_inner(&dep_source, &label, visited, ctx)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
||||
+84
-4
@@ -5,6 +5,7 @@ use anyhow::Result;
|
||||
use clap::{Parser, Subcommand, ValueEnum};
|
||||
|
||||
use marqdo::catalog::{write_catalog, CatalogOptions};
|
||||
use marqdo::diagnostics::{Diagnostic, Span};
|
||||
use marqdo::ext_cli::{add_ext, list_ext, remove_ext};
|
||||
use marqdo::input_feed::load_stdin_file;
|
||||
use marqdo::view::{serve, serve_debug, write_static, DebugOptions, OutputOptions, ViewOptions};
|
||||
@@ -64,7 +65,27 @@ enum Commands {
|
||||
/// Write `# main` return value as JSON (used by file subtasks).
|
||||
#[arg(long, value_name = "FILE")]
|
||||
emit_result: Option<PathBuf>,
|
||||
|
||||
/// Artifact Metadata Binding: `KEY=VALUE` (repeatable). Feeds `arg.KEY` and overlays metadata.
|
||||
#[arg(long = "bind", value_name = "KEY=VALUE")]
|
||||
binds: Vec<String>,
|
||||
|
||||
/// Emit diagnostics as JSON on stderr (machine-readable; AI/MLSP 面).
|
||||
#[arg(long)]
|
||||
json: bool,
|
||||
},
|
||||
/// Static contract cross-check (渐进式契约): drift, unknown types, misplaced tables
|
||||
Check {
|
||||
/// Program path (.mq.md)
|
||||
#[arg(value_name = "FILE")]
|
||||
file: PathBuf,
|
||||
|
||||
/// Emit one JSON diagnostic per line on stdout (machine-readable; AI/MLSP 面)
|
||||
#[arg(long)]
|
||||
json: bool,
|
||||
},
|
||||
/// MLSP for AI: line-delimited JSON on stdio (locate / syntax / validate / repair_targets / schema)
|
||||
Mlsp,
|
||||
/// Browse `.mq.md` structure + execution (live server or static output)
|
||||
View {
|
||||
#[command(subcommand)]
|
||||
@@ -173,17 +194,31 @@ enum ViewAction {
|
||||
}
|
||||
|
||||
fn main() -> ExitCode {
|
||||
match try_main() {
|
||||
let cli = Cli::parse();
|
||||
let json_errors = matches!(
|
||||
&cli.command,
|
||||
Commands::Run { json: true, .. } | Commands::Check { json: true, .. }
|
||||
);
|
||||
match try_main(cli) {
|
||||
Ok(code) => ExitCode::from(code as u8),
|
||||
Err(err) => {
|
||||
eprintln!("error: {err:#}");
|
||||
if json_errors {
|
||||
let diag = match err.downcast::<Diagnostic>() {
|
||||
Ok(d) => d,
|
||||
Err(e) => Diagnostic::find(&e).cloned().unwrap_or_else(|| {
|
||||
Diagnostic::new(None, Span::new(0, 0), format!("{e:#}"))
|
||||
}),
|
||||
};
|
||||
eprintln!("{}", diag.to_json());
|
||||
} else {
|
||||
eprintln!("error: {err:#}");
|
||||
}
|
||||
ExitCode::from(1)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn try_main() -> Result<i32> {
|
||||
let cli = Cli::parse();
|
||||
fn try_main(cli: Cli) -> Result<i32> {
|
||||
match cli.command {
|
||||
Commands::Run {
|
||||
file,
|
||||
@@ -197,16 +232,23 @@ fn try_main() -> Result<i32> {
|
||||
dump_all,
|
||||
stdin_file,
|
||||
emit_result,
|
||||
binds,
|
||||
json: _,
|
||||
} => {
|
||||
let path = file.unwrap_or_else(|| PathBuf::from("index.mq.md"));
|
||||
let stdin_lines = match stdin_file {
|
||||
Some(p) => load_stdin_file(&p)?,
|
||||
None => Vec::new(),
|
||||
};
|
||||
let mut bind_pairs = Vec::new();
|
||||
for b in &binds {
|
||||
bind_pairs.push(marqdo::binding::parse_bind_kv(b)?);
|
||||
}
|
||||
let mut opts = if dump_all {
|
||||
RunOptions {
|
||||
stdin_lines: stdin_lines.clone(),
|
||||
emit_result: emit_result.clone(),
|
||||
binds: bind_pairs.clone(),
|
||||
..RunOptions::dump_all()
|
||||
}
|
||||
} else {
|
||||
@@ -220,6 +262,7 @@ fn try_main() -> Result<i32> {
|
||||
backend: backend.into(),
|
||||
stdin_lines,
|
||||
emit_result,
|
||||
binds: bind_pairs,
|
||||
..RunOptions::default()
|
||||
}
|
||||
};
|
||||
@@ -229,6 +272,43 @@ fn try_main() -> Result<i32> {
|
||||
marqdo::run_file(&path, &opts)?;
|
||||
Ok(0)
|
||||
}
|
||||
Commands::Check { file, json } => {
|
||||
let diags = marqdo::check::check_path(&file)?;
|
||||
let mut errors = 0i32;
|
||||
for d in &diags {
|
||||
if matches!(d.severity, marqdo::diagnostics::Severity::Error) {
|
||||
errors += 1;
|
||||
}
|
||||
if json {
|
||||
println!("{}", d.to_json());
|
||||
} else {
|
||||
println!("{}", d.format_message());
|
||||
}
|
||||
}
|
||||
if !json {
|
||||
if diags.is_empty() {
|
||||
println!("contract check OK: 无契约问题");
|
||||
} else {
|
||||
println!("contract check: {} 条诊断({} error)", diags.len(), errors);
|
||||
}
|
||||
}
|
||||
Ok(errors.min(1))
|
||||
}
|
||||
Commands::Mlsp => {
|
||||
use std::io::{BufRead, Write};
|
||||
let stdin = std::io::stdin();
|
||||
let stdout = std::io::stdout();
|
||||
let mut out = stdout.lock();
|
||||
for line in stdin.lock().lines() {
|
||||
let line = line?;
|
||||
if line.trim().is_empty() {
|
||||
continue;
|
||||
}
|
||||
writeln!(out, "{}", marqdo::mlsp::handle_line(&line))?;
|
||||
out.flush()?;
|
||||
}
|
||||
Ok(0)
|
||||
}
|
||||
Commands::View {
|
||||
action,
|
||||
path,
|
||||
|
||||
+392
@@ -0,0 +1,392 @@
|
||||
//! Marqdo Language Server for AI(MLSP)—— AI 的可查询工具面(T3.1/T3.2,T2.6 并入 `schema`)。
|
||||
//!
|
||||
//! 目标([three-problems.md](../../doc/roadmap/three-problems.md) §3):**AI 不必背 Marqdo
|
||||
//! 语法细节**——把「记语法」换成「查工具」。语法唯一事实来源是 `parse::CORE_CONSTRUCTS`
|
||||
//! (与 `doc/design/core-surface.md` 由 CI 守卫同步);本模块只加**检索别名**,不定义语法。
|
||||
//!
|
||||
//! 行分隔 JSON 协议:每行一个请求 `{"id":…,"method":…,"params":{…}}`,
|
||||
//! 每行一个响应 `{"id":…,"ok":true,"result":{…}}` / `{"id":…,"ok":false,"error":{…}}`。
|
||||
//!
|
||||
//! | method | 作用 |
|
||||
//! |--------|------|
|
||||
//! | `locate` | 语法/符号定位:构造卡片,或定义位置 + 契约 |
|
||||
//! | `syntax` | 语法问答:关键字/别名 → 构造卡片 |
|
||||
//! | `validate` | 程序校验:结构化诊断数组(parse + 契约 check) |
|
||||
//! | `repair_targets` | 修复靶点:错误 → **有界**修复目标(越界必拒,护栏) |
|
||||
//! | `repair_apply` | 应用行编辑(全有或全无;越界 ⇒ `mlsp.repair_out_of_scope` abstain) |
|
||||
//! | `schema` | 契约查询(T2.6):单元名 → 形参/返回/字段契约 |
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use serde_json::{json, Value};
|
||||
|
||||
use crate::ast::{Function, Module};
|
||||
use crate::check::check_module;
|
||||
use crate::contract::{Contract, Ty};
|
||||
use crate::diagnostics::Diagnostic;
|
||||
use crate::load;
|
||||
use crate::parse::CORE_CONSTRUCTS;
|
||||
|
||||
/// 行分隔 JSON:一行请求 → 一行响应(永不 panic;错误也是结构化 JSON)。
|
||||
pub fn handle_line(line: &str) -> String {
|
||||
let trimmed = line.trim();
|
||||
let id = match serde_json::from_str::<Value>(trimmed) {
|
||||
Ok(req) => req.get("id").cloned().unwrap_or(Value::Null),
|
||||
Err(e) => {
|
||||
return json!({
|
||||
"id": null,
|
||||
"ok": false,
|
||||
"error": {"code": "mlsp.bad_request", "message": format!("请求不是合法 JSON: {e}")}
|
||||
})
|
||||
.to_string();
|
||||
}
|
||||
};
|
||||
let req: Value = serde_json::from_str(trimmed).expect("already parsed");
|
||||
match dispatch(&req) {
|
||||
Ok(result) => json!({"id": id, "ok": true, "result": result}).to_string(),
|
||||
Err(error) => json!({"id": id, "ok": false, "error": error}).to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
fn dispatch(req: &Value) -> Result<Value, Value> {
|
||||
let params = req.get("params").cloned().unwrap_or(json!({}));
|
||||
match req.get("method").and_then(|m| m.as_str()) {
|
||||
Some("locate") => locate(¶ms),
|
||||
Some("syntax") => syntax(¶ms),
|
||||
Some("validate") => validate(¶ms),
|
||||
Some("repair_targets") => repair_targets(¶ms),
|
||||
Some("repair_apply") => repair_apply(¶ms),
|
||||
Some("schema") => schema(¶ms),
|
||||
other => Err(err(
|
||||
"mlsp.unknown_method",
|
||||
format!("未知方法 {other:?};可用:locate / syntax / validate / repair_targets / repair_apply / schema"),
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
fn err(code: &str, message: impl Into<String>) -> Value {
|
||||
json!({"code": code, "message": message.into()})
|
||||
}
|
||||
|
||||
// ---------- 构造卡片(语法唯一事实来源 = CORE_CONSTRUCTS) ----------
|
||||
|
||||
/// 检索别名(中/英关键字)——只做匹配,不定义语法。
|
||||
const ALIASES: &[(&str, &[&str])] = &[
|
||||
("narrative", &["叙述", "文字", "说明", "注释", "narrative", "prose"]),
|
||||
("blank", &["空行", "blank"]),
|
||||
("heading-unit", &["标题", "单元", "函数", "对象", "heading", "定义"]),
|
||||
("call-line", &["调用", "call"]),
|
||||
("branch-list", &["分支", "条件", "判断", "branch", "if"]),
|
||||
("loop-list", &["循环", "遍历", "loop", "for", "while"]),
|
||||
("param", &["参数", "形参", "param"]),
|
||||
("table", &["表", "表格", "table", "集合", "数据"]),
|
||||
("fence", &["代码块", "围栏", "fence", "公式", "code"]),
|
||||
("hr", &["分隔线", "水平线", "hr"]),
|
||||
("writeback", &["回写", "输出块", "writeback"]),
|
||||
("ident", &["变量", "标识符", "ident", "名字"]),
|
||||
("bold-code", &["粗体", "赋值", "语句", "bold", "执行"]),
|
||||
("italic-return", &["返回", "return", "斜体"]),
|
||||
("empty-return", &["空返回", "none", "结束", "副作用"]),
|
||||
("link-index", &["取元", "索引", "index", "键", "行"]),
|
||||
("bracket-call", &["修饰", "括号", "bracket", "开关参数"]),
|
||||
("footnote-index", &["脚注", "footnote", "取列", "列"]),
|
||||
("soft-emphasis", &["强调", "emphasis", "软强调"]),
|
||||
];
|
||||
|
||||
/// core-surface.md 全文(与 `tests/core_surface.rs` 守卫同源)——卡片机械携带
|
||||
/// 语义/戒律/样例**原文**,AI 一次查询即可自持,不必回读文档(three-problems.md §3.2「单构造戒律」)。
|
||||
const SURFACE_MD: &str = include_str!("../doc/design/core-surface.md");
|
||||
|
||||
/// 机械提取 core-surface.md 清单行:`| \`id\` | 外形 | 语义 | 戒律 | 样例 |` →
|
||||
///(语义, 戒律, 样例, 行原文);只读不改,卡片内容与文档永不漂移。
|
||||
fn surface_row(id: &str) -> Option<(String, String, String, String)> {
|
||||
for line in SURFACE_MD.lines() {
|
||||
let t = line.trim();
|
||||
if !t.starts_with('|') {
|
||||
continue;
|
||||
}
|
||||
let cells: Vec<&str> = t.trim_matches('|').split('|').map(str::trim).collect();
|
||||
if cells.len() != 5 {
|
||||
continue;
|
||||
}
|
||||
let rid = cells[0].trim_matches('`');
|
||||
if rid == id && !rid.is_empty() && !rid.contains(' ') && !rid.starts_with('-') {
|
||||
return Some((
|
||||
cells[2].to_string(),
|
||||
cells[3].to_string(),
|
||||
cells[4].to_string(),
|
||||
t.to_string(),
|
||||
));
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
fn card(id: &str) -> Value {
|
||||
let form = CORE_CONSTRUCTS
|
||||
.iter()
|
||||
.find(|(cid, _)| *cid == id)
|
||||
.map(|(_, f)| *f)
|
||||
.unwrap_or("?");
|
||||
let mut v = json!({
|
||||
"id": id,
|
||||
"form": form,
|
||||
"manual": "doc/design/core-surface.md",
|
||||
});
|
||||
if let Some((semantics, rule, sample, quote)) = surface_row(id) {
|
||||
v["semantics"] = semantics.into();
|
||||
v["rule"] = rule.into();
|
||||
v["sample"] = sample.into();
|
||||
v["doc_quote"] = quote.into();
|
||||
}
|
||||
v
|
||||
}
|
||||
|
||||
fn match_constructs(query: &str) -> Vec<Value> {
|
||||
let q = query.trim().to_lowercase();
|
||||
if q.is_empty() {
|
||||
return ALIASES.iter().map(|(id, _)| card(id)).collect();
|
||||
}
|
||||
ALIASES
|
||||
.iter()
|
||||
.filter(|(_, keys)| {
|
||||
keys.iter()
|
||||
.any(|k| k.to_lowercase().contains(&q) || q.contains(&k.to_lowercase()))
|
||||
})
|
||||
.map(|(id, _)| card(id))
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn syntax(params: &Value) -> Result<Value, Value> {
|
||||
let query = params.get("query").and_then(|q| q.as_str()).unwrap_or("");
|
||||
let cards = match_constructs(query);
|
||||
if cards.is_empty() {
|
||||
return Err(err(
|
||||
"mlsp.no_match",
|
||||
format!("没有匹配「{query}」的语法构造;换关键字(如 返回/表/调用)或省略 query 列出全部"),
|
||||
));
|
||||
}
|
||||
Ok(json!({"cards": cards}))
|
||||
}
|
||||
|
||||
fn locate(params: &Value) -> Result<Value, Value> {
|
||||
if params.get("path").is_some() || params.get("source").is_some() {
|
||||
let (module, path) = module_from_params(params)?;
|
||||
let symbol = params
|
||||
.get("symbol")
|
||||
.and_then(|s| s.as_str())
|
||||
.ok_or_else(|| err("mlsp.bad_request", "locate 定位符号需要 params.symbol"))?;
|
||||
let fun = find_fn(&module, symbol)
|
||||
.ok_or_else(|| err("mlsp.unknown_unit", format!("没有名为 `{symbol}` 的单元")))?;
|
||||
return Ok(json!({
|
||||
"unit": fun.name,
|
||||
"level": fun.level,
|
||||
"span": span_json(path.as_deref(), fun.span),
|
||||
"params": fun.params.iter().map(|p| p.name.clone()).collect::<Vec<_>>(),
|
||||
"contract": fun.contract.as_ref().map(|c| contract_json(c, path.as_deref())),
|
||||
"manual": "doc/design/core-surface.md",
|
||||
}));
|
||||
}
|
||||
Ok(json!({"cards": match_constructs(
|
||||
params.get("query").and_then(|q| q.as_str()).unwrap_or("")
|
||||
)}))
|
||||
}
|
||||
|
||||
fn validate(params: &Value) -> Result<Value, Value> {
|
||||
let (module, path) = module_from_params(params)?;
|
||||
let mut diags = check_module(&module, path.as_deref());
|
||||
diags.sort_by_key(|d| (d.span.line, d.span.col));
|
||||
let jsons: Vec<Value> = diags.iter().map(|d| d.to_json()).collect();
|
||||
let ok = !diags
|
||||
.iter()
|
||||
.any(|d| matches!(d.severity, crate::diagnostics::Severity::Error));
|
||||
Ok(json!({"ok": ok, "diagnostics": jsons}))
|
||||
}
|
||||
|
||||
/// 修复靶点(T3.3 护栏):锚点局部修复,**越界必拒**(禁止整文件重写/无关重构)。
|
||||
fn repair_targets(params: &Value) -> Result<Value, Value> {
|
||||
let code = params.get("code").and_then(|c| c.as_str()).unwrap_or("");
|
||||
let anchor = params.get("doc_anchor").and_then(|a| a.as_str());
|
||||
let suggestion = params.get("suggestion").and_then(|s| s.as_str());
|
||||
if code.is_empty() {
|
||||
return Err(err("mlsp.bad_request", "repair_targets 需要 params.code"));
|
||||
}
|
||||
Ok(json!({
|
||||
"targets": [{
|
||||
"code": code,
|
||||
"doc_anchor": anchor,
|
||||
"span": params.get("span").cloned().unwrap_or(Value::Null),
|
||||
}],
|
||||
"repair": {
|
||||
"strategy": "anchored-local",
|
||||
"bounded": true,
|
||||
"max_attempts": 2,
|
||||
"scope": "doc_anchor 所锚定的契约表/语句行",
|
||||
"suggestion": suggestion,
|
||||
"on_violation": "abstain",
|
||||
"note": "只许改动锚定范围内的行;越界改动必须拒绝并报 abstain"
|
||||
}
|
||||
}))
|
||||
}
|
||||
|
||||
/// 应用有界修复(T3.3):行编辑全有或全无;越界 ⇒ `mlsp.repair_out_of_scope`(abstain),源零字节不改。
|
||||
fn repair_apply(params: &Value) -> Result<Value, Value> {
|
||||
let source = source_text(params)?;
|
||||
let ranges = params
|
||||
.get("ranges")
|
||||
.and_then(|r| r.as_array())
|
||||
.ok_or_else(|| err("mlsp.bad_request", "repair_apply 需要 params.ranges([[起,止],…])"))?
|
||||
.iter()
|
||||
.map(|pair| {
|
||||
let a = pair.as_array()?;
|
||||
Some((a.first()?.as_u64()? as u32, a.get(1)?.as_u64()? as u32))
|
||||
})
|
||||
.collect::<Option<Vec<(u32, u32)>>>()
|
||||
.ok_or_else(|| err("mlsp.bad_request", "ranges 必须是 [[起,止],…]"))?;
|
||||
|
||||
let mut edits = Vec::new();
|
||||
let edit_list = params
|
||||
.get("edits")
|
||||
.and_then(|e| e.as_array())
|
||||
.ok_or_else(|| err("mlsp.bad_request", "repair_apply 需要 params.edits"))?;
|
||||
for e in edit_list {
|
||||
let line = e
|
||||
.get("line")
|
||||
.and_then(|l| l.as_u64())
|
||||
.ok_or_else(|| err("mlsp.bad_request", "edit 缺 line"))? as u32;
|
||||
let text = e.get("text").and_then(|t| t.as_str()).unwrap_or("").to_string();
|
||||
edits.push(match e.get("op").and_then(|o| o.as_str()) {
|
||||
Some("replace") => crate::repair::Edit::Replace { line, text },
|
||||
Some("delete") => crate::repair::Edit::Delete { line },
|
||||
Some("insert") => crate::repair::Edit::InsertBefore { line, text },
|
||||
other => {
|
||||
return Err(err(
|
||||
"mlsp.bad_request",
|
||||
format!("未知编辑 op {other:?}(replace / delete / insert)"),
|
||||
))
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
match crate::repair::apply_edits(&source, &ranges, &edits) {
|
||||
Ok(new_source) => {
|
||||
if params.get("write").and_then(|w| w.as_bool()).unwrap_or(false) {
|
||||
let path = params
|
||||
.get("path")
|
||||
.and_then(|p| p.as_str())
|
||||
.ok_or_else(|| err("mlsp.bad_request", "write=true 需要 params.path"))?;
|
||||
std::fs::write(path, &new_source)
|
||||
.map_err(|e| err("mlsp.io_error", format!("写回 {path} 失败: {e}")))?;
|
||||
}
|
||||
Ok(json!({"source": new_source, "edits_applied": edits.len()}))
|
||||
}
|
||||
Err(oos) => Err(json!({
|
||||
"code": "mlsp.repair_out_of_scope",
|
||||
"message": format!(
|
||||
"修复越界(abstain):触及行 {:?} 不在靶点范围 {:?} 内;已拒绝全部编辑,源未改动",
|
||||
oos.lines, oos.ranges
|
||||
),
|
||||
"lines": oos.lines,
|
||||
"ranges": oos.ranges,
|
||||
"on_violation": "abstain",
|
||||
})),
|
||||
}
|
||||
}
|
||||
|
||||
/// 取待修源码(source 直给 / path 读取)。
|
||||
fn source_text(params: &Value) -> Result<String, Value> {
|
||||
if let Some(s) = params.get("source").and_then(|s| s.as_str()) {
|
||||
return Ok(s.to_string());
|
||||
}
|
||||
let path = params
|
||||
.get("path")
|
||||
.and_then(|p| p.as_str())
|
||||
.ok_or_else(|| err("mlsp.bad_request", "需要 params.source 或 params.path"))?;
|
||||
std::fs::read_to_string(path)
|
||||
.map_err(|e| err("mlsp.io_error", format!("读取 {path} 失败: {e}")))
|
||||
}
|
||||
|
||||
/// 契约查询(T2.6):单元名 → 形参/返回/字段契约。
|
||||
fn schema(params: &Value) -> Result<Value, Value> {
|
||||
let (module, path) = module_from_params(params)?;
|
||||
let unit = params
|
||||
.get("unit")
|
||||
.and_then(|u| u.as_str())
|
||||
.ok_or_else(|| err("mlsp.bad_request", "schema 需要 params.unit"))?;
|
||||
let fun = find_fn(&module, unit)
|
||||
.ok_or_else(|| err("mlsp.unknown_unit", format!("没有名为 `{unit}` 的单元")))?;
|
||||
let mut vars = serde_json::Map::new();
|
||||
for (name, c) in &fun.var_contracts {
|
||||
vars.insert(name.clone(), contract_json(c, path.as_deref()));
|
||||
}
|
||||
Ok(json!({
|
||||
"unit": fun.name,
|
||||
"contract": fun.contract.as_ref().map(|c| contract_json(c, path.as_deref())),
|
||||
"vars": Value::Object(vars),
|
||||
"params_inferred": fun.params.iter().map(|p| json!({
|
||||
"name": p.name, "inferred": p.inferred
|
||||
})).collect::<Vec<_>>(),
|
||||
}))
|
||||
}
|
||||
|
||||
// ---------- 公共小工具 ----------
|
||||
|
||||
fn module_from_params(params: &Value) -> Result<(Module, Option<PathBuf>), Value> {
|
||||
if let Some(src) = params.get("source").and_then(|s| s.as_str()) {
|
||||
let module = load::load_module_from_source(src)
|
||||
.map_err(|e| diagnostic_err("mlsp.parse_error", &e))?;
|
||||
return Ok((module, None));
|
||||
}
|
||||
let path = params
|
||||
.get("path")
|
||||
.and_then(|p| p.as_str())
|
||||
.ok_or_else(|| err("mlsp.bad_request", "需要 params.path 或 params.source"))?;
|
||||
let path = PathBuf::from(path);
|
||||
let module = load::load_module(&path).map_err(|e| diagnostic_err("mlsp.parse_error", &e))?;
|
||||
Ok((module, Some(path)))
|
||||
}
|
||||
|
||||
/// 错误链里有结构化诊断就原样透出(AI/MLSP 面不许拍平)。
|
||||
fn diagnostic_err(code: &str, e: &anyhow::Error) -> Value {
|
||||
match Diagnostic::find(e) {
|
||||
Some(d) => d.to_json(),
|
||||
None => err(code, format!("{e:#}")),
|
||||
}
|
||||
}
|
||||
|
||||
fn find_fn<'a>(module: &'a Module, name: &str) -> Option<&'a Function> {
|
||||
fn walk<'a>(funs: &'a [Function], name: &str) -> Option<&'a Function> {
|
||||
funs.iter().find(|f| f.name == name).or_else(|| {
|
||||
funs.iter()
|
||||
.find_map(|f| walk(&f.children, name))
|
||||
})
|
||||
}
|
||||
walk(&module.functions, name)
|
||||
}
|
||||
|
||||
fn span_json(path: Option<&Path>, span: crate::diagnostics::Span) -> Value {
|
||||
json!({
|
||||
"file": path.map(|p| p.display().to_string()),
|
||||
"line": span.line,
|
||||
"col": span.col,
|
||||
})
|
||||
}
|
||||
|
||||
fn ty_json(ty: &Ty) -> Value {
|
||||
json!(ty.name())
|
||||
}
|
||||
|
||||
fn contract_json(c: &Contract, path: Option<&Path>) -> Value {
|
||||
json!({
|
||||
"params": c.params.iter().map(|p| json!({
|
||||
"name": p.name, "type": ty_json(&p.ty), "desc": p.desc
|
||||
})).collect::<Vec<_>>(),
|
||||
"returns": c.returns.as_ref().map(|r| json!({
|
||||
"type": ty_json(&r.ty), "desc": r.desc
|
||||
})),
|
||||
"fields": c.fields.iter().map(|f| json!({
|
||||
"name": f.name, "type": ty_json(&f.ty), "nullable": f.nullable, "desc": f.desc
|
||||
})).collect::<Vec<_>>(),
|
||||
"doc_anchor": c.anchor(path),
|
||||
})
|
||||
}
|
||||
+314
-6
@@ -28,7 +28,7 @@ pub fn parse_source(source: &str) -> Result<Module> {
|
||||
|
||||
pub fn parse_classified(lines: &[ClassifiedLine]) -> Result<Module> {
|
||||
let mut cur = Cursor::new(lines);
|
||||
let (imports, uses) = cur.parse_frontmatter()?;
|
||||
let (imports, uses, metadata_raw) = cur.parse_frontmatter()?;
|
||||
let mut functions = Vec::new();
|
||||
while cur.skip_noise() {
|
||||
let Some(line) = cur.peek() else { break };
|
||||
@@ -65,6 +65,8 @@ pub fn parse_classified(lines: &[ClassifiedLine]) -> Result<Module> {
|
||||
uses,
|
||||
functions,
|
||||
import_modules: std::collections::HashMap::new(),
|
||||
metadata_raw,
|
||||
metadata: Vec::new(),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -134,17 +136,24 @@ impl<'a> Cursor<'a> {
|
||||
})
|
||||
}
|
||||
|
||||
fn parse_frontmatter(&mut self) -> Result<(Vec<Import>, Vec<Use>)> {
|
||||
fn parse_frontmatter(
|
||||
&mut self,
|
||||
) -> Result<(
|
||||
Vec<Import>,
|
||||
Vec<Use>,
|
||||
Vec<(String, crate::binding::MetaValue)>,
|
||||
)> {
|
||||
let mut imports = Vec::new();
|
||||
let mut uses = Vec::new();
|
||||
let mut metadata_raw = Vec::new();
|
||||
if !self.skip_noise() {
|
||||
return Ok((imports, uses));
|
||||
return Ok((imports, uses, metadata_raw));
|
||||
}
|
||||
let Some(first) = self.peek() else {
|
||||
return Ok((imports, uses));
|
||||
return Ok((imports, uses, metadata_raw));
|
||||
};
|
||||
if first.text.trim() != "---" {
|
||||
return Ok((imports, uses));
|
||||
return Ok((imports, uses, metadata_raw));
|
||||
}
|
||||
self.bump();
|
||||
while let Some(line) = self.peek() {
|
||||
@@ -169,6 +178,8 @@ impl<'a> Cursor<'a> {
|
||||
));
|
||||
}
|
||||
imports.push(imp);
|
||||
self.bump();
|
||||
continue;
|
||||
}
|
||||
Ok(Some(ImportLine::Member(u))) => {
|
||||
if imports.iter().any(|i| i.bind == u.bind)
|
||||
@@ -181,13 +192,43 @@ impl<'a> Cursor<'a> {
|
||||
));
|
||||
}
|
||||
uses.push(u);
|
||||
self.bump();
|
||||
continue;
|
||||
}
|
||||
Ok(None) => {}
|
||||
Err(e) => return Err(bail_at(None, span, e.to_string())),
|
||||
}
|
||||
// Artifact metadata key: value (flat Phase 1)
|
||||
if let Some((key, rest)) = split_frontmatter_key(t) {
|
||||
self.bump();
|
||||
let value_raw = if rest.is_empty()
|
||||
|| rest == "|"
|
||||
|| rest == "|-"
|
||||
|| rest == "|+"
|
||||
|| rest == ">"
|
||||
|| rest == ">-"
|
||||
|| rest == ">+"
|
||||
{
|
||||
collect_block_scalar(self, rest)?
|
||||
} else {
|
||||
rest.to_string()
|
||||
};
|
||||
let mv = crate::binding::parse_meta_value(&value_raw)
|
||||
.map_err(|e| bail_at(None, span, e.to_string()))?;
|
||||
if metadata_raw.iter().any(|(k, _)| k == key) {
|
||||
return Err(bail_at(
|
||||
None,
|
||||
span,
|
||||
format!("duplicate metadata key `{key}`"),
|
||||
));
|
||||
}
|
||||
metadata_raw.push((key.to_string(), mv));
|
||||
continue;
|
||||
}
|
||||
// Unknown / non-key line: skip (compat with freeform frontmatter notes)
|
||||
self.bump();
|
||||
}
|
||||
Ok((imports, uses))
|
||||
Ok((imports, uses, metadata_raw))
|
||||
}
|
||||
|
||||
fn parse_function(&mut self, min_level: u8) -> Result<Function> {
|
||||
@@ -212,6 +253,8 @@ impl<'a> Cursor<'a> {
|
||||
children: Vec::new(),
|
||||
base,
|
||||
dead_binds: Vec::new(),
|
||||
contract: None,
|
||||
var_contracts: std::collections::HashMap::new(),
|
||||
};
|
||||
// Prose decls pending promotion (name → optional default text).
|
||||
let mut prose_decls: Vec<(String, Option<String>)> = Vec::new();
|
||||
@@ -342,6 +385,8 @@ impl<'a> Cursor<'a> {
|
||||
}
|
||||
}
|
||||
|
||||
// 契约提取(T2.1)必须在升参推断**之前**:契约表的单元不污染推断。
|
||||
extract_function_contract(&mut fun, self.lines)?;
|
||||
merge_inferred_params(&mut fun, &prose_decls)?;
|
||||
Ok(fun)
|
||||
}
|
||||
@@ -1165,6 +1210,80 @@ pub enum ImportLine {
|
||||
Member(Use),
|
||||
}
|
||||
|
||||
fn split_frontmatter_key(trimmed: &str) -> Option<(&str, &str)> {
|
||||
let t = trimmed.trim();
|
||||
if t.is_empty() || t.starts_with('#') {
|
||||
return None;
|
||||
}
|
||||
// import lines are handled earlier
|
||||
let Some((key, rest)) = t.split_once(':') else {
|
||||
return None;
|
||||
};
|
||||
let key = key.trim();
|
||||
if key.is_empty() || key.contains(' ') {
|
||||
return None;
|
||||
}
|
||||
// Avoid treating `http://…` style as key (no spaces in key already; also reject if key has `/`)
|
||||
if key.contains('/') {
|
||||
return None;
|
||||
}
|
||||
Some((key, rest.trim()))
|
||||
}
|
||||
|
||||
fn collect_block_scalar(cur: &mut Cursor<'_>, indicator: &str) -> Result<String> {
|
||||
let mut lines = Vec::new();
|
||||
while let Some(line) = cur.peek() {
|
||||
let raw = line.text.trim_end_matches(['\r', '\n']);
|
||||
let bt = raw.trim();
|
||||
if bt == "---" {
|
||||
break;
|
||||
}
|
||||
// Next top-level key or import ends the block
|
||||
if !raw.starts_with(' ') && !raw.starts_with('\t') && !bt.is_empty() {
|
||||
if split_frontmatter_key(bt).is_some() || parse_import_line(bt).ok().flatten().is_some()
|
||||
{
|
||||
break;
|
||||
}
|
||||
if indicator.is_empty() {
|
||||
// empty rest on key line without block indicator: no body
|
||||
break;
|
||||
}
|
||||
}
|
||||
if indicator.is_empty() {
|
||||
break;
|
||||
}
|
||||
cur.bump();
|
||||
if raw.starts_with(' ') || raw.starts_with('\t') {
|
||||
lines.push(strip_yaml_indent(raw));
|
||||
} else if bt.is_empty() {
|
||||
lines.push(String::new());
|
||||
} else {
|
||||
// folded content without indent — take as-is once
|
||||
lines.push(bt.to_string());
|
||||
break;
|
||||
}
|
||||
}
|
||||
if indicator.starts_with('>') {
|
||||
Ok(lines
|
||||
.into_iter()
|
||||
.filter(|l| !l.is_empty())
|
||||
.collect::<Vec<_>>()
|
||||
.join(" "))
|
||||
} else {
|
||||
Ok(lines.join("\n"))
|
||||
}
|
||||
}
|
||||
|
||||
fn strip_yaml_indent(line: &str) -> String {
|
||||
if line.starts_with(" ") {
|
||||
line[2..].to_string()
|
||||
} else if line.starts_with('\t') {
|
||||
line[1..].to_string()
|
||||
} else {
|
||||
line.trim_start_matches([' ', '\t']).to_string()
|
||||
}
|
||||
}
|
||||
|
||||
fn legacy_frontmatter_import_hint(trimmed: &str) -> Option<String> {
|
||||
let t = trimmed.trim();
|
||||
if !t.starts_with('>') {
|
||||
@@ -1302,6 +1421,166 @@ fn stmt_from_italic_return(inner: &str, span: Span) -> Result<Stmt> {
|
||||
/// Design §8.2: elevate only when an executable read cannot be satisfied by a prior
|
||||
/// local assign that does not itself read the name (e.g. `score = > json.get …` is local;
|
||||
/// `n = n + 1` still needs external `n`).
|
||||
// ---------- 契约提取(T2.1 · 渐进式契约 / 标注类型) ----------
|
||||
//
|
||||
// 未绑定文档表 → 契约;**位置 + 形状**双重判定;在升参推断之前执行。
|
||||
// 契约表从 body 移除(元数据永不执行、不污染升参推断);
|
||||
// 非契约表(普通文档/数据表,含一切**绑定**表)原样保留 ⇒ 无契约文件零回归。
|
||||
|
||||
/// 从表头行起收集连续 `|` 行作为契约原文(诊断 `doc_quote`)。
|
||||
fn table_quote(lines: &[ClassifiedLine], start_line: u32) -> (u32, Vec<String>) {
|
||||
let Some(mut idx) = (0..lines.len()).find(|&i| lines[i].line_no == start_line) else {
|
||||
return (start_line, Vec::new());
|
||||
};
|
||||
let mut quote = Vec::new();
|
||||
let mut end = start_line;
|
||||
while idx < lines.len() && lines[idx].text.trim().starts_with('|') {
|
||||
quote.push(lines[idx].text.trim_end().to_string());
|
||||
end = lines[idx].line_no;
|
||||
idx += 1;
|
||||
}
|
||||
(end, quote)
|
||||
}
|
||||
|
||||
/// 是否「字段契约形状」的未绑定文档表(紧邻判定的透明跳过用)。
|
||||
fn is_fields_contract_stmt(stmt: &Stmt, lines: &[ClassifiedLine]) -> bool {
|
||||
let Stmt::Expr { value: Expr::Map(pairs), span } = stmt else {
|
||||
return false;
|
||||
};
|
||||
let (end_line, quote) = table_quote(lines, span.line);
|
||||
matches!(
|
||||
crate::contract::parse_contract_table(pairs, span.line, end_line, quote),
|
||||
Some(Ok(ref c)) if !c.fields.is_empty()
|
||||
)
|
||||
}
|
||||
|
||||
/// 紧邻绑定名:跳过连续的字段契约表后,下一条是不是 `` `名` = `` 绑定。
|
||||
fn next_binding_name(body: &[Stmt], start: usize, lines: &[ClassifiedLine]) -> Option<String> {
|
||||
let mut j = start;
|
||||
while j < body.len() && is_fields_contract_stmt(&body[j], lines) {
|
||||
j += 1;
|
||||
}
|
||||
match body.get(j) {
|
||||
Some(Stmt::Assign { name, .. }) => Some(name.clone()),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
fn empty_like(piece: &crate::contract::Contract) -> crate::contract::Contract {
|
||||
crate::contract::Contract {
|
||||
params: Vec::new(),
|
||||
returns: None,
|
||||
fields: Vec::new(),
|
||||
start_line: piece.start_line,
|
||||
end_line: piece.end_line,
|
||||
quote: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// 合并契约块;重复行/重复表 → `contract.duplicate` 诊断(防两份契约打架)。
|
||||
fn merge_piece(
|
||||
dst: &mut Option<crate::contract::Contract>,
|
||||
piece: crate::contract::Contract,
|
||||
unit: &str,
|
||||
span: Span,
|
||||
) -> Result<()> {
|
||||
match dst {
|
||||
None => *dst = Some(piece),
|
||||
Some(slot) => {
|
||||
if let Err(m) = crate::contract::merge_contract(slot, piece.clone()) {
|
||||
return Err(piece
|
||||
.diagnostic(
|
||||
"contract.duplicate",
|
||||
unit,
|
||||
m,
|
||||
"删除重复契约表,只保留一份".to_string(),
|
||||
None,
|
||||
span,
|
||||
)
|
||||
.into());
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn extract_function_contract(fun: &mut Function, lines: &[ClassifiedLine]) -> Result<()> {
|
||||
let body = std::mem::take(&mut fun.body);
|
||||
let mut fun_contract: Option<crate::contract::Contract> = None;
|
||||
let mut var_contracts: std::collections::HashMap<String, crate::contract::Contract> =
|
||||
std::collections::HashMap::new();
|
||||
let mut keep: Vec<Stmt> = Vec::with_capacity(body.len());
|
||||
// 「函数体首个可执行行之前」:未绑定表(Map/List 形状的 `Stmt::Expr`)不算可执行行。
|
||||
let mut prefix_zone = true;
|
||||
let mut i = 0usize;
|
||||
|
||||
while i < body.len() {
|
||||
let stmt = &body[i];
|
||||
let (pairs, span) = match stmt {
|
||||
Stmt::Expr {
|
||||
value: Expr::Map(pairs),
|
||||
span,
|
||||
} => (pairs, *span),
|
||||
other => {
|
||||
if !matches!(other, Stmt::Expr { value: Expr::List(_), .. }) {
|
||||
prefix_zone = false;
|
||||
}
|
||||
keep.push(other.clone());
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
};
|
||||
let (end_line, quote) = table_quote(lines, span.line);
|
||||
let Some(parsed) = crate::contract::parse_contract_table(pairs, span.line, end_line, quote)
|
||||
else {
|
||||
// 非契约形状:普通文档/数据表 → 原样保留(P1 零回归的关键)。
|
||||
keep.push(stmt.clone());
|
||||
i += 1;
|
||||
continue;
|
||||
};
|
||||
let piece = parsed?; // 契约形状但畸形 → 必产诊断(contract.malformed)
|
||||
let is_fn_contract = !piece.params.is_empty() || piece.returns.is_some();
|
||||
|
||||
if is_fn_contract {
|
||||
// 形参/返回表只在「首个可执行行之前」生效,否则视为普通表。
|
||||
if !prefix_zone {
|
||||
keep.push(stmt.clone());
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
merge_piece(&mut fun_contract, piece, &fun.name, span)?;
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
// 字段表:紧邻 `` `名` = `` 绑定之前 → 集合契约;对象体首成员前 → 对象字段契约。
|
||||
if let Some(name) = next_binding_name(&body, i + 1, lines) {
|
||||
let mut slot = Some(
|
||||
var_contracts
|
||||
.remove(&name)
|
||||
.unwrap_or_else(|| empty_like(&piece)),
|
||||
);
|
||||
merge_piece(&mut slot, piece, &name, span)?;
|
||||
var_contracts.insert(name, slot.unwrap());
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
if prefix_zone && fun.is_object() {
|
||||
merge_piece(&mut fun_contract, piece, &fun.name, span)?;
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
// 不在允许位置 → 不是契约,原样保留。
|
||||
keep.push(stmt.clone());
|
||||
i += 1;
|
||||
}
|
||||
|
||||
fun.body = keep;
|
||||
fun.contract = fun_contract;
|
||||
fun.var_contracts = var_contracts;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn merge_inferred_params(
|
||||
fun: &mut Function,
|
||||
prose_decls: &[(String, Option<String>)],
|
||||
@@ -1853,3 +2132,32 @@ Candidates may include lexical `score`.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 核心表面构造注册表(T1.3)——`(id, Markdown 外形)`。
|
||||
///
|
||||
/// **紧邻分类代码维护**;与 `doc/design/core-surface.md` 的 id 列强一致,
|
||||
/// 由 `tests/core_surface.rs` 守卫:改动任一侧不同步 → CI 红。
|
||||
/// 新增条目必须走 ADR(`doc/design/layers.md` 准入戒律)。
|
||||
pub const CORE_CONSTRUCTS: &[(&str, &str)] = &[
|
||||
// 行构造
|
||||
("narrative", "无标记文字"),
|
||||
("blank", "空行"),
|
||||
("heading-unit", "# / ##"),
|
||||
("call-line", ">"),
|
||||
("branch-list", "1. 递增"),
|
||||
("loop-list", "-"),
|
||||
("param", "+ `名`"),
|
||||
("table", "GFM 表"),
|
||||
("fence", "```lang"),
|
||||
("hr", "--- / ***"),
|
||||
("writeback", "<!-- marqdo-out -->"),
|
||||
// 内联标记
|
||||
("ident", "`名`"),
|
||||
("bold-code", "**…**"),
|
||||
("italic-return", "*…*"),
|
||||
("empty-return", "*None* / ****"),
|
||||
("link-index", "[键](集合)"),
|
||||
("bracket-call", "修饰… [函数] …"),
|
||||
("footnote-index", "集合[^键]"),
|
||||
("soft-emphasis", "*仅* / **说明**"),
|
||||
];
|
||||
+129
@@ -0,0 +1,129 @@
|
||||
//! 有界修复(T3.3 护栏的**机械执行面**):锚点局部修复,越界必拒(abstain),全有或全无。
|
||||
//!
|
||||
//! 修复循环三步:诊断(run/check)→ `mlsp repair_targets`(靶点 + 范围)→ `mlsp repair_apply`(改行)。
|
||||
//! 护栏不是提示词愿望,而是这里代码强制的:
|
||||
//!
|
||||
//! - 只允许三种行编辑:`Replace` / `Delete` / `InsertBefore`;
|
||||
//! - 被触碰的每一行都必须落在靶点范围内(`InsertBefore` 额外允许紧跟范围末尾追加),
|
||||
//! 否则**整体拒绝**并报越界行清单——一个字节都不改(全有或全无);
|
||||
//! - 应用结果交给调用方复验(`validate` / `run`)。
|
||||
//!
|
||||
//! 目标([three-problems.md](../../doc/roadmap/three-problems.md) §5.3):易锚定错误
|
||||
//! ≥80% 修复率的同时,**越权范围的改动必拒**、失败后 abstain 而非乱改。
|
||||
|
||||
use std::collections::{HashMap, HashSet};
|
||||
|
||||
/// 一行编辑(1-based 行号)。
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum Edit {
|
||||
/// 把 `line` 整行换成 `text`。
|
||||
Replace { line: u32, text: String },
|
||||
/// 删除 `line`。
|
||||
Delete { line: u32 },
|
||||
/// 在 `line` **之前**插入 `text`(允许 `line == 范围末行 + 1`,即紧跟块尾追加)。
|
||||
InsertBefore { line: u32, text: String },
|
||||
}
|
||||
|
||||
impl Edit {
|
||||
/// 被触碰的行(范围检查用)。
|
||||
pub fn touched(&self) -> u32 {
|
||||
match self {
|
||||
Edit::Replace { line, .. } | Edit::Delete { line } | Edit::InsertBefore { line, .. } => {
|
||||
*line
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 越界拒绝(abstain):报告越界行与允许范围,源保持不变。
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct OutOfScope {
|
||||
pub lines: Vec<u32>,
|
||||
pub ranges: Vec<(u32, u32)>,
|
||||
}
|
||||
|
||||
fn in_ranges(ranges: &[(u32, u32)], line: u32) -> bool {
|
||||
ranges.iter().any(|(s, e)| line >= *s && line <= *e)
|
||||
}
|
||||
|
||||
fn at_tail(ranges: &[(u32, u32)], line: u32) -> bool {
|
||||
ranges.iter().any(|(_, e)| line == e + 1)
|
||||
}
|
||||
|
||||
/// 全有或全无地应用行编辑。
|
||||
///
|
||||
/// - 任何被触碰行越出 `ranges`(或行号非法)⇒ [`Err`](越界清单),**源不变**;
|
||||
/// - 同一行被多个编辑触碰时 `Delete` 优先于 `Replace`,同种后者覆盖前者。
|
||||
pub fn apply_edits(
|
||||
source: &str,
|
||||
ranges: &[(u32, u32)],
|
||||
edits: &[Edit],
|
||||
) -> Result<String, OutOfScope> {
|
||||
let lines: Vec<&str> = source.lines().collect();
|
||||
let n_lines = lines.len() as u32;
|
||||
|
||||
// ---- 范围检查(先全体验完再动源) ----
|
||||
let mut bad = Vec::new();
|
||||
for e in edits {
|
||||
let line = e.touched();
|
||||
let legal = match e {
|
||||
Edit::InsertBefore { .. } => {
|
||||
(line >= 1 && line <= n_lines + 1)
|
||||
&& (in_ranges(ranges, line) || at_tail(ranges, line))
|
||||
}
|
||||
_ => line >= 1 && line <= n_lines && in_ranges(ranges, line),
|
||||
};
|
||||
if !legal {
|
||||
bad.push(line);
|
||||
}
|
||||
}
|
||||
bad.sort_unstable();
|
||||
bad.dedup();
|
||||
if !bad.is_empty() {
|
||||
return Err(OutOfScope {
|
||||
lines: bad,
|
||||
ranges: ranges.to_vec(),
|
||||
});
|
||||
}
|
||||
|
||||
// ---- 应用 ----
|
||||
let mut replaced: HashMap<u32, String> = HashMap::new();
|
||||
let mut deleted: HashSet<u32> = HashSet::new();
|
||||
let mut inserts: HashMap<u32, Vec<String>> = HashMap::new();
|
||||
for e in edits {
|
||||
match e {
|
||||
Edit::Replace { line, text } => {
|
||||
replaced.insert(*line, text.clone());
|
||||
}
|
||||
Edit::Delete { line } => {
|
||||
deleted.insert(*line);
|
||||
replaced.remove(line); // Delete 优先
|
||||
}
|
||||
Edit::InsertBefore { line, text } => {
|
||||
inserts.entry(*line).or_default().push(text.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let trailing_newline = source.ends_with('\n');
|
||||
let mut out: Vec<String> = Vec::with_capacity(lines.len() + inserts.len());
|
||||
for (i, l) in lines.iter().enumerate() {
|
||||
let n = (i + 1) as u32;
|
||||
if let Some(texts) = inserts.get(&n) {
|
||||
out.extend(texts.iter().cloned());
|
||||
}
|
||||
if deleted.contains(&n) {
|
||||
continue;
|
||||
}
|
||||
out.push(replaced.get(&n).cloned().unwrap_or_else(|| l.to_string()));
|
||||
}
|
||||
if let Some(texts) = inserts.get(&(n_lines + 1)) {
|
||||
out.extend(texts.iter().cloned());
|
||||
}
|
||||
|
||||
let mut s = out.join("\n");
|
||||
if trailing_newline {
|
||||
s.push('\n');
|
||||
}
|
||||
Ok(s)
|
||||
}
|
||||
@@ -27,6 +27,8 @@ pub enum Value {
|
||||
Formula(FormulaExpr),
|
||||
/// External language source from ```lang fence (`type` → `code`).
|
||||
Code(CodeBlock),
|
||||
/// Sensitive text from `${secret.*}` — Display shows `<secret>`; reveal on Text concat.
|
||||
Secret(String),
|
||||
}
|
||||
|
||||
impl Value {
|
||||
@@ -37,6 +39,7 @@ impl Value {
|
||||
Value::Int(n) => *n != 0,
|
||||
Value::Num(n) => *n != 0.0 && !n.is_nan(),
|
||||
Value::Text(s) => !s.is_empty(),
|
||||
Value::Secret(s) => !s.is_empty(),
|
||||
Value::List(xs) => !xs.is_empty(),
|
||||
Value::Map(xs) => !xs.is_empty(),
|
||||
Value::Formula(_) => true,
|
||||
@@ -44,6 +47,14 @@ impl Value {
|
||||
}
|
||||
}
|
||||
|
||||
/// Plaintext for Text / Secret (Authorization etc.). Prefer [`as_display`] for logs.
|
||||
pub fn reveal_text(&self) -> Option<&str> {
|
||||
match self {
|
||||
Value::Text(s) | Value::Secret(s) => Some(s.as_str()),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn as_display(&self) -> String {
|
||||
match self {
|
||||
Value::None => "None".into(),
|
||||
@@ -52,6 +63,7 @@ impl Value {
|
||||
Value::Int(n) => n.to_string(),
|
||||
Value::Num(n) => crate::formula::format_num(*n),
|
||||
Value::Text(s) => s.clone(),
|
||||
Value::Secret(_) => "<secret>".into(),
|
||||
Value::List(xs) => {
|
||||
let parts: Vec<String> = xs.iter().map(|v| v.as_display()).collect();
|
||||
format!("[{}]", parts.join(", "))
|
||||
|
||||
@@ -98,6 +98,7 @@ pub fn api_debug_start(root: &Path, body: &str) -> serde_json::Value {
|
||||
};
|
||||
let mut host = HostContext::for_capture(Some(&path_for_thread), opts.host_caps());
|
||||
host.fs_root = Some(root_for_thread);
|
||||
host.entry_metadata = module.metadata.clone();
|
||||
if let Some(lim) = opts.sleep_limit_ms {
|
||||
host.sleep_limit_ms = Some(lim);
|
||||
}
|
||||
|
||||
@@ -415,6 +415,7 @@ fn value_kind_label(v: &crate::value::Value) -> String {
|
||||
Value::Int(_) => "int".into(),
|
||||
Value::Num(_) => "num".into(),
|
||||
Value::Text(_) => "text".into(),
|
||||
Value::Secret(_) => "secret".into(),
|
||||
Value::Formula(crate::formula::Expr::Matrix { rows, .. }) => {
|
||||
format!("formula · {}×{}", rows.len(), rows.first().map(|r| r.len()).unwrap_or(0))
|
||||
}
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
<!--
|
||||
P4 实验 (b) 材料 · D 臂(全文档背诵组)上下文包。
|
||||
来源:commit 199b5c5^ 的 `.cursor/skills/marqdo/SKILL.md`(背诵式「压缩操作手册」)
|
||||
语言部分原样冻结(Markup→meaning / 最小程序 / 函数调用 / 语句分支 / 导入与内建面)。
|
||||
只读实验材料——**请勿修改**(改了就不是同一个实验)。
|
||||
-->
|
||||
|
||||
## Markup → meaning (v0.3)
|
||||
|
||||
| Marker | Meaning |
|
||||
|--------|---------|
|
||||
| `#` | **Object / type** (constructor body). `# main` = entry object. `# Child = > Parent` = inherit (**no** implicit parent constructor; call `` `self` = > Parent … `` explicitly when needed). |
|
||||
| `##` … `######` | **Function / method** (nesting by heading depth). |
|
||||
| `` + `name` `` under heading | Parameter (optional `` `name`=default ``) |
|
||||
| `1.` `2.` … in body | Branch arm (condition or `N. *` else). **Same-indent restart at `1.` = new branch statement** (not more arms of the previous list). |
|
||||
| `- …` inside body | Loop (`while` or `` [item](coll) `` / `` [`item`](`coll`) ``) |
|
||||
| Line `N. *` | Else arm |
|
||||
| `> fn args` | Call (named `k=v` or positional) |
|
||||
| `修饰 [fn] args` | Bracket-marked call; leading words → `名=True` |
|
||||
| `` > `obj`.method args `` | Method call (`obj` must be a map with `_type`) |
|
||||
| Frontmatter `import bind:path.mq.md` / `导入` | Import file (bind library name) |
|
||||
| Frontmatter `import bind:lib.member` / `导入` | Short name for a library member (same keyword; no separate `use`) |
|
||||
| `> lib.member …` / `> lib.Type.member …` | Call via bare dotted path (instance methods need `` `var`.m ``) |
|
||||
| `**…**` | **Code** (assign / call) — closing `**` touches last token; bare ids are variables; quote text literals |
|
||||
| `*…*` | **Return value** — closing `*` touches last token; bare ids are variables |
|
||||
| `****` / whole-line `**` / `*None*` | Return `None` and end function body |
|
||||
| Lone `---` / `***` | Markdown thematic break (skip; not function end) |
|
||||
| GFM table after empty RHS bind | Collection; get with `[key](coll)` (preferred) or legacy `` `m`[^key] `` |
|
||||
| `- [item](coll)` | Foreach (unchanged) |
|
||||
| `` ```lang `` | Foreign code block (via `lib/foreign`) |
|
||||
| `` `"text"` `` | Quoted string (`\n` `\t` `\\` `\"`; `` `var` `` inside); bare tokens unescaped |
|
||||
| Unmarked prose | Comment |
|
||||
|
||||
Entry: load file (often `index.mq.md`) → collect defs → run `# main`.
|
||||
|
||||
## Minimal program
|
||||
|
||||
```markdown
|
||||
# main
|
||||
|
||||
> print text=Hello World!
|
||||
```
|
||||
|
||||
Chinese builtins (same functions, no import):
|
||||
|
||||
```markdown
|
||||
# main
|
||||
|
||||
> 打印 内容=你好
|
||||
```
|
||||
|
||||
## Functions, calls, end-body
|
||||
|
||||
```markdown
|
||||
# main
|
||||
|
||||
> greet who=Marqdo
|
||||
|
||||
## greet
|
||||
+ `who`
|
||||
|
||||
> print text=Hello, `who`!
|
||||
|
||||
*None*
|
||||
```
|
||||
|
||||
- Call: `> name key=value` or `> name value`.
|
||||
- Nested helpers use deeper `#` (`##`, `###`, …).
|
||||
- After a helper with only side effects, end with `*None*` / whole-line `**` / `****` when later lines must stay in the outer function. Lone `---` is only a thematic break (skipped).
|
||||
- In bold assigns, prefer bare calls: `**摘要 = 结账摘要 …**` (omit `>`).
|
||||
|
||||
## Statements, returns, branches
|
||||
|
||||
```markdown
|
||||
**x = 1**
|
||||
**y = x + 2**
|
||||
|
||||
## add_one
|
||||
+ `n`
|
||||
|
||||
*n + 1*
|
||||
```
|
||||
|
||||
```markdown
|
||||
**n = 0**
|
||||
|
||||
1. `n` > 0
|
||||
> print text=positive
|
||||
2. `n` < 0
|
||||
> print text=negative
|
||||
3. *
|
||||
> print text=zero
|
||||
```
|
||||
|
||||
## Frontmatter imports + stdlib
|
||||
|
||||
Imports use `import bind:target` (`导入` equivalent): file (`.mq.md`) or short name (`lib.member`). Members are **not** flattened into the caller. Call with a bare dotted path:
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: example
|
||||
import text:lib/text.mq.md
|
||||
import clock:lib/time.mq.md
|
||||
---
|
||||
|
||||
# main
|
||||
|
||||
**xs = > text.str_split s="a,b,c" sep=","**
|
||||
> print text=`xs`
|
||||
**t = > clock.now_unix**
|
||||
```
|
||||
|
||||
- Import **English** or **Chinese** library file; call that file's API names via `lib.member` (do not mix languages).
|
||||
- Instance methods stay `` > `obj`.method `` (backticks on the receiver only).
|
||||
- `lib/…` resolves via `MARQDO_LIB`, cwd `lib/`, or `lib/` next to the `marqdo` binary.
|
||||
- Prefer `table.put` / `表.改` for list/map element updates; keep `json` for parse/stringify/quote.
|
||||
- Builtins (no import): `print`/`打印`, `input`/`输入`, `len`/`长度`, `str`/`文本`, `int`/`整数`; literals `True`/`真`, `False`/`假`, `None`/`空`; logic `and`/`且`, `or`/`或`, `not`/`非`.
|
||||
@@ -0,0 +1,36 @@
|
||||
# Task: claims_reserve_settle (insurance claims reserve)
|
||||
|
||||
Implement an **insurance claims reserve settlement** desk.
|
||||
This is a **business-policy** pipeline (deductible, coinsurance, sublimit, recovery, floor), **not** an algorithms puzzle.
|
||||
Use **integer arithmetic only**. Hardcode the demo inputs below and **print one integer**: the final reserve.
|
||||
|
||||
## Business requirements (authoritative)
|
||||
|
||||
1. **After deductible:** `after_deduct = claim - deductible`. If `after_deduct < 0`, set `after_deduct = 0`.
|
||||
2. **Coinsurance share (insurer):** `coinsured = after_deduct * coinsure_pct / 100` (demo coinsure_pct = 80). Apply coinsurance **before** recovery.
|
||||
3. **Sublimit cap:** `capped = coinsured` if `coinsured <= sublimit`, else `capped = sublimit` (never enlarge beyond coinsured).
|
||||
4. **Recovery offset:** `reserved = capped - recovery`. If `reserved < 0`, set `reserved = 0`.
|
||||
5. **Reserve floor:** if `reserved < reserve_floor`, set `reserved = reserve_floor`.
|
||||
|
||||
## Demo inputs (hardcode)
|
||||
|
||||
- `claim = 50000`
|
||||
- `deductible = 5000`
|
||||
- `coinsure_pct = 80`
|
||||
- `sublimit = 30000`
|
||||
- `recovery = 2000`
|
||||
- `reserve_floor = 1000`
|
||||
|
||||
## Requirements
|
||||
|
||||
- Provide named helpers (or clear stages) for deductible, coinsurance, sublimit, recovery, and floor — not one opaque expression.
|
||||
- Print only the final integer (no labels).
|
||||
- Align to the **business requirements above**; do not invent alternate order or comparison directions.
|
||||
|
||||
## Expected output (self-check)
|
||||
|
||||
Exactly one line: `28000`
|
||||
|
||||
## Deliverable
|
||||
|
||||
Write the program as a Marqdo `.mq.md` file. Return the complete source in a single ```markdown fence — no prose outside the fence.
|
||||
@@ -0,0 +1,42 @@
|
||||
# Task: contract_milestone_pay (construction milestone payable)
|
||||
|
||||
Implement a **contract milestone payment** desk for an EPC / construction settlement.
|
||||
This is a **business-policy** pipeline (retention, late penalty, change order, tax), **not** an algorithms puzzle.
|
||||
Use **integer arithmetic only**. Hardcode the demo inputs below and **print one integer**: the final payable.
|
||||
|
||||
## Business requirements (authoritative)
|
||||
|
||||
1. **Earned value:** `earned = contract_value * progress_pct / 100`.
|
||||
2. **Retention withhold:** `retention = earned * retain_pct / 100` (demo uses retain_pct = 5).
|
||||
3. **Net after retention:** `net = earned - retention`.
|
||||
4. **Late penalty:** if `days_late > grace_days` (grace_days = 7), set `penalty = net * late_rate / 100` (late_rate = 2); otherwise `penalty = 0`. Penalty is always computed from **`net`**, never from `earned`.
|
||||
5. **After penalty:** `net2 = net - penalty`.
|
||||
6. **Change order:** `taxed_base = net2 + change_order`. If `taxed_base < 0`, set `taxed_base = 0`.
|
||||
7. **Tax:** `tax = taxed_base * tax_rate / 100` (tax_rate = 6).
|
||||
8. **Payable:** `payable = taxed_base + tax`. If `payable < min_pay` (min_pay = 500), set `payable = min_pay`.
|
||||
|
||||
## Demo inputs (hardcode)
|
||||
|
||||
- `contract_value = 100000`
|
||||
- `progress_pct = 40`
|
||||
- `retain_pct = 5`
|
||||
- `days_late = 10`
|
||||
- `grace_days = 7`
|
||||
- `late_rate = 2`
|
||||
- `change_order = -2000`
|
||||
- `tax_rate = 6`
|
||||
- `min_pay = 500`
|
||||
|
||||
## Requirements
|
||||
|
||||
- Provide named helpers (or clear stages) for earned, retention, penalty, change+tax, and floor — not one opaque expression.
|
||||
- Print only the final integer (no labels).
|
||||
- Align to the **business requirements above**; do not invent alternate thresholds or penalty bases.
|
||||
|
||||
## Expected output (self-check)
|
||||
|
||||
Exactly one line: `37354`
|
||||
|
||||
## Deliverable
|
||||
|
||||
Write the program as a Marqdo `.mq.md` file. Return the complete source in a single ```markdown fence — no prose outside the fence.
|
||||
@@ -0,0 +1,30 @@
|
||||
# Task: qb_gcd (easy) — Greatest Common Divisor
|
||||
|
||||
Implement **Greatest Common Divisor** as a function named `gcd` in a Marqdo `.mq.md` program (Euclidean algorithm).
|
||||
|
||||
## Requirements
|
||||
|
||||
1. Function name `gcd`, taking two integer parameters and returning their greatest common divisor (non-negative; `gcd(0, 5)` is `5`).
|
||||
2. In `# main`, hardcode the demo calls below and print each result on its own line — exactly four lines, no labels.
|
||||
|
||||
## Demo calls (hardcode)
|
||||
|
||||
- `gcd(48, 18)`
|
||||
- `gcd(1071, 462)`
|
||||
- `gcd(0, 5)`
|
||||
- `gcd(17, 5)`
|
||||
|
||||
## Expected output (self-check)
|
||||
|
||||
Four lines:
|
||||
|
||||
```
|
||||
6
|
||||
21
|
||||
5
|
||||
1
|
||||
```
|
||||
|
||||
## Deliverable
|
||||
|
||||
Write the program as a Marqdo `.mq.md` file. Return the complete source in a single ```markdown fence — no prose outside the fence.
|
||||
@@ -0,0 +1,28 @@
|
||||
# Task: qb_get_factors (hard) — Prime Factorization
|
||||
|
||||
Implement **prime factorization** as a function named `get_factors` in a Marqdo `.mq.md` program. The function takes an integer `n ≥ 2` and returns its prime factors in ascending order.
|
||||
|
||||
## Requirements
|
||||
|
||||
1. Function name `get_factors`.
|
||||
2. In `# main`, hardcode the demo inputs below and print each result on its own line — factors joined by `*` with no spaces, exactly three lines, no labels.
|
||||
|
||||
## Demo inputs (hardcode)
|
||||
|
||||
- `12`
|
||||
- `13`
|
||||
- `60`
|
||||
|
||||
## Expected output (self-check)
|
||||
|
||||
Three lines:
|
||||
|
||||
```
|
||||
2*2*3
|
||||
13
|
||||
2*2*3*5
|
||||
```
|
||||
|
||||
## Deliverable
|
||||
|
||||
Write the program as a Marqdo `.mq.md` file. Return the complete source in a single ```markdown fence — no prose outside the fence.
|
||||
Loaded 100 of 139 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user