Add local n-gram near match for OKF agent-kb reuse.
Sparse bigram cosine closes near-duplicate goal fragmentation; soft_match parents now judge ranked scored candidates instead of a flat task list. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
1 parent
f240bdf156
commit
ab8ea8438f
9 files changed
+427
-88
No files matched your search
@@ -83,7 +83,7 @@ cargo build -p marqdo_plugin_agent
|
||||
```
|
||||
|
||||
- Demo (ABI v1): `demo_add`, `demo_echo` — [`tests/lib/plugin-demo.mq.md`](../../tests/lib/plugin-demo.mq.md).
|
||||
- Agent (ABI v2): layout + session bag + context queries + **OKF agent-kb** (`agent_goal_sig`, `agent_goal_slug`, `agent_kb_lookup`, `agent_kb_canonicalize`, `agent_kb_list_tasks`, `agent_kb_add_alias`, `agent_kb_promote`, `agent_kb_record_hit`, `agent_workbook_solidify`, `agent_kb_task_files`) — [`plugins/agent`](../../plugins/agent); used by [`ext/ai/agent.mq.md`](../../ext/ai/agent.mq.md). Lookup: `plugin.native_path name=agent` or [ext-cli.md](ext-cli.md) install / `MARQDO_AGENT_PLUGIN`.
|
||||
- Agent (ABI v2): layout + session bag + context queries + **OKF agent-kb** (`agent_goal_sig`, `agent_goal_slug`, `agent_kb_lookup`, `agent_kb_near_match`, `agent_kb_canonicalize`, `agent_kb_list_tasks`, `agent_kb_add_alias`, `agent_kb_promote`, `agent_kb_record_hit`, `agent_workbook_solidify`, `agent_kb_task_files`) — [`plugins/agent`](../../plugins/agent); used by [`ext/ai/agent.mq.md`](../../ext/ai/agent.mq.md). Lookup: `plugin.native_path name=agent` or [ext-cli.md](ext-cli.md) install / `MARQDO_AGENT_PLUGIN`.
|
||||
|
||||
## Layering
|
||||
|
||||
|
||||
+7
-4
@@ -347,8 +347,9 @@ Prefer spawning `resource` over re-planning.
|
||||
1. **精确**:slug 试探 + frontmatter `sig` → 扫目录 `sig:` → 旧 hex 文件名。
|
||||
2. **别名(Accepted)**:精确 miss 后,扫 Task FM `aliases:`(YAML 列表或 `[a, b]`);`normalize_goal(query)` 与某条 alias **精确相等**则命中同一 Skill/resource。lookup 返回 `match: exact|alias`;`plan` 对 alias 写 `cache=soft-hit`。
|
||||
3. **规范句(Accepted)**:仍 miss 时对 goal 做轻量 `agent_kb_canonicalize`(剥站立前缀、去尾标点),再精确/别名各试一次;命中返回 `match: canonical`,`cache=soft-hit`。
|
||||
4. **父裁决 soft_match(Accepted,默认关)**:`plan soft_match=True` 且仍 miss 时,把 `agent_kb_list_tasks` 候选交给父模型:`DECISION: REUSE` + `SLUG:` 或 `DECISION: NEW`;REUSE 后可 `agent_kb_add_alias` 把当前 goal 写入别名。
|
||||
5. **不做**默认 embedding / 编辑距离主路径。向量层见 [okf-near-match.md](../roadmap/okf-near-match.md) E。
|
||||
4. **近义(Accepted,默认开)**:仍 miss 且 `near_match=True` 时,对 Task `title|description|aliases` 做本地**字符 bigram 词袋 + 余弦**;`best.score >= near_threshold`(默认 `0.78`)则 `match: near` 并带 `score`,`cache=soft-hit`,并可 `agent_kb_add_alias`。这是稀疏词法近邻,**不是** embedding 模型。
|
||||
5. **父裁决 soft_match(Accepted,默认关)**:仍 miss 且 `plan soft_match=True` 时,把 `agent_kb_near_match` 排序候选(含 score;空则退回 `list_tasks`)交给父模型:`DECISION: REUSE` + `SLUG:` 或 `DECISION: NEW`;REUSE 后可 `agent_kb_add_alias`。
|
||||
6. **不做**默认 dense embedding / 远程向量库。远期 dense 仍见 [okf-near-match.md](../roadmap/okf-near-match.md) E。
|
||||
|
||||
`agent_kb_promote` 可选 `aliases=`(字符串或列表)写入 Task;再晋升时合并既有 aliases。
|
||||
|
||||
@@ -359,7 +360,9 @@ Prefer spawning `resource` over re-planning.
|
||||
| `reuse` / `复用` | `True` | 允许知识包快路径 |
|
||||
| `optimize` / `优化` | `False` | 跳过知识包,强制再规划 |
|
||||
| `force` / `强制` | `False` | 忽略知识包 |
|
||||
| `soft_match` / `软命中` | `False` | miss 后父裁决 REUSE/NEW(须显式打开) |
|
||||
| `soft_match` / `软命中` | `False` | miss 后父裁决 REUSE/NEW(须显式打开;候选为 near 排序列表) |
|
||||
| `near_match` / `近义命中` | `True` | lookup 第四趟本地 n-gram;`False` 关闭 |
|
||||
| `near_threshold` / `近义阈值` | `0.78` | near 自动命中最低余弦分 |
|
||||
| `promote` / `晋升` | `True` | DONE 后固化并写入/更新概念与 resource |
|
||||
| `kb_dir` / `知识库目录` | `.marqdo/agent-kb` | bundle 根 |
|
||||
| `workbook_dir` / `工作簿目录` | `None` | 默认直接写 `kb_dir/resources/<slug>.mq.md`;显式传入则用该目录下的临时 `workbook-<slug>-<ts>.mq.md` |
|
||||
@@ -435,7 +438,7 @@ agent-kb 运行时实现归属 **agent 插件**(`agent_goal_sig` / `agent_kb_l
|
||||
| **O2** | 本文锁定的任务知识包:`Marqdo Task` / `Agent Skill`;`plan` reuse/promote | **done**(host kb_* + plan 快路径) |
|
||||
| **O3** | catalog 增强:`verified`/`sources` 可选;模块页可点击依赖;扫入 `agent-kb` 概念供人浏览 | **done** |
|
||||
| **O4** | `log.md` 策展、可选 `kb query`、Attested-style「再现跑通」收据(非 BigQuery) | 远期 |
|
||||
| **O5** | 相近任务软命中:aliases + canonicalize + `soft_match` 父裁决已落地;向量仍见 [okf-near-match.md](../roadmap/okf-near-match.md) E | **A/B/C done** |
|
||||
| **O5** | 相近任务软命中:aliases + canonicalize + 本地 n-gram `near` + `soft_match` 父裁决已落地;dense embedding 仍见 [okf-near-match.md](../roadmap/okf-near-match.md) E | **A/B/C/D-near done** |
|
||||
|
||||
O2 设计落盘后,智能体侧实现文档可薄化为指向本文 §7,避免两套真理。
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# OKF 相近任务命中(未来规划)
|
||||
# OKF 相近任务命中
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **A 规范句 + B aliases + C 父裁决(soft_match)已落地**;E 向量仍待 |
|
||||
| 状态 | **A 规范句 + B aliases + C 父裁决(soft_match)+ D 本地 n-gram `near` 已落地**;E dense embedding 仍非目标 |
|
||||
| 日期 | 2026-08-11 |
|
||||
| 相关 | [okf.md](../design/okf.md) §7 · [ext-agent-plan.md](../design/ext-agent-plan.md) §4.2 · [ext-agent.md](../design/ext-agent.md) |
|
||||
| 触发观察 | `man-test`:相近 goal 生成两份独立 resource |
|
||||
@@ -11,127 +11,98 @@
|
||||
|
||||
## 1. 问题
|
||||
|
||||
v1 agent-kb **默认只做精确命中**(仍是默认快路径):
|
||||
v1 agent-kb **精确命中仍是默认快路径**:
|
||||
|
||||
1. `normalize_goal`:去首尾空白、合并连续空白(**不做**近义改写)。
|
||||
2. `sig` = 规范化字符串的稳定短哈希;`slug` = 可读路径名。
|
||||
3. `agent_kb_lookup`:精确 → aliases → **canonicalize 后再精确/别名**;仍 miss 且 `plan soft_match=True` 时父裁决 REUSE/NEW。
|
||||
3. `agent_kb_lookup`:精确 → aliases → canonicalize → **本地 n-gram `near`(默认开)**;仍 miss 且 `plan soft_match=True` 时父裁决 REUSE/NEW(候选为排序 near 列表)。
|
||||
|
||||
历史痛点(已由 A/B/C 缓解):
|
||||
历史痛点(已由 A/B/C/D 缓解):
|
||||
|
||||
| goal(调用方输入) | 旧行为 |
|
||||
|--------------------|--------|
|
||||
| `你是一个智能体,帮我规划明天的行程` | 独立 slug,与下方不命中 |
|
||||
| `帮我规划明天的行程` | 另一份 resource |
|
||||
| `帮我规划明天行程` | 又一份(近句碎片) |
|
||||
|
||||
调用方直觉:「这不是同一件事吗?」——现可用 canonicalize(剥站立前缀)或显式 `soft_match` 父裁决。
|
||||
调用方直觉:「这不是同一件事吗?」——现可用 canonicalize、默认 `near_match`,或显式 `soft_match` 父裁决。
|
||||
|
||||
---
|
||||
|
||||
## 2. 为何 v1 坚持精确指纹
|
||||
## 2. 为何精确指纹仍优先
|
||||
|
||||
| 理由 | 说明 |
|
||||
|------|------|
|
||||
| 假阳性代价高 | 错复用旧工作簿比多建一个文件更糟(静默答错题) |
|
||||
| 可审计 | `sig` 可复算;命中不依赖黑盒相似度阈值 |
|
||||
| 体量假设 | 单仓 agent-kb 通常不大;调用方可固定规范 goal |
|
||||
| 假阳性代价高 | 错复用旧工作簿比多建一个文件更糟 |
|
||||
| 可审计 | `sig` 可复算;`near` 留下 `match`/`score` |
|
||||
| 哲学 | 多轮 = 子文件接力 + OKF 索引,不是聊天式模糊检索 |
|
||||
|
||||
**结论:** 精确命中应长期保留为**默认快路径**;相近命中只能是**显式可选层**,且必须可关闭、可解释。
|
||||
**结论:** 精确命中长期保留为**默认快路径**;`near` 是可关闭的第四趟(`near_match=False`)。
|
||||
|
||||
---
|
||||
|
||||
## 3. 目标
|
||||
|
||||
在 **不破坏** 精确 `sig` 契约的前提下:
|
||||
|
||||
1. **已落地**:A 规范句、B aliases、C `soft_match` 父裁决(默认关)。
|
||||
2. **仍待**:E 向量(仅有证据再开)。
|
||||
|
||||
非目标(仍不做为默认):
|
||||
|
||||
- 把 agent-kb 做成通用向量知识库 / RAG 产品。
|
||||
- 用不可审计的静默相似度覆盖精确 `sig`。
|
||||
- 单文件对话日志 / 状态表来「记住」同主题(与 [ext-agent-plan.md](../design/ext-agent-plan.md) §4.2 冲突)。
|
||||
|
||||
---
|
||||
|
||||
## 4. 候选方案(不必上向量库)
|
||||
|
||||
相近命中 **不等于** 必须上向量检索。按与 Marqdo 契合度排序:
|
||||
## 3. 已落地层
|
||||
|
||||
### A. 规范化后再精确命中 — **已落地**
|
||||
|
||||
| | |
|
||||
|--|--|
|
||||
| 做法 | `agent_kb_canonicalize`:剥站立前缀、去尾 `?`/`?`/`。`,再 `normalize_goal`;lookup 第三趟 |
|
||||
| 落地 | 插件 + `match: canonical`;`cache=soft-hit` |
|
||||
`agent_kb_canonicalize`:剥站立前缀、去尾标点;`match: canonical`。
|
||||
|
||||
### B. 别名 / 标签表 — **已落地**
|
||||
|
||||
| | |
|
||||
|--|--|
|
||||
| 做法 | Task FM `aliases:`;lookup 第二趟精确匹配 |
|
||||
| 落地 | `match: alias`;promote / `agent_kb_add_alias` |
|
||||
Task FM `aliases:`;`match: alias`。
|
||||
|
||||
### C. 父智能体软裁决(miss 后)— **已落地(默认关)**
|
||||
### C. 父智能体软裁决 — **已落地(默认关)**
|
||||
|
||||
`plan soft_match=True`:`agent_kb_near_match` 排序候选(含 `score`;空则退回 `list_tasks`)→ 父 `DECISION: REUSE` + `SLUG:` 或 `DECISION: NEW`。
|
||||
|
||||
### D. 本地字符 bigram + 余弦 — **已落地(默认开)**
|
||||
|
||||
| | |
|
||||
|--|--|
|
||||
| 做法 | `plan soft_match=True`:`agent_kb_list_tasks` → 父 `DECISION: REUSE` + `SLUG:` 或 `DECISION: NEW` |
|
||||
| 落地 | REUSE → 现有 hit 分支;可选 append alias |
|
||||
| 做法 | 稀疏词法向量化(**不是** embedding 模型):canonicalize 后对 goal 与 Task `title\|description\|aliases` 建字符 bigram 词袋,算余弦 |
|
||||
| API | `agent_kb_near_match`;`agent_kb_lookup` 第四趟 |
|
||||
| 参数 | `near_match`/`近义命中` 默认 `True`;`near_threshold`/`近义阈值` 默认 `0.78` |
|
||||
| 痕迹 | `match: near` + `score`;`cache=soft-hit`;可写 alias |
|
||||
|
||||
### D. 字面近似(弱,仅拼写容错)
|
||||
术语:这是 **sparse lexical near-match**,勿宣传为向量库 / embedding 检索。
|
||||
|
||||
| | |
|
||||
|--|--|
|
||||
| 做法 | 编辑距离、共用字符 n-gram、去标点后再比 |
|
||||
| 优点 | 零模型、便宜 |
|
||||
| 风险 | 中文近义(「行程 / 一日游」)基本无效;易假阳性 |
|
||||
| 建议 | 最多作 D 的辅助,不单独作为语义层 |
|
||||
### E. Dense embedding(可选远期)
|
||||
|
||||
### E. 向量检索(可选远期)
|
||||
|
||||
| | |
|
||||
|--|--|
|
||||
| 做法 | 对 task `description` / 规范句做 embedding,ANN Top-K,再经阈值或父模型确认 |
|
||||
| 优点 | 开放域措辞更稳 |
|
||||
| 风险 | 依赖外部分数与阈值;运维与可审计性差;偏离「Markdown 即索引」 |
|
||||
| 建议 | **仅当** A–C 在真实包体量下不够时再评估;默认关闭 |
|
||||
远程 / 本地 dense 向量与 ANN **仍非本仓默认目标**;仅当 D 在真实包体量下证据不足时再评估。
|
||||
|
||||
---
|
||||
|
||||
## 5. 演进顺序
|
||||
## 4. 演进顺序
|
||||
|
||||
```text
|
||||
精确 sig(永不删除)
|
||||
→ A 规范句 + B aliases(已落地)
|
||||
→ C soft_match 父裁决(已落地,默认关)
|
||||
→ E 向量(仅有证据再开)
|
||||
→ D 本地 n-gram near(已落地,默认开)
|
||||
→ E dense embedding(仅有证据再开)
|
||||
```
|
||||
|
||||
验收原则(仍适用):
|
||||
验收原则:
|
||||
|
||||
1. 默认行为与精确路径一致(`soft_match` 默认 False;canonicalize 无模型、低风险)。
|
||||
2. 软命中留下 `cache=soft-hit` / `match` 痕迹。
|
||||
3. 假阳性可被 `force=True` / 显式 `workbook=` 覆盖。
|
||||
1. `near_match=False` 时与无 near 行为一致;`force`/`optimize` 仍跳过 reuse。
|
||||
2. 软/近义命中留下 `cache=soft-hit` / `match` / 可选 `score`。
|
||||
3. 假阳性可被 `force=True` / 显式 `workbook=` / 调高 `near_threshold` 覆盖。
|
||||
4. 实现仍在 **agent 插件 + `ext/ai`**,不进核心 `src/host`。
|
||||
|
||||
---
|
||||
|
||||
## 6. 调用方建议
|
||||
## 5. 调用方建议
|
||||
|
||||
- 固定规范 goal 字符串;站立语可由 canonicalize 剥掉,但仍宜少拼。
|
||||
- 续跑同一主题时传 `workbook=` 或已知 resource 路径。
|
||||
- 近义但字面差大时显式 `soft_match=True`(多一次父 LLM)。
|
||||
- 策展:合并重复 task,次要措辞写入 `aliases`。
|
||||
- 固定规范 goal;站立语可由 canonicalize 剥掉。
|
||||
- 近句碎片依赖默认 `near_match`;字面差更大时开 `soft_match=True`。
|
||||
- 策展:合并重复 task,次要措辞写入 `aliases`(O4)。
|
||||
|
||||
---
|
||||
|
||||
## 7. 与现有路线图的关系
|
||||
## 6. 与现有路线图的关系
|
||||
|
||||
| 条目 | 关系 |
|
||||
|------|------|
|
||||
| [okf.md](../design/okf.md) O2 / O5 | 精确 reuse + A/B/C 软命中;E 向量仍待 |
|
||||
| [okf.md](../design/okf.md) O2 / O5 | 精确 reuse + A/B/C/D;E dense 仍待 |
|
||||
| [okf.md](../design/okf.md) O4 | `kb query` / 策展可继续扩 aliases |
|
||||
| [ext-agent-plan.md](../design/ext-agent-plan.md) | 多轮仍只走子文件接力;软命中只影响「选哪个子文件」 |
|
||||
+26
-9
@@ -936,19 +936,24 @@ Parse `SLUG: …` / `标识:…` after a soft-match REUSE decision.
|
||||
+ `goal`
|
||||
+ `tasks`
|
||||
|
||||
Short parent prompt: REUSE+SLUG or NEW only.
|
||||
Short parent prompt: REUSE+SLUG or NEW only. Candidates may include lexical `score` (prefer higher).
|
||||
|
||||
*`esc` = > json.parse text={"a":"\n\n--- goal ---\n","b":"\n\n--- candidate tasks (slug | title) ---\n","c":"\n\n--- how to act ---\nExact reuse miss. Reply with ONE protocol only:\nDECISION: REUSE\nSLUG: <exact-slug-from-list>\nor\nDECISION: NEW\nNo other prose.\n","nl":"\n","sep":" | "} *
|
||||
*`esc` = > json.parse text={"a":"\n\n--- goal ---\n","b":"\n\n--- candidate tasks (slug | title | score) ---\n","c":"\n\n--- how to act ---\nExact reuse miss; candidates are ranked by local n-gram score when present. Prefer higher scores. Reply with ONE protocol only:\nDECISION: REUSE\nSLUG: <exact-slug-from-list>\nor\nDECISION: NEW\nNo other prose.\n","nl":"\n","sep":" | ","sp":"score="} *
|
||||
*`a` = > json.get value=`esc` key=a *
|
||||
*`b` = > json.get value=`esc` key=b *
|
||||
*`c` = > json.get value=`esc` key=c *
|
||||
*`nl` = > json.get value=`esc` key=nl *
|
||||
*`sep` = > json.get value=`esc` key=sep *
|
||||
*`sp` = > json.get value=`esc` key=sp *
|
||||
*`lines` = > json.parse text=[] *
|
||||
- [`t`](`tasks`)
|
||||
*`slug` = > json.get value=`t` key=slug *
|
||||
*`title` = > json.get value=`t` key=title *
|
||||
*`row` = `slug` + `sep` + `title` *
|
||||
*`score` = > json.get value=`t` key=score *
|
||||
1. `score`
|
||||
*`row` = `slug` + `sep` + `title` + `sep` + `sp` + `score` *
|
||||
2. *
|
||||
*`row` = `slug` + `sep` + `title` *
|
||||
*`lines` = > json.append list=`lines` item=`row` *
|
||||
*`catalog` = > join value=`lines` sep=`nl` *
|
||||
*`p` = Soft-match: same task intent? *
|
||||
@@ -1408,6 +1413,8 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
|
||||
+ `optimize`=False
|
||||
+ `force`=False
|
||||
+ `soft_match`=False
|
||||
+ `near_match`=True
|
||||
+ `near_threshold`=0.78
|
||||
+ `promote`=True
|
||||
+ `kb_dir`=.marqdo/agent-kb
|
||||
+ `improve_every`=3
|
||||
@@ -1418,7 +1425,7 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
|
||||
|
||||
Multi-step with OKF agent-kb. Default workbook is `kb_dir/resources/<slug>.mq.md`. While task file count `< explore_n` and skill is not llm_free, force a new explore variant under `kb_dir/explore/<slug>/`. Code-first: llm_free hits skip parent LLM. File children return via `# main`; `plan` exposes that as `result`.
|
||||
|
||||
Non-hit path: optional `soft_match=True` parent REUSE/NEW; else **decompose** before first child spawn (`DECISION: RUN` / `CONTINUE`+patch / solidified `DONE`). Then `await` → revise loop. With `stream=True`, emit parent `delta` / `decision` / `tool_start`/`tool_end` / `round` / `done` on `events`. `echo=True` prints `plan:decompose` / `plan:await` / deltas. `trace=True` writes events to writeback slot `trace`. Quiet child subtasks stay quiet.
|
||||
Reuse lookup: exact → alias → canonicalize → optional local n-gram `near` when `near_match=True` and score ≥ `near_threshold`. Non-hit path: optional `soft_match=True` parent REUSE/NEW over ranked `agent_kb_near_match` candidates; else **decompose** before first child spawn (`DECISION: RUN` / `CONTINUE`+patch / solidified `DONE`). Then `await` → revise loop. With `stream=True`, emit parent `delta` / `decision` / `tool_start`/`tool_end` / `round` / `done` on `events`. `echo=True` prints `plan:decompose` / `plan:await` / deltas. `trace=True` writes events to writeback slot `trace`. Quiet child subtasks stay quiet.
|
||||
|
||||
*`tools` = > json.get value=`self` key=tools *
|
||||
*`cache` = miss*
|
||||
@@ -1437,7 +1444,7 @@ Non-hit path: optional `soft_match=True` parent REUSE/NEW; else **decompose** be
|
||||
2. `optimize`
|
||||
*`_` = 1*
|
||||
3. `reuse`
|
||||
*`hit` = > agent_kb_lookup kb_dir=`kb_dir` goal=`goal` tools=`tools` *
|
||||
*`hit` = > agent_kb_lookup kb_dir=`kb_dir` goal=`goal` tools=`tools` near_match=`near_match` near_threshold=`near_threshold` *
|
||||
1. `hit`
|
||||
*`match_kind` = > json.get value=`hit` key=match *
|
||||
1. `match_kind` == alias
|
||||
@@ -1446,9 +1453,13 @@ Non-hit path: optional `soft_match=True` parent REUSE/NEW; else **decompose** be
|
||||
*`cache_label` = soft-hit*
|
||||
*`hit_slug` = > json.get value=`hit` key=slug *
|
||||
> agent_kb_add_alias kb_dir=`kb_dir` slug=`hit_slug` alias=`goal`
|
||||
3. `match_kind` == soft
|
||||
3. `match_kind` == near
|
||||
*`cache_label` = soft-hit*
|
||||
4. *
|
||||
*`hit_slug` = > json.get value=`hit` key=slug *
|
||||
> agent_kb_add_alias kb_dir=`kb_dir` slug=`hit_slug` alias=`goal`
|
||||
4. `match_kind` == soft
|
||||
*`cache_label` = soft-hit*
|
||||
5. *
|
||||
*`cache_label` = hit*
|
||||
*`lf` = > json.get value=`hit` key=llm_free *
|
||||
1. `lf`
|
||||
@@ -1533,9 +1544,15 @@ Non-hit path: optional `soft_match=True` parent REUSE/NEW; else **decompose** be
|
||||
*`path` = None*
|
||||
2. *
|
||||
1. `soft_match`
|
||||
*`listed` = > agent_kb_list_tasks kb_dir=`kb_dir` *
|
||||
*`tasks` = > json.get value=`listed` key=tasks *
|
||||
*`near` = > agent_kb_near_match kb_dir=`kb_dir` goal=`goal` *
|
||||
*`tasks` = > json.get value=`near` key=candidates *
|
||||
*`tn` = > len value=`tasks` *
|
||||
1. `tn` < 1
|
||||
*`listed` = > agent_kb_list_tasks kb_dir=`kb_dir` *
|
||||
*`tasks` = > json.get value=`listed` key=tasks *
|
||||
*`tn` = > len value=`tasks` *
|
||||
2. *
|
||||
*`_` = 1*
|
||||
1. `tn` > 0
|
||||
*`soft_prompt` = > build_soft_match_prompt goal=`goal` tasks=`tasks` *
|
||||
*`model` = > json.get value=`self` key=model *
|
||||
|
||||
+4
-2
@@ -216,6 +216,8 @@ description: 文档驱动智能体 — 单步 / 多步;工具经 lib/子任务
|
||||
+ `优化`=False
|
||||
+ `强制`=False
|
||||
+ `软命中`=False
|
||||
+ `近义命中`=True
|
||||
+ `近义阈值`=0.78
|
||||
+ `晋升`=True
|
||||
+ `知识库目录`=.marqdo/agent-kb
|
||||
+ `改进周期`=3
|
||||
@@ -224,10 +226,10 @@ description: 文档驱动智能体 — 单步 / 多步;工具经 lib/子任务
|
||||
+ `打印增量`=False
|
||||
+ `轨迹`=False
|
||||
|
||||
多步工作簿默认写入 OKF `知识库目录/resources/<slug>.mq.md`;仅当传入 `工作簿目录` 时才用临时 runs。转发至英文 `agent.plan`。`流式` / `打印增量` / `轨迹` 对应 `stream` / `echo` / `trace`;`软命中` → `soft_match`。
|
||||
多步工作簿默认写入 OKF `知识库目录/resources/<slug>.mq.md`;仅当传入 `工作簿目录` 时才用临时 runs。转发至英文 `agent.plan`。`流式` / `打印增量` / `轨迹` 对应 `stream` / `echo` / `trace`;`软命中` → `soft_match`;`近义命中` / `近义阈值` → `near_match` / `near_threshold`。
|
||||
|
||||
*`model` = > json.get value=`self` key=model *
|
||||
*`tools` = > json.get value=`self` key=tools *
|
||||
*`standing` = > json.get value=`self` key=standing *
|
||||
*`en` = > agent.agent model=`model` tools=`tools` standing=`standing` *
|
||||
**> `en`.plan goal=`目标` workbook_dir=`工作簿目录` workbook=`工作簿` max_rounds=`最大轮次` confirm=`确认分解` writeback=`写回` skeleton=`骨架` reuse=`复用` optimize=`优化` force=`强制` soft_match=`软命中` promote=`晋升` kb_dir=`知识库目录` improve_every=`改进周期` explore_n=`探索次数` stream=`流式` echo=`打印增量` trace=`轨迹`**
|
||||
**> `en`.plan goal=`目标` workbook_dir=`工作簿目录` workbook=`工作簿` max_rounds=`最大轮次` confirm=`确认分解` writeback=`写回` skeleton=`骨架` reuse=`复用` optimize=`优化` force=`强制` soft_match=`软命中` near_match=`近义命中` near_threshold=`近义阈值` promote=`晋升` kb_dir=`知识库目录` improve_every=`改进周期` explore_n=`探索次数` stream=`流式` echo=`打印增量` trace=`轨迹`**
|
||||
+248
-4
@@ -1,5 +1,6 @@
|
||||
//! OKF agent-kb helpers (plugin-side). No dependency on marqdo `src/host`.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
@@ -606,6 +607,158 @@ pub fn kb_canonicalize(args: &Value) -> Result<Value, String> {
|
||||
Ok(Value::String(canonicalize_goal_str(goal)))
|
||||
}
|
||||
|
||||
fn arg_bool(args: &Value, key: &str, default: bool) -> bool {
|
||||
match args.get(key) {
|
||||
None | Some(Value::Null) => default,
|
||||
Some(Value::Bool(b)) => *b,
|
||||
Some(Value::Number(n)) => n.as_i64().unwrap_or(0) != 0,
|
||||
Some(Value::String(s)) => matches!(
|
||||
s.trim(),
|
||||
"1" | "true" | "True" | "TRUE" | "yes" | "Yes" | "on" | "ON"
|
||||
),
|
||||
_ => default,
|
||||
}
|
||||
}
|
||||
|
||||
fn arg_f64(args: &Value, key: &str, default: f64) -> f64 {
|
||||
match args.get(key) {
|
||||
None | Some(Value::Null) => default,
|
||||
Some(Value::Number(n)) => n.as_f64().unwrap_or(default),
|
||||
Some(Value::String(s)) => s.parse().unwrap_or(default),
|
||||
_ => default,
|
||||
}
|
||||
}
|
||||
|
||||
/// Character bigram bag (whitespace stripped). Sparse lexical vectorization — not an embedding model.
|
||||
pub fn char_bigram_tf(text: &str) -> HashMap<String, f64> {
|
||||
let chars: Vec<char> = text.chars().filter(|c| !c.is_whitespace()).collect();
|
||||
let mut m = HashMap::new();
|
||||
if chars.is_empty() {
|
||||
return m;
|
||||
}
|
||||
if chars.len() == 1 {
|
||||
m.insert(chars[0].to_string(), 1.0);
|
||||
return m;
|
||||
}
|
||||
for w in chars.windows(2) {
|
||||
let key = format!("{}{}", w[0], w[1]);
|
||||
*m.entry(key).or_insert(0.0) += 1.0;
|
||||
}
|
||||
m
|
||||
}
|
||||
|
||||
pub fn cosine_tf(a: &HashMap<String, f64>, b: &HashMap<String, f64>) -> f64 {
|
||||
let mut na = 0.0;
|
||||
let mut nb = 0.0;
|
||||
for v in a.values() {
|
||||
na += v * v;
|
||||
}
|
||||
for v in b.values() {
|
||||
nb += v * v;
|
||||
}
|
||||
if na == 0.0 || nb == 0.0 {
|
||||
return 0.0;
|
||||
}
|
||||
let mut dot = 0.0;
|
||||
if a.len() <= b.len() {
|
||||
for (k, va) in a {
|
||||
if let Some(vb) = b.get(k) {
|
||||
dot += va * vb;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
for (k, vb) in b {
|
||||
if let Some(va) = a.get(k) {
|
||||
dot += va * vb;
|
||||
}
|
||||
}
|
||||
}
|
||||
dot / (na.sqrt() * nb.sqrt())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
fn near_score(query: &str, candidate: &str) -> f64 {
|
||||
let q = canonicalize_goal_str(query);
|
||||
let c = canonicalize_goal_str(candidate);
|
||||
if q.is_empty() || c.is_empty() {
|
||||
return 0.0;
|
||||
}
|
||||
cosine_tf(&char_bigram_tf(&q), &char_bigram_tf(&c))
|
||||
}
|
||||
|
||||
fn task_near_material(src: &str, slug: &str) -> String {
|
||||
let title = extract_fm_field(src, "title").unwrap_or_else(|| slug.to_string());
|
||||
let desc = extract_fm_field(src, "description").unwrap_or_default();
|
||||
let aliases = extract_fm_aliases(src);
|
||||
let mut parts = vec![title, desc];
|
||||
parts.extend(aliases);
|
||||
parts.join(" ")
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
struct NearCand {
|
||||
slug: String,
|
||||
title: String,
|
||||
score: f64,
|
||||
}
|
||||
|
||||
fn rank_near_tasks(root: &Path, goal: &str, limit: usize) -> Vec<NearCand> {
|
||||
let tasks_dir = root.join("concepts/tasks");
|
||||
let skills_dir = root.join("concepts/skills");
|
||||
let res_dir = root.join("resources");
|
||||
if !tasks_dir.is_dir() {
|
||||
return Vec::new();
|
||||
}
|
||||
let q = canonicalize_goal_str(goal);
|
||||
let qv = char_bigram_tf(&q);
|
||||
if qv.is_empty() {
|
||||
return Vec::new();
|
||||
}
|
||||
let mut out = Vec::new();
|
||||
let Ok(rd) = fs::read_dir(&tasks_dir) else {
|
||||
return Vec::new();
|
||||
};
|
||||
for ent in rd.flatten() {
|
||||
let p = ent.path();
|
||||
if p.extension().and_then(|e| e.to_str()) != Some("md") {
|
||||
continue;
|
||||
}
|
||||
let Ok(src) = fs::read_to_string(&p) else {
|
||||
continue;
|
||||
};
|
||||
let slug = p
|
||||
.file_stem()
|
||||
.map(|s| s.to_string_lossy().into_owned())
|
||||
.unwrap_or_default();
|
||||
if slug.is_empty() {
|
||||
continue;
|
||||
}
|
||||
if !skills_dir.join(format!("{slug}.md")).is_file()
|
||||
|| !res_dir.join(format!("{slug}.mq.md")).is_file()
|
||||
{
|
||||
continue;
|
||||
}
|
||||
let title = extract_fm_field(&src, "title").unwrap_or_else(|| slug.clone());
|
||||
let material = task_near_material(&src, &slug);
|
||||
let score = cosine_tf(&qv, &char_bigram_tf(&canonicalize_goal_str(&material)));
|
||||
if score > 0.0 {
|
||||
out.push(NearCand {
|
||||
slug,
|
||||
title,
|
||||
score,
|
||||
});
|
||||
}
|
||||
}
|
||||
out.sort_by(|a, b| {
|
||||
b.score
|
||||
.partial_cmp(&a.score)
|
||||
.unwrap_or(std::cmp::Ordering::Equal)
|
||||
.then_with(|| a.slug.cmp(&b.slug))
|
||||
});
|
||||
out.truncate(limit.max(1));
|
||||
out
|
||||
}
|
||||
|
||||
fn lookup_hit_payload(
|
||||
kb: &str,
|
||||
root: &Path,
|
||||
@@ -613,6 +766,7 @@ fn lookup_hit_payload(
|
||||
slug: &str,
|
||||
match_kind: &str,
|
||||
query_sig: &str,
|
||||
score: Option<f64>,
|
||||
) -> Result<Value, String> {
|
||||
let skill_src =
|
||||
fs::read_to_string(skill_path).map_err(|e| format!("kb_lookup read skill: {e}"))?;
|
||||
@@ -645,7 +799,7 @@ fn lookup_hit_payload(
|
||||
}
|
||||
}
|
||||
|
||||
Ok(json!({
|
||||
let mut out = json!({
|
||||
"hit": true,
|
||||
"match": match_kind,
|
||||
"sig": skill_sig,
|
||||
@@ -658,7 +812,13 @@ fn lookup_hit_payload(
|
||||
"resource": kb_rel(kb, &format!("resources/{slug}.mq.md")),
|
||||
"task": kb_rel(kb, &format!("concepts/tasks/{slug}.md")),
|
||||
"skill": kb_rel(kb, &format!("concepts/skills/{slug}.md")),
|
||||
}))
|
||||
});
|
||||
if let Some(s) = score {
|
||||
if let Some(obj) = out.as_object_mut() {
|
||||
obj.insert("score".into(), json!((s * 1000.0).round() / 1000.0));
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn resolve_skill_paths_by_slug(
|
||||
@@ -699,7 +859,7 @@ pub fn kb_lookup(args: &Value) -> Result<Value, String> {
|
||||
.and_then(|v| v.as_str())
|
||||
.map(|g| goal_sig_hex(&material_for(g, tools)))
|
||||
.unwrap_or_default();
|
||||
return lookup_hit_payload(kb, &root, &skill_path, &slug, "soft", &query_sig);
|
||||
return lookup_hit_payload(kb, &root, &skill_path, &slug, "soft", &query_sig, None);
|
||||
}
|
||||
|
||||
let goal = arg_text(args, "goal")?;
|
||||
@@ -707,6 +867,7 @@ pub fn kb_lookup(args: &Value) -> Result<Value, String> {
|
||||
let query_sig = goal_sig_hex(&material_for(goal, tools));
|
||||
let slug_hint = goal_slug_str(goal);
|
||||
let mut match_kind = "exact";
|
||||
let mut near_score_hit: Option<f64> = None;
|
||||
let resolved = resolve_skill_paths(&root, &query_sig, &slug_hint)
|
||||
.or_else(|| {
|
||||
match_kind = "alias";
|
||||
@@ -722,12 +883,72 @@ pub fn kb_lookup(args: &Value) -> Result<Value, String> {
|
||||
let slug2 = goal_slug_str(&canon);
|
||||
resolve_skill_paths(&root, &sig2, &slug2)
|
||||
.or_else(|| resolve_skill_paths_by_alias(&root, &canon))
|
||||
})
|
||||
.or_else(|| {
|
||||
if !arg_bool(args, "near_match", false) {
|
||||
return None;
|
||||
}
|
||||
let threshold = arg_f64(args, "near_threshold", 0.78);
|
||||
let ranked = rank_near_tasks(&root, goal, 8);
|
||||
let best = ranked.first()?;
|
||||
if best.score < threshold {
|
||||
return None;
|
||||
}
|
||||
match_kind = "near";
|
||||
near_score_hit = Some(best.score);
|
||||
resolve_skill_paths_by_slug(&root, &best.slug)
|
||||
});
|
||||
let Some((_task, skill_path, _res, slug)) = resolved else {
|
||||
return Ok(Value::Null);
|
||||
};
|
||||
|
||||
lookup_hit_payload(kb, &root, &skill_path, &slug, match_kind, &query_sig)
|
||||
lookup_hit_payload(
|
||||
kb,
|
||||
&root,
|
||||
&skill_path,
|
||||
&slug,
|
||||
match_kind,
|
||||
&query_sig,
|
||||
near_score_hit,
|
||||
)
|
||||
}
|
||||
|
||||
pub fn kb_near_match(args: &Value) -> Result<Value, String> {
|
||||
let kb = arg_text(args, "kb_dir")?;
|
||||
let goal = arg_text(args, "goal")?;
|
||||
let limit = match args.get("limit") {
|
||||
None | Some(Value::Null) => 8usize,
|
||||
Some(Value::Number(n)) => n.as_u64().unwrap_or(8) as usize,
|
||||
Some(Value::String(s)) => s.parse().unwrap_or(8),
|
||||
_ => 8,
|
||||
}
|
||||
.clamp(1, 40);
|
||||
let root = resolve_path(kb);
|
||||
let ranked = rank_near_tasks(&root, goal, limit);
|
||||
let candidates: Vec<Value> = ranked
|
||||
.iter()
|
||||
.map(|c| {
|
||||
json!({
|
||||
"slug": c.slug,
|
||||
"title": c.title,
|
||||
"score": (c.score * 1000.0).round() / 1000.0,
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
let best = ranked.first().map(|c| {
|
||||
json!({
|
||||
"slug": c.slug,
|
||||
"title": c.title,
|
||||
"score": (c.score * 1000.0).round() / 1000.0,
|
||||
})
|
||||
});
|
||||
let score = ranked.first().map(|c| (c.score * 1000.0).round() / 1000.0);
|
||||
Ok(json!({
|
||||
"candidates": candidates,
|
||||
"count": ranked.len() as i64,
|
||||
"best": best,
|
||||
"score": score,
|
||||
}))
|
||||
}
|
||||
|
||||
pub fn kb_list_tasks(args: &Value) -> Result<Value, String> {
|
||||
@@ -1102,4 +1323,27 @@ mod tests {
|
||||
let c2 = canonicalize_goal_str("你是一个智能体,帮我规划明天的行程?");
|
||||
assert_eq!(c2, "帮我规划明天的行程");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn near_score_trip_near_dupes_high() {
|
||||
let a = "帮我规划明天行程";
|
||||
let b = "帮我规划明天的行程";
|
||||
let s = near_score(a, b);
|
||||
assert!(s >= 0.78, "expected high near score, got {s}");
|
||||
let c = "帮我完成明天的行程规划";
|
||||
let s2 = near_score(b, c);
|
||||
let unrelated = near_score(b, "明天做什么好");
|
||||
assert!(
|
||||
s2 > unrelated,
|
||||
"paraphrase should beat unrelated: paraphrase={s2} unrelated={unrelated}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn near_score_unrelated_low() {
|
||||
let a = "帮我规划明天的行程";
|
||||
let b = "明天做什么好";
|
||||
let s = near_score(a, b);
|
||||
assert!(s < 0.78, "expected below default threshold, got {s}");
|
||||
}
|
||||
}
|
||||
@@ -879,6 +879,7 @@ kb_ffi!(agent_goal_sig, kb::goal_sig, "agent_goal_sig");
|
||||
kb_ffi!(agent_goal_slug, kb::goal_slug, "agent_goal_slug");
|
||||
kb_ffi!(agent_kb_canonicalize, kb::kb_canonicalize, "agent_kb_canonicalize");
|
||||
kb_ffi!(agent_kb_lookup, kb::kb_lookup, "agent_kb_lookup");
|
||||
kb_ffi!(agent_kb_near_match, kb::kb_near_match, "agent_kb_near_match");
|
||||
kb_ffi!(agent_kb_list_tasks, kb::kb_list_tasks, "agent_kb_list_tasks");
|
||||
kb_ffi!(agent_kb_add_alias, kb::kb_add_alias, "agent_kb_add_alias");
|
||||
kb_ffi!(agent_kb_promote, kb::kb_promote, "agent_kb_promote");
|
||||
@@ -971,6 +972,9 @@ pub unsafe extern "C" fn marqdo_plugin_init(host: *const MarqdoHostApi) -> c_int
|
||||
if register(host, "agent_kb_lookup", "kb_dir,goal", agent_kb_lookup) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_kb_near_match", "kb_dir,goal", agent_kb_near_match) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_kb_list_tasks", "kb_dir", agent_kb_list_tasks) != 0 {
|
||||
return 1;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
---
|
||||
title: OKF agent-kb near_match (offline)
|
||||
description: Promote trip resource; near-dupe goals hit match=near; unrelated misses.
|
||||
> ext/ai/agent.mq.md
|
||||
> lib/fs.mq.md
|
||||
> lib/json.mq.md
|
||||
> lib/plugin.mq.md
|
||||
> lib/sys.mq.md
|
||||
---
|
||||
|
||||
# main
|
||||
|
||||
*`p` = > plugin.native_path name=agent *
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
2. *
|
||||
> print text=no-agent-plugin
|
||||
> sys.exit code=1
|
||||
|
||||
*`kb` = .marqdo/agent-kb-near-test *
|
||||
*`wb` = .marqdo/agent-runs/near-trip.mq.md *
|
||||
*`goal` = 帮我规划明天的行程 *
|
||||
*`body` = "---\ntitle: trip\n---\n\n# main\n\n**trip-ok**\n" *
|
||||
|
||||
> fs.write_text path=`wb` text=`body`
|
||||
*`prom` = > agent_kb_promote kb_dir=`kb` goal=`goal` workbook=`wb` *
|
||||
*`okp` = > json.get value=`prom` key=promoted *
|
||||
1. `okp`
|
||||
> print text=promoted
|
||||
2. *
|
||||
> print text=promote-fail
|
||||
|
||||
*`slug` = > json.get value=`prom` key=slug *
|
||||
|
||||
*`g1` = 帮我规划明天行程 *
|
||||
*`hit1` = > agent_kb_lookup kb_dir=`kb` goal=`g1` near_match=True near_threshold=0.78 *
|
||||
1. `hit1`
|
||||
> print text=near-ok
|
||||
2. *
|
||||
> print text=near-miss
|
||||
*`mk1` = > json.get value=`hit1` key=match *
|
||||
> print text=`mk1`
|
||||
*`slug1` = > json.get value=`hit1` key=slug *
|
||||
1. `slug` == `slug1`
|
||||
> print text=same-slug
|
||||
2. *
|
||||
> print text=slug-drift
|
||||
|
||||
*`off` = > agent_kb_lookup kb_dir=`kb` goal=`g1` near_match=False *
|
||||
1. `off`
|
||||
> print text=off-hit
|
||||
2. *
|
||||
> print text=off-miss
|
||||
|
||||
*`g2` = 明天做什么好 *
|
||||
*`hit2` = > agent_kb_lookup kb_dir=`kb` goal=`g2` near_match=True near_threshold=0.78 *
|
||||
1. `hit2`
|
||||
> print text=neg-hit
|
||||
2. *
|
||||
> print text=neg-miss
|
||||
|
||||
*`ranked` = > agent_kb_near_match kb_dir=`kb` goal=`g1` *
|
||||
*`best` = > json.get value=`ranked` key=best *
|
||||
*`bs` = > json.get value=`best` key=slug *
|
||||
1. `bs` == `slug`
|
||||
> print text=rank-ok
|
||||
2. *
|
||||
> print text=rank-bad
|
||||
|
||||
*`tasks` = > json.get value=`ranked` key=candidates *
|
||||
*`prompt` = > agent.build_soft_match_prompt goal=`g1` tasks=`tasks` *
|
||||
*`has` = > split value=`prompt` sep=score= *
|
||||
*`hn` = > len value=`has` *
|
||||
1. `hn` > 1
|
||||
> print text=prompt-score-ok
|
||||
2. *
|
||||
> print text=prompt-score-bad
|
||||
@@ -1432,6 +1432,27 @@ list-ok",
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ext_agent_kb_near() {
|
||||
let status = Command::new("cargo")
|
||||
.args(["build", "-p", "marqdo_plugin_agent"])
|
||||
.current_dir(env!("CARGO_MANIFEST_DIR"))
|
||||
.status()
|
||||
.expect("build agent plugin");
|
||||
assert!(status.success(), "failed to build marqdo_plugin_agent");
|
||||
assert_out(
|
||||
"tests/ext/agent-kb-near.mq.md",
|
||||
"promoted
|
||||
near-ok
|
||||
near
|
||||
same-slug
|
||||
off-miss
|
||||
neg-miss
|
||||
rank-ok
|
||||
prompt-score-ok",
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ext_agent_soft_match() {
|
||||
assert_out(
|
||||
|
||||
Reference in new issue
Block a user