feat(labels): M9 两级词表检索 — labels CLI + MLSP labels.search

第一级(内置,数据常量):六组关键词 —— 未完成/假数据/示例/实验/废弃/待校验,
pending = 未完成 ∪ 假数据。作用是默认查询、LLM 范例、跨项目可比。
第二级(参数):--grep 替换,--also-builtin 并集,--group 选组。
**词表是数据不是语法**:扩充它不改任何解析规则。

CLI:
  marqdo labels .                              # pending 口径
  marqdo labels . --grep "联调"                 # 自定义替换
  marqdo labels . --grep "联调" --also-builtin   # 并集
  marqdo labels . --group 示例,实验
  marqdo labels . --count / --json

MLSP:{"method":"labels.search","params":{path|source, grep?, also_builtin?, group?}}
→ {"ok":true,"words":[…],"count":N,"hits":[{line,unit,label,words,has_payload}]}
与 CLI 共用同一份 label::search。

检索语义 v1:子串匹配(CJK 无词边界)、多词 OR、英文大小写不敏感;
误判只影响计数,输出带描述原文可人工判断;不做否定语法(留 v1.1)。

测试:4 个集成(pending=2 / grep 替换=1 / also-builtin 并集=3 / MLSP 返回 hits);
全量 cargo test --release:587 passed / 0 failed(M4 后 583 + 4)。

实测:
  pending → 2 命中("待开发"、"模拟"两处)
  --grep 联调 → 1 命中
  --grep 联调 --also-builtin → 3 命中

文档:doc/design/change-m9-label-search.md
This commit is contained in:
cflmy committed 2026-10-22 15:16:39 +08:00
1 parent 6dd0ce47b1
commit fd0cfcb566
6 files changed
+468

No files matched your search

+113
View File
@@ -0,0 +1,113 @@
# 变更记录:M9 两级词表检索(`labels` CLI + MLSP `labels.search`)
| | |
|---|---|
| 状态 | **Implemented** |
| 日期 | 2026-10-22 |
| 里程碑 | M9([occlusion-description-and-search.md](occlusion-description-and-search.md) §3 · [occlusion-syntax.md](occlusion-syntax.md) §13) |
| 改动文件 | `src/label.rs` · `src/main.rs` · `src/mlsp.rs` · `tests/occlusion.rs` |
| 影响面 | 新增 CLI 子命令与 MLSP 方法(**不改语法**) |
---
## 1. 两级词表
| 级 | 在哪 | 作用 |
|----|------|------|
| **第一级:内置** | `label::BUILTIN_GROUPS`(数据常量) | 默认查询 + LLM 范例 + 跨项目可比 |
| **第二级:参数** | `--grep` / `grep` | 替换或并集,项目自己的说法照样能搜 |
内置六组:`未完成`(待开发/未开发/开发中/未实现/待实现/未接/接口未接/TODO)、
`假数据`(模拟/假数据/桩/占位/stub/mock)、`示例`、`实验`、`废弃`、`待校验`。
`pending` = `未完成 ∪ 假数据`。
**词表是数据不是语法**:扩充它不改任何解析规则。
---
## 2. CLI
```bash
marqdo labels . # 默认:pending 口径(内置两组)
marqdo labels . --grep "联调" # 替换为自定义词
marqdo labels . --grep "联调" --also-builtin # 自定义 ∪ 内置
marqdo labels . --group 示例,实验 # 选内置组
marqdo labels . --count # 只给数字(CI/棘轮用)
marqdo labels . --json # 一个 JSON 数组
```
输出(人类):
```
a.mq.md:3 [main] 按关键词检索,接口未接,待开发 ← 待开发,未接,接口未接
a.mq.md:4 [main] 客户留存率,模拟 ← 模拟
label search: 2 命中(1 文件,词=[…])
```
`--json`:
```json
[{"file":"a.mq.md","line":3,"unit":"main","label":"…","words":["待开发"],"has_payload":true}]
```
---
## 3. MLSP
```json
{"id":1,"method":"labels.search",
"params":{"source":"…"|"path":"…","grep":["联调"],"also_builtin":true}}
```
返回:
```json
{"ok":true,"words":[…],"count":2,"hits":[{"line":3,"unit":"main","label":"…",
"words":["待开发"],"has_payload":true}]}
```
语义与 CLI 完全一致(同一份 `label::search`)。
---
## 4. 检索语义(v1)
| 项 | 规则 |
|----|------|
| 匹配 | **子串**(CJK 无词边界,子串最稳) |
| 多词 | OR |
| 大小写 | 英文不敏感(`TODO`/`todo` 等价) |
| 误判 | 必然存在(`[不属于待开发范围]` 会命中)——只影响计数,输出带描述原文可人工判断 |
不做否定语法;留作 v1.1 可选项。
---
## 5. 测试
| 层 | 内容 | 结果 |
|----|------|------|
| 集成(4) | pending 计数=2;`--grep` 替换=1;`--also-builtin` 并集=3;MLSP `labels.search` 返回 hits | ✅ |
| 全量 | `cargo test --release` | **587 passed / 0 failed**(M4 后 583 + 4) |
---
## 6. 边界
| 项 | 说明 |
|----|------|
| 只在 occlusion 模式产生命中 | classic 模式下没有 `Expr::Occluded`;M7 翻转默认后自然可用 |
| 解析失败的文件跳过 | 那是 `check` 的职责,检索不应因一个坏文件整体失败 |
| 不做棘轮 | `--baseline` / `--fail-on-increase` 留给 M8(门禁) |
---
## 7. 变更清单
| 文件 | 变更 |
|------|------|
| `src/label.rs` | `WordGroup` / `BUILTIN_GROUPS` / `PENDING_GROUPS` / `group_words` / `parse_words` / `matched_words` / `Hit` / `search` |
| `src/main.rs` | `labels` 子命令 + `collect_mq_files` 递归扫描 |
| `src/mlsp.rs` | `labels.search` 方法 + 分发 |
| `tests/occlusion.rs` + `tests/occlusion/labels.mq.md` | 4 个集成测试 |
| `doc/design/change-m9-label-search.md` | 本文件 |
+101
View File
@@ -336,3 +336,104 @@ mod tests {
assert!(refs("空 `` 引号").is_empty());
}
}
// ---------------------------------------------------------------------------
// M9: 两级词表检索(built-in groups = 默认/范例;`--grep` = 调用方自定义)
// ---------------------------------------------------------------------------
/// One built-in keyword group. `<TAB>` M9
///
/// The table is **data**, not syntax: it seeds the default query
/// (`marqdo labels pending`) and gives LLMs a stable vocabulary; any project can
/// override or extend it at the call site with `--grep`.
pub struct WordGroup {
pub name: &'static str,
pub words: &'static [&'static str],
}
/// Built-in groups (v1). Extending this table never changes the grammar.
pub const BUILTIN_GROUPS: &[WordGroup] = &[
WordGroup {
name: "未完成",
words: &[
"待开发", "未开发", "开发中", "未实现", "待实现", "未接", "接口未接", "TODO",
],
},
WordGroup {
name: "假数据",
words: &["模拟", "假数据", "桩", "占位", "stub", "mock"],
},
WordGroup {
name: "示例",
words: &["示例", "样例", "demo"],
},
WordGroup {
name: "实验",
words: &["实验", "试验", "experimental"],
},
WordGroup {
name: "废弃",
words: &["废弃", "弃用", "deprecated"],
},
WordGroup {
name: "待校验",
words: &["待校验", "未验证", "待确认"],
},
];
/// Groups that count as "pending" (default query for `labels pending`).
pub const PENDING_GROUPS: &[&str] = &["未完成", "假数据"];
/// Expand group names into their words; unknown names are ignored.
pub fn group_words(names: &[&str]) -> Vec<String> {
let mut out = Vec::new();
for g in BUILTIN_GROUPS {
if names.iter().any(|n| *n == g.name) {
out.extend(g.words.iter().map(|w| w.to_string()));
}
}
out
}
/// Split a `--grep` value (`a,b` / `a,b`) into words.
pub fn parse_words(spec: &str) -> Vec<String> {
spec.split([',', ','])
.map(|w| w.trim())
.filter(|w| !w.is_empty())
.map(|w| w.to_string())
.collect()
}
/// Words that hit `label` (substring, ASCII case-insensitive). `<TAB>` M9
pub fn matched_words(label: &str, words: &[String]) -> Vec<String> {
let hay = label.to_lowercase();
let mut out = Vec::new();
for w in words {
if w.is_empty() {
continue;
}
if hay.contains(&w.to_lowercase()) {
out.push(w.clone());
}
}
out
}
/// One search hit: where, what the label says, which words matched.
#[derive(Debug, Clone)]
pub struct Hit {
pub site: OcclusionSite,
pub words: Vec<String>,
}
/// Search every occlusion whose label matches any of `words`.
pub fn search(module: &Module, words: &[String]) -> Vec<Hit> {
let mut out = Vec::new();
for site in collect(module) {
let words = matched_words(&site.label, words);
if !words.is_empty() {
out.push(Hit { site, words });
}
}
out
}
+127
View File
@@ -97,6 +97,32 @@ enum Commands {
#[arg(long)]
json: bool,
},
/// Occlusion label queries: pending stubs (built-in words) or keyword search
Labels {
/// File or directory (recursively scans `*.mq.md`)
#[arg(value_name = "PATH", default_value = ".")]
path: PathBuf,
/// Keywords (comma separated). Replaces the built-in pending set.
#[arg(long, value_name = "WORDS")]
grep: Option<String>,
/// Union with the built-in groups instead of replacing them
#[arg(long)]
also_builtin: bool,
/// Built-in group names, comma separated (default: pending = 未完成,假数据)
#[arg(long, value_name = "GROUPS")]
group: Option<String>,
/// One JSON array on stdout
#[arg(long)]
json: bool,
/// Print only the hit count
#[arg(long)]
count: bool,
},
/// MLSP for AI: line-delimited JSON on stdio (locate / syntax / validate / repair_targets / schema)
Mlsp,
/// Browse `.mq.md` structure + execution (live server or static output)
@@ -426,6 +452,32 @@ fn main() -> ExitCode {
}
}
/// Recursively collect `*.mq.md` under `path` (file → itself).
fn collect_mq_files(path: &std::path::Path) -> Result<Vec<std::path::PathBuf>> {
fn walk(dir: &std::path::Path, out: &mut Vec<std::path::PathBuf>) {
let Ok(rd) = std::fs::read_dir(dir) else { return };
for e in rd.flatten() {
let p = e.path();
let name = p.file_name().and_then(|n| n.to_str()).unwrap_or("");
if p.is_dir() {
if matches!(name, "target" | "dist" | "node_modules" | ".git" | ".marqdo") {
continue;
}
walk(&p, out);
} else if name.ends_with(".mq.md") {
out.push(p);
}
}
}
if path.is_file() {
return Ok(vec![path.to_path_buf()]);
}
let mut out = Vec::new();
walk(path, &mut out);
out.sort();
Ok(out)
}
fn try_main(cli: Cli) -> Result<i32> {
match cli.command {
Commands::Run {
@@ -507,6 +559,81 @@ fn try_main(cli: Cli) -> Result<i32> {
}
Ok(errors.min(1))
}
Commands::Labels {
path,
grep,
also_builtin,
group,
json,
count,
} => {
let mut words: Vec<String> = match &grep {
Some(spec) => marqdo::label::parse_words(spec),
None => {
let groups: Vec<&str> = match &group {
Some(g) => g.split(',').map(|s| s.trim()).filter(|s| !s.is_empty()).collect(),
None => marqdo::label::PENDING_GROUPS.to_vec(),
};
marqdo::label::group_words(&groups)
}
};
if also_builtin {
let groups: Vec<&str> = match &group {
Some(g) => g.split(',').map(|s| s.trim()).filter(|s| !s.is_empty()).collect(),
None => marqdo::label::PENDING_GROUPS.to_vec(),
};
for w in marqdo::label::group_words(&groups) {
if !words.contains(&w) {
words.push(w);
}
}
}
let files = collect_mq_files(&path)?;
let mut rows: Vec<(String, u32, String, String, Vec<String>, bool)> = Vec::new();
for f in &files {
let module = match marqdo::load::load_module(f) {
Ok(m) => m,
Err(_) => continue, // unparsable files are `check`'s problem, not ours
};
for hit in marqdo::label::search(&module, &words) {
rows.push((
f.display().to_string(),
hit.site.line,
hit.site.unit.clone(),
hit.site.label.clone(),
hit.words.clone(),
hit.site.has_payload,
));
}
}
if json {
let arr: Vec<serde_json::Value> = rows
.iter()
.map(|(file, line, unit, label, words, has_payload)| {
serde_json::json!({
"file": file, "line": line, "unit": unit,
"label": label, "words": words, "has_payload": has_payload,
})
})
.collect();
println!("{}", serde_json::Value::Array(arr));
} else if count {
println!("{}", rows.len());
} else {
for (file, line, unit, label, words, _) in &rows {
println!("{file}:{line} [{unit}] {label} ← {}", words.join(","));
}
println!(
"label search: {} 命中({} 文件,词={:?})",
rows.len(),
files.len(),
words
);
}
Ok(0)
}
Commands::Mlsp => {
use std::io::{BufRead, Write};
let stdin = std::io::stdin();
+47
View File
@@ -57,6 +57,8 @@ fn dispatch(req: &Value) -> Result<Value, Value> {
Some("repair_targets") => repair_targets(&params),
Some("repair_apply") => repair_apply(&params),
Some("schema") => schema(&params),
// `<TAB>` M9: occlusion label search (two-tier vocabulary).
Some("labels.search") => labels_search(&params),
other => Err(err(
"mlsp.unknown_method",
format!("未知方法 {other:?};可用:locate / syntax / validate / repair_targets / repair_apply / schema"),
@@ -331,6 +333,51 @@ fn schema(params: &Value) -> Result<Value, Value> {
// ---------- 公共小工具 ----------
/// `labels.search` — occlusion label query. `<TAB>` M9
///
/// params: `{ source | path, grep?: [..] | "a,b", also_builtin?: bool,
/// group?: "未完成,假数据" }`
fn labels_search(params: &Value) -> Result<Value, Value> {
let (module, _path) = module_from_params(params)?;
let mut words: Vec<String> = match params.get("grep") {
Some(Value::String(s)) => crate::label::parse_words(s),
Some(Value::Array(a)) => a
.iter()
.filter_map(|v| v.as_str())
.map(|s| s.to_string())
.collect(),
_ => Vec::new(),
};
let groups: Vec<String> = match params.get("group").and_then(|g| g.as_str()) {
Some(spec) => crate::label::parse_words(spec),
None => crate::label::PENDING_GROUPS
.iter()
.map(|s| s.to_string())
.collect(),
};
let group_refs: Vec<&str> = groups.iter().map(|s| s.as_str()).collect();
if words.is_empty() || params.get("also_builtin").and_then(|b| b.as_bool()) == Some(true) {
for w in crate::label::group_words(&group_refs) {
if !words.contains(&w) {
words.push(w);
}
}
}
let hits: Vec<Value> = crate::label::search(&module, &words)
.into_iter()
.map(|h| {
json!({
"line": h.site.line,
"unit": h.site.unit,
"label": h.site.label,
"words": h.words,
"has_payload": h.site.has_payload,
})
})
.collect();
Ok(json!({"ok": true, "words": words, "count": hits.len(), "hits": hits}))
}
fn module_from_params(params: &Value) -> Result<(Module, Option<PathBuf>), Value> {
if let Some(src) = params.get("source").and_then(|s| s.as_str()) {
let module = load::load_module_from_source(src)
+69
View File
@@ -99,3 +99,72 @@ fn label_refs_that_resolve_are_fine() {
assert_eq!(code, 0, "text={text}");
assert!(!text.contains("不存在的实体"), "text={text}");
}
// ---- M9: two-tier label search (`labels` CLI + MLSP `labels.search`) ----
fn labels(args: &[&str]) -> (i32, String) {
let out = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.arg("labels")
.args(args)
.env("MARQDO_MARKUP", "occlusion")
.output()
.expect("run marqdo labels");
(
out.status.code().unwrap_or(1),
format!(
"{}{}",
String::from_utf8_lossy(&out.stdout),
String::from_utf8_lossy(&out.stderr)
),
)
}
#[test]
fn labels_pending_uses_builtin_groups() {
let (code, text) = labels(&["tests/occlusion/labels.mq.md", "--count"]);
assert_eq!(code, 0, "text={text}");
assert_eq!(text.trim(), "2", "expected 待开发 + 模拟: {text}");
}
#[test]
fn labels_grep_replaces_builtin() {
let (code, text) = labels(&["tests/occlusion/labels.mq.md", "--grep", "联调", "--count"]);
assert_eq!(code, 0, "text={text}");
assert_eq!(text.trim(), "1", "text={text}");
}
#[test]
fn labels_grep_with_builtin_unions() {
let (code, text) = labels(&[
"tests/occlusion/labels.mq.md",
"--grep",
"联调",
"--also-builtin",
"--count",
]);
assert_eq!(code, 0, "text={text}");
assert_eq!(text.trim(), "3", "text={text}");
}
#[test]
fn mlsp_labels_search_returns_hits() {
use std::io::Write;
let mut child = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.arg("mlsp")
.env("MARQDO_MARKUP", "occlusion")
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped())
.spawn()
.expect("spawn mlsp");
let req = r#"{"id":1,"method":"labels.search","params":{"path":"tests/occlusion/labels.mq.md","also_builtin":true}}"#;
child
.stdin
.as_mut()
.unwrap()
.write_all(format!("{req}\n").as_bytes())
.unwrap();
let out = child.wait_with_output().unwrap();
let text = String::from_utf8_lossy(&out.stdout).into_owned();
assert!(text.contains("\"count\":2"), "text={text}");
assert!(text.contains("客户留存率"), "text={text}");
}
+11
View File
@@ -0,0 +1,11 @@
---
title: label search fixture
---
# main
**x = [按关键词检索,接口未接,待开发](None)**
**y = [客户留存率,模拟](0.0)**
**z = [导出报表,示例](1)**
**w = [接口还在联调,先返回空](0)**
> print text=`x`