feat(labels): M9 两级词表检索 — labels CLI + MLSP labels.search
第一级(内置,数据常量):六组关键词 —— 未完成/假数据/示例/实验/废弃/待校验,
pending = 未完成 ∪ 假数据。作用是默认查询、LLM 范例、跨项目可比。
第二级(参数):--grep 替换,--also-builtin 并集,--group 选组。
**词表是数据不是语法**:扩充它不改任何解析规则。
CLI:
marqdo labels . # pending 口径
marqdo labels . --grep "联调" # 自定义替换
marqdo labels . --grep "联调" --also-builtin # 并集
marqdo labels . --group 示例,实验
marqdo labels . --count / --json
MLSP:{"method":"labels.search","params":{path|source, grep?, also_builtin?, group?}}
→ {"ok":true,"words":[…],"count":N,"hits":[{line,unit,label,words,has_payload}]}
与 CLI 共用同一份 label::search。
检索语义 v1:子串匹配(CJK 无词边界)、多词 OR、英文大小写不敏感;
误判只影响计数,输出带描述原文可人工判断;不做否定语法(留 v1.1)。
测试:4 个集成(pending=2 / grep 替换=1 / also-builtin 并集=3 / MLSP 返回 hits);
全量 cargo test --release:587 passed / 0 failed(M4 后 583 + 4)。
实测:
pending → 2 命中("待开发"、"模拟"两处)
--grep 联调 → 1 命中
--grep 联调 --also-builtin → 3 命中
文档:doc/design/change-m9-label-search.md
This commit is contained in:
1 parent
6dd0ce47b1
commit
fd0cfcb566
6 files changed
+468
No files matched your search
@@ -0,0 +1,113 @@
|
||||
# 变更记录:M9 两级词表检索(`labels` CLI + MLSP `labels.search`)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Implemented** |
|
||||
| 日期 | 2026-10-22 |
|
||||
| 里程碑 | M9([occlusion-description-and-search.md](occlusion-description-and-search.md) §3 · [occlusion-syntax.md](occlusion-syntax.md) §13) |
|
||||
| 改动文件 | `src/label.rs` · `src/main.rs` · `src/mlsp.rs` · `tests/occlusion.rs` |
|
||||
| 影响面 | 新增 CLI 子命令与 MLSP 方法(**不改语法**) |
|
||||
|
||||
---
|
||||
|
||||
## 1. 两级词表
|
||||
|
||||
| 级 | 在哪 | 作用 |
|
||||
|----|------|------|
|
||||
| **第一级:内置** | `label::BUILTIN_GROUPS`(数据常量) | 默认查询 + LLM 范例 + 跨项目可比 |
|
||||
| **第二级:参数** | `--grep` / `grep` | 替换或并集,项目自己的说法照样能搜 |
|
||||
|
||||
内置六组:`未完成`(待开发/未开发/开发中/未实现/待实现/未接/接口未接/TODO)、
|
||||
`假数据`(模拟/假数据/桩/占位/stub/mock)、`示例`、`实验`、`废弃`、`待校验`。
|
||||
`pending` = `未完成 ∪ 假数据`。
|
||||
|
||||
**词表是数据不是语法**:扩充它不改任何解析规则。
|
||||
|
||||
---
|
||||
|
||||
## 2. CLI
|
||||
|
||||
```bash
|
||||
marqdo labels . # 默认:pending 口径(内置两组)
|
||||
marqdo labels . --grep "联调" # 替换为自定义词
|
||||
marqdo labels . --grep "联调" --also-builtin # 自定义 ∪ 内置
|
||||
marqdo labels . --group 示例,实验 # 选内置组
|
||||
marqdo labels . --count # 只给数字(CI/棘轮用)
|
||||
marqdo labels . --json # 一个 JSON 数组
|
||||
```
|
||||
|
||||
输出(人类):
|
||||
|
||||
```
|
||||
a.mq.md:3 [main] 按关键词检索,接口未接,待开发 ← 待开发,未接,接口未接
|
||||
a.mq.md:4 [main] 客户留存率,模拟 ← 模拟
|
||||
label search: 2 命中(1 文件,词=[…])
|
||||
```
|
||||
|
||||
`--json`:
|
||||
|
||||
```json
|
||||
[{"file":"a.mq.md","line":3,"unit":"main","label":"…","words":["待开发"],"has_payload":true}]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. MLSP
|
||||
|
||||
```json
|
||||
{"id":1,"method":"labels.search",
|
||||
"params":{"source":"…"|"path":"…","grep":["联调"],"also_builtin":true}}
|
||||
```
|
||||
|
||||
返回:
|
||||
|
||||
```json
|
||||
{"ok":true,"words":[…],"count":2,"hits":[{"line":3,"unit":"main","label":"…",
|
||||
"words":["待开发"],"has_payload":true}]}
|
||||
```
|
||||
|
||||
语义与 CLI 完全一致(同一份 `label::search`)。
|
||||
|
||||
---
|
||||
|
||||
## 4. 检索语义(v1)
|
||||
|
||||
| 项 | 规则 |
|
||||
|----|------|
|
||||
| 匹配 | **子串**(CJK 无词边界,子串最稳) |
|
||||
| 多词 | OR |
|
||||
| 大小写 | 英文不敏感(`TODO`/`todo` 等价) |
|
||||
| 误判 | 必然存在(`[不属于待开发范围]` 会命中)——只影响计数,输出带描述原文可人工判断 |
|
||||
|
||||
不做否定语法;留作 v1.1 可选项。
|
||||
|
||||
---
|
||||
|
||||
## 5. 测试
|
||||
|
||||
| 层 | 内容 | 结果 |
|
||||
|----|------|------|
|
||||
| 集成(4) | pending 计数=2;`--grep` 替换=1;`--also-builtin` 并集=3;MLSP `labels.search` 返回 hits | ✅ |
|
||||
| 全量 | `cargo test --release` | **587 passed / 0 failed**(M4 后 583 + 4) |
|
||||
|
||||
---
|
||||
|
||||
## 6. 边界
|
||||
|
||||
| 项 | 说明 |
|
||||
|----|------|
|
||||
| 只在 occlusion 模式产生命中 | classic 模式下没有 `Expr::Occluded`;M7 翻转默认后自然可用 |
|
||||
| 解析失败的文件跳过 | 那是 `check` 的职责,检索不应因一个坏文件整体失败 |
|
||||
| 不做棘轮 | `--baseline` / `--fail-on-increase` 留给 M8(门禁) |
|
||||
|
||||
---
|
||||
|
||||
## 7. 变更清单
|
||||
|
||||
| 文件 | 变更 |
|
||||
|------|------|
|
||||
| `src/label.rs` | `WordGroup` / `BUILTIN_GROUPS` / `PENDING_GROUPS` / `group_words` / `parse_words` / `matched_words` / `Hit` / `search` |
|
||||
| `src/main.rs` | `labels` 子命令 + `collect_mq_files` 递归扫描 |
|
||||
| `src/mlsp.rs` | `labels.search` 方法 + 分发 |
|
||||
| `tests/occlusion.rs` + `tests/occlusion/labels.mq.md` | 4 个集成测试 |
|
||||
| `doc/design/change-m9-label-search.md` | 本文件 |
|
||||
+101
@@ -336,3 +336,104 @@ mod tests {
|
||||
assert!(refs("空 `` 引号").is_empty());
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// M9: 两级词表检索(built-in groups = 默认/范例;`--grep` = 调用方自定义)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// One built-in keyword group. `<TAB>` M9
|
||||
///
|
||||
/// The table is **data**, not syntax: it seeds the default query
|
||||
/// (`marqdo labels pending`) and gives LLMs a stable vocabulary; any project can
|
||||
/// override or extend it at the call site with `--grep`.
|
||||
pub struct WordGroup {
|
||||
pub name: &'static str,
|
||||
pub words: &'static [&'static str],
|
||||
}
|
||||
|
||||
/// Built-in groups (v1). Extending this table never changes the grammar.
|
||||
pub const BUILTIN_GROUPS: &[WordGroup] = &[
|
||||
WordGroup {
|
||||
name: "未完成",
|
||||
words: &[
|
||||
"待开发", "未开发", "开发中", "未实现", "待实现", "未接", "接口未接", "TODO",
|
||||
],
|
||||
},
|
||||
WordGroup {
|
||||
name: "假数据",
|
||||
words: &["模拟", "假数据", "桩", "占位", "stub", "mock"],
|
||||
},
|
||||
WordGroup {
|
||||
name: "示例",
|
||||
words: &["示例", "样例", "demo"],
|
||||
},
|
||||
WordGroup {
|
||||
name: "实验",
|
||||
words: &["实验", "试验", "experimental"],
|
||||
},
|
||||
WordGroup {
|
||||
name: "废弃",
|
||||
words: &["废弃", "弃用", "deprecated"],
|
||||
},
|
||||
WordGroup {
|
||||
name: "待校验",
|
||||
words: &["待校验", "未验证", "待确认"],
|
||||
},
|
||||
];
|
||||
|
||||
/// Groups that count as "pending" (default query for `labels pending`).
|
||||
pub const PENDING_GROUPS: &[&str] = &["未完成", "假数据"];
|
||||
|
||||
/// Expand group names into their words; unknown names are ignored.
|
||||
pub fn group_words(names: &[&str]) -> Vec<String> {
|
||||
let mut out = Vec::new();
|
||||
for g in BUILTIN_GROUPS {
|
||||
if names.iter().any(|n| *n == g.name) {
|
||||
out.extend(g.words.iter().map(|w| w.to_string()));
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Split a `--grep` value (`a,b` / `a,b`) into words.
|
||||
pub fn parse_words(spec: &str) -> Vec<String> {
|
||||
spec.split([',', ','])
|
||||
.map(|w| w.trim())
|
||||
.filter(|w| !w.is_empty())
|
||||
.map(|w| w.to_string())
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Words that hit `label` (substring, ASCII case-insensitive). `<TAB>` M9
|
||||
pub fn matched_words(label: &str, words: &[String]) -> Vec<String> {
|
||||
let hay = label.to_lowercase();
|
||||
let mut out = Vec::new();
|
||||
for w in words {
|
||||
if w.is_empty() {
|
||||
continue;
|
||||
}
|
||||
if hay.contains(&w.to_lowercase()) {
|
||||
out.push(w.clone());
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// One search hit: where, what the label says, which words matched.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Hit {
|
||||
pub site: OcclusionSite,
|
||||
pub words: Vec<String>,
|
||||
}
|
||||
|
||||
/// Search every occlusion whose label matches any of `words`.
|
||||
pub fn search(module: &Module, words: &[String]) -> Vec<Hit> {
|
||||
let mut out = Vec::new();
|
||||
for site in collect(module) {
|
||||
let words = matched_words(&site.label, words);
|
||||
if !words.is_empty() {
|
||||
out.push(Hit { site, words });
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
+127
@@ -97,6 +97,32 @@ enum Commands {
|
||||
#[arg(long)]
|
||||
json: bool,
|
||||
},
|
||||
/// Occlusion label queries: pending stubs (built-in words) or keyword search
|
||||
Labels {
|
||||
/// File or directory (recursively scans `*.mq.md`)
|
||||
#[arg(value_name = "PATH", default_value = ".")]
|
||||
path: PathBuf,
|
||||
|
||||
/// Keywords (comma separated). Replaces the built-in pending set.
|
||||
#[arg(long, value_name = "WORDS")]
|
||||
grep: Option<String>,
|
||||
|
||||
/// Union with the built-in groups instead of replacing them
|
||||
#[arg(long)]
|
||||
also_builtin: bool,
|
||||
|
||||
/// Built-in group names, comma separated (default: pending = 未完成,假数据)
|
||||
#[arg(long, value_name = "GROUPS")]
|
||||
group: Option<String>,
|
||||
|
||||
/// One JSON array on stdout
|
||||
#[arg(long)]
|
||||
json: bool,
|
||||
|
||||
/// Print only the hit count
|
||||
#[arg(long)]
|
||||
count: bool,
|
||||
},
|
||||
/// MLSP for AI: line-delimited JSON on stdio (locate / syntax / validate / repair_targets / schema)
|
||||
Mlsp,
|
||||
/// Browse `.mq.md` structure + execution (live server or static output)
|
||||
@@ -426,6 +452,32 @@ fn main() -> ExitCode {
|
||||
}
|
||||
}
|
||||
|
||||
/// Recursively collect `*.mq.md` under `path` (file → itself).
|
||||
fn collect_mq_files(path: &std::path::Path) -> Result<Vec<std::path::PathBuf>> {
|
||||
fn walk(dir: &std::path::Path, out: &mut Vec<std::path::PathBuf>) {
|
||||
let Ok(rd) = std::fs::read_dir(dir) else { return };
|
||||
for e in rd.flatten() {
|
||||
let p = e.path();
|
||||
let name = p.file_name().and_then(|n| n.to_str()).unwrap_or("");
|
||||
if p.is_dir() {
|
||||
if matches!(name, "target" | "dist" | "node_modules" | ".git" | ".marqdo") {
|
||||
continue;
|
||||
}
|
||||
walk(&p, out);
|
||||
} else if name.ends_with(".mq.md") {
|
||||
out.push(p);
|
||||
}
|
||||
}
|
||||
}
|
||||
if path.is_file() {
|
||||
return Ok(vec![path.to_path_buf()]);
|
||||
}
|
||||
let mut out = Vec::new();
|
||||
walk(path, &mut out);
|
||||
out.sort();
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn try_main(cli: Cli) -> Result<i32> {
|
||||
match cli.command {
|
||||
Commands::Run {
|
||||
@@ -507,6 +559,81 @@ fn try_main(cli: Cli) -> Result<i32> {
|
||||
}
|
||||
Ok(errors.min(1))
|
||||
}
|
||||
Commands::Labels {
|
||||
path,
|
||||
grep,
|
||||
also_builtin,
|
||||
group,
|
||||
json,
|
||||
count,
|
||||
} => {
|
||||
let mut words: Vec<String> = match &grep {
|
||||
Some(spec) => marqdo::label::parse_words(spec),
|
||||
None => {
|
||||
let groups: Vec<&str> = match &group {
|
||||
Some(g) => g.split(',').map(|s| s.trim()).filter(|s| !s.is_empty()).collect(),
|
||||
None => marqdo::label::PENDING_GROUPS.to_vec(),
|
||||
};
|
||||
marqdo::label::group_words(&groups)
|
||||
}
|
||||
};
|
||||
if also_builtin {
|
||||
let groups: Vec<&str> = match &group {
|
||||
Some(g) => g.split(',').map(|s| s.trim()).filter(|s| !s.is_empty()).collect(),
|
||||
None => marqdo::label::PENDING_GROUPS.to_vec(),
|
||||
};
|
||||
for w in marqdo::label::group_words(&groups) {
|
||||
if !words.contains(&w) {
|
||||
words.push(w);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let files = collect_mq_files(&path)?;
|
||||
let mut rows: Vec<(String, u32, String, String, Vec<String>, bool)> = Vec::new();
|
||||
for f in &files {
|
||||
let module = match marqdo::load::load_module(f) {
|
||||
Ok(m) => m,
|
||||
Err(_) => continue, // unparsable files are `check`'s problem, not ours
|
||||
};
|
||||
for hit in marqdo::label::search(&module, &words) {
|
||||
rows.push((
|
||||
f.display().to_string(),
|
||||
hit.site.line,
|
||||
hit.site.unit.clone(),
|
||||
hit.site.label.clone(),
|
||||
hit.words.clone(),
|
||||
hit.site.has_payload,
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
if json {
|
||||
let arr: Vec<serde_json::Value> = rows
|
||||
.iter()
|
||||
.map(|(file, line, unit, label, words, has_payload)| {
|
||||
serde_json::json!({
|
||||
"file": file, "line": line, "unit": unit,
|
||||
"label": label, "words": words, "has_payload": has_payload,
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
println!("{}", serde_json::Value::Array(arr));
|
||||
} else if count {
|
||||
println!("{}", rows.len());
|
||||
} else {
|
||||
for (file, line, unit, label, words, _) in &rows {
|
||||
println!("{file}:{line} [{unit}] {label} ← {}", words.join(","));
|
||||
}
|
||||
println!(
|
||||
"label search: {} 命中({} 文件,词={:?})",
|
||||
rows.len(),
|
||||
files.len(),
|
||||
words
|
||||
);
|
||||
}
|
||||
Ok(0)
|
||||
}
|
||||
Commands::Mlsp => {
|
||||
use std::io::{BufRead, Write};
|
||||
let stdin = std::io::stdin();
|
||||
|
||||
+47
@@ -57,6 +57,8 @@ fn dispatch(req: &Value) -> Result<Value, Value> {
|
||||
Some("repair_targets") => repair_targets(¶ms),
|
||||
Some("repair_apply") => repair_apply(¶ms),
|
||||
Some("schema") => schema(¶ms),
|
||||
// `<TAB>` M9: occlusion label search (two-tier vocabulary).
|
||||
Some("labels.search") => labels_search(¶ms),
|
||||
other => Err(err(
|
||||
"mlsp.unknown_method",
|
||||
format!("未知方法 {other:?};可用:locate / syntax / validate / repair_targets / repair_apply / schema"),
|
||||
@@ -331,6 +333,51 @@ fn schema(params: &Value) -> Result<Value, Value> {
|
||||
|
||||
// ---------- 公共小工具 ----------
|
||||
|
||||
/// `labels.search` — occlusion label query. `<TAB>` M9
|
||||
///
|
||||
/// params: `{ source | path, grep?: [..] | "a,b", also_builtin?: bool,
|
||||
/// group?: "未完成,假数据" }`
|
||||
fn labels_search(params: &Value) -> Result<Value, Value> {
|
||||
let (module, _path) = module_from_params(params)?;
|
||||
let mut words: Vec<String> = match params.get("grep") {
|
||||
Some(Value::String(s)) => crate::label::parse_words(s),
|
||||
Some(Value::Array(a)) => a
|
||||
.iter()
|
||||
.filter_map(|v| v.as_str())
|
||||
.map(|s| s.to_string())
|
||||
.collect(),
|
||||
_ => Vec::new(),
|
||||
};
|
||||
let groups: Vec<String> = match params.get("group").and_then(|g| g.as_str()) {
|
||||
Some(spec) => crate::label::parse_words(spec),
|
||||
None => crate::label::PENDING_GROUPS
|
||||
.iter()
|
||||
.map(|s| s.to_string())
|
||||
.collect(),
|
||||
};
|
||||
let group_refs: Vec<&str> = groups.iter().map(|s| s.as_str()).collect();
|
||||
if words.is_empty() || params.get("also_builtin").and_then(|b| b.as_bool()) == Some(true) {
|
||||
for w in crate::label::group_words(&group_refs) {
|
||||
if !words.contains(&w) {
|
||||
words.push(w);
|
||||
}
|
||||
}
|
||||
}
|
||||
let hits: Vec<Value> = crate::label::search(&module, &words)
|
||||
.into_iter()
|
||||
.map(|h| {
|
||||
json!({
|
||||
"line": h.site.line,
|
||||
"unit": h.site.unit,
|
||||
"label": h.site.label,
|
||||
"words": h.words,
|
||||
"has_payload": h.site.has_payload,
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
Ok(json!({"ok": true, "words": words, "count": hits.len(), "hits": hits}))
|
||||
}
|
||||
|
||||
fn module_from_params(params: &Value) -> Result<(Module, Option<PathBuf>), Value> {
|
||||
if let Some(src) = params.get("source").and_then(|s| s.as_str()) {
|
||||
let module = load::load_module_from_source(src)
|
||||
|
||||
@@ -99,3 +99,72 @@ fn label_refs_that_resolve_are_fine() {
|
||||
assert_eq!(code, 0, "text={text}");
|
||||
assert!(!text.contains("不存在的实体"), "text={text}");
|
||||
}
|
||||
|
||||
// ---- M9: two-tier label search (`labels` CLI + MLSP `labels.search`) ----
|
||||
|
||||
fn labels(args: &[&str]) -> (i32, String) {
|
||||
let out = Command::new(env!("CARGO_BIN_EXE_marqdo"))
|
||||
.arg("labels")
|
||||
.args(args)
|
||||
.env("MARQDO_MARKUP", "occlusion")
|
||||
.output()
|
||||
.expect("run marqdo labels");
|
||||
(
|
||||
out.status.code().unwrap_or(1),
|
||||
format!(
|
||||
"{}{}",
|
||||
String::from_utf8_lossy(&out.stdout),
|
||||
String::from_utf8_lossy(&out.stderr)
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn labels_pending_uses_builtin_groups() {
|
||||
let (code, text) = labels(&["tests/occlusion/labels.mq.md", "--count"]);
|
||||
assert_eq!(code, 0, "text={text}");
|
||||
assert_eq!(text.trim(), "2", "expected 待开发 + 模拟: {text}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn labels_grep_replaces_builtin() {
|
||||
let (code, text) = labels(&["tests/occlusion/labels.mq.md", "--grep", "联调", "--count"]);
|
||||
assert_eq!(code, 0, "text={text}");
|
||||
assert_eq!(text.trim(), "1", "text={text}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn labels_grep_with_builtin_unions() {
|
||||
let (code, text) = labels(&[
|
||||
"tests/occlusion/labels.mq.md",
|
||||
"--grep",
|
||||
"联调",
|
||||
"--also-builtin",
|
||||
"--count",
|
||||
]);
|
||||
assert_eq!(code, 0, "text={text}");
|
||||
assert_eq!(text.trim(), "3", "text={text}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mlsp_labels_search_returns_hits() {
|
||||
use std::io::Write;
|
||||
let mut child = Command::new(env!("CARGO_BIN_EXE_marqdo"))
|
||||
.arg("mlsp")
|
||||
.env("MARQDO_MARKUP", "occlusion")
|
||||
.stdin(std::process::Stdio::piped())
|
||||
.stdout(std::process::Stdio::piped())
|
||||
.spawn()
|
||||
.expect("spawn mlsp");
|
||||
let req = r#"{"id":1,"method":"labels.search","params":{"path":"tests/occlusion/labels.mq.md","also_builtin":true}}"#;
|
||||
child
|
||||
.stdin
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.write_all(format!("{req}\n").as_bytes())
|
||||
.unwrap();
|
||||
let out = child.wait_with_output().unwrap();
|
||||
let text = String::from_utf8_lossy(&out.stdout).into_owned();
|
||||
assert!(text.contains("\"count\":2"), "text={text}");
|
||||
assert!(text.contains("客户留存率"), "text={text}");
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
---
|
||||
title: label search fixture
|
||||
---
|
||||
|
||||
# main
|
||||
|
||||
**x = [按关键词检索,接口未接,待开发](None)**
|
||||
**y = [客户留存率,模拟](0.0)**
|
||||
**z = [导出报表,示例](1)**
|
||||
**w = [接口还在联调,先返回空](0)**
|
||||
> print text=`x`
|
||||
Reference in new issue
Block a user