Add Python spike that passes S1–S5 parsing checks.
Document line-scan for +/- bullets and hybrid frontmatter imports; defer choosing the full reference-implementation language. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
1 parent
c1a4bb47b6
commit
23fa4284a8
8 files changed
+253
-21
No files matched your search
+1
-1
@@ -20,7 +20,7 @@
|
||||
| [design/return-hr-and-code-surface.md](design/return-hr-and-code-surface.md) | 语句/返回/输出/分割线决议记录 |
|
||||
| [design/generated-yaml-manifest.md](design/generated-yaml-manifest.md) | 生成式 OKF 清单 |
|
||||
| [design/tech-stack.md](design/tech-stack.md) | 实现栈对比分析 |
|
||||
| [adr/0001-implementation-language.md](adr/0001-implementation-language.md) | **选型 ADR(提议中)** |
|
||||
| [adr/0001-implementation-language.md](adr/0001-implementation-language.md) | 选型 ADR(Python Spike **已通过**) |
|
||||
| [research/okf-and-marqdo.md](research/okf-and-marqdo.md) | OKF 调研 |
|
||||
|
||||
示例程序见仓库 [`examples/`](../examples/)。
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **提议中(Proposed)** — 待确认后 Accepted |
|
||||
| 状态 | **Spike 通过(Python)** — 参考实现语言待勾选 |
|
||||
| 日期 | 2026-08-04 |
|
||||
| 决策者 | chaungming + 协作 |
|
||||
| 相关 | [tech-stack.md](../design/tech-stack.md) · [markdown-mapping.md](../design/markdown-mapping.md) |
|
||||
@@ -48,13 +48,15 @@
|
||||
|
||||
### 验证期(现在 → 跑通 examples)
|
||||
|
||||
**采用选项 A:TypeScript + remark-gfm + mdast + Vitest + Node ≥ 20。**
|
||||
**Spike 优先采用 Python(快速验证),不默认上 npm/TypeScript 脚手架。**
|
||||
|
||||
理由:
|
||||
| 步骤 | 选择 |
|
||||
|------|------|
|
||||
| **Spike(进行中)** | **Python 3.11+** + 轻量 Markdown 库(如 `markdown-it-py`)+ `pytest` |
|
||||
| 验证通过后的参考实现 | 另议:可继续 Python,或再评 TypeScript / Rust(见原选项 A/B) |
|
||||
|
||||
1. 与 GitHub 预览最接近,减少「能渲染不能跑」。
|
||||
2. mdast 保留 list 的 `spread` / 子节点;需 Spike 确认 **bullet 字符 `-`/`+` 可取**(取不到则自研行扫描补一层)。
|
||||
3. 迭代宪法快,符合「设计刚定、语义还在磨」的阶段。
|
||||
理由:npm 安装过慢会阻塞 Spike;Python 足以验收 S1–S5 与「行分类 / bullet 区分」风险。
|
||||
**原选项 A(TypeScript + remark)降为验证通过后的候选实现,不再作为 Spike 必经之路。**
|
||||
|
||||
### 长期(宪法稳定后)
|
||||
|
||||
@@ -77,22 +79,14 @@
|
||||
| S4 | 文件头 `---` + 正文内 `---` | 能分开 frontmatter 与可选框线 |
|
||||
| S5 | frontmatter 行 `> x.mq.md` | 不被 YAML 解析掐死,可抽出导入 |
|
||||
|
||||
**提议流程:** 先做 **Spike A(≤1 日)** → 若 S2 失败再评估自研行扫描或转 Spike B → 将本 ADR 标为 **Accepted** → 再允许添加 `package.json` 与 `src/`。
|
||||
|
||||
---
|
||||
|
||||
## 后果
|
||||
|
||||
- Accepted 后:仓库可建 `src/`、`tests/`,CLI `marqdo run`。
|
||||
- 生成 OKF 清单、LSP 等不在本 ADR 范围。
|
||||
- 实现须遵守:先做行分类,再求值([code-vs-comment.md](../design/code-vs-comment.md))。
|
||||
**提议流程:** 先做 **Python Spike(本目录 `spike/`,≤1 日)** → 记录 S1–S5 通过/补救 → 再决定参考实现语言(Python 延续 / TS / Rust)→ Accepted 后才建正式 `src/`。
|
||||
|
||||
---
|
||||
|
||||
## 确认栏
|
||||
|
||||
- [ ] 同意验证期 = TypeScript + remark
|
||||
- [ ] 改选:验证期 = Rust(先 Spike B)
|
||||
- [ ] 改选:其它:________
|
||||
|
||||
确认后把状态改为 **Accepted**,并开始 Spike(不是直接铺全量解释器)。
|
||||
- [x] Spike 使用 Python(进行中)
|
||||
- [ ] Spike 通过后:参考实现继续 Python
|
||||
- [ ] Spike 通过后:改选 TypeScript
|
||||
- [ ] Spike 通过后:改选 Rust
|
||||
- [ ] 其它:________
|
||||
@@ -0,0 +1,4 @@
|
||||
.venv/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
.pytest_cache/
|
||||
@@ -0,0 +1,14 @@
|
||||
# Marqdo Spike(Python)
|
||||
|
||||
快速验收 ADR 0001 中的 S1–S5,**不是**完整解释器。
|
||||
|
||||
```bash
|
||||
cd spike
|
||||
python -m venv .venv
|
||||
# Windows:
|
||||
.\.venv\Scripts\activate
|
||||
pip install -r requirements.txt
|
||||
pytest -q
|
||||
```
|
||||
|
||||
若需代理:`pip install -r requirements.txt --proxy http://127.0.0.1:7897`
|
||||
@@ -0,0 +1,35 @@
|
||||
# Spike 报告:Python + markdown-it-py(S1–S5)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 日期 | 2026-08-04 |
|
||||
| 状态 | **通过** |
|
||||
| 命令 | `cd spike && pytest -q` → `6 passed` |
|
||||
|
||||
---
|
||||
|
||||
## 结果
|
||||
|
||||
| ID | 项 | 结果 | 备注 |
|
||||
|----|----|------|------|
|
||||
| S1 | `*` vs `**` | 通过 | `em_*` / `strong_*` 在 inline children 中 |
|
||||
| S2 | `-` vs `+` | 通过 | **行扫描** `scan_bullets` 保留项目符;库只保证有 `bullet_list` |
|
||||
| S3 | 单列表格 | 通过 | `table` 规则开启后可解析 |
|
||||
| S4 | FM `---` vs 正文 `---` | 通过 | 自研 `split_front_matter` |
|
||||
| S5 | FM 内 `> *.mq.md` | 通过 | 行正则抽取,不整块丢给严格 YAML |
|
||||
|
||||
---
|
||||
|
||||
## 对实现的含义
|
||||
|
||||
1. **行扫描是必需层**:`-`/`+` 消歧、frontmatter 导入、注释行分类,不宜只靠 MD 库 AST。
|
||||
2. **markdown-it-py** 足够做块/inline 结构(标题、表、强调、列表);避免 `gfm-like` 默认 `linkify`(缺可选依赖会炸)。
|
||||
3. Spike **不**等于选定长期实现语言;仅证明 Python 路径可验证宪法。参考实现可继续 Python,或再评 TS/Rust。
|
||||
|
||||
---
|
||||
|
||||
## 建议下一步
|
||||
|
||||
1. 将 ADR 0001 中 Spike 标为通过。
|
||||
2. 拍板:参考实现是否 **继续 Python**(推荐,与 Spike 同栈)或换栈。
|
||||
3. 再搭建正式包(`src/marqdo`)与 `marqdo run`,按 examples 金样例推进。
|
||||
@@ -0,0 +1,78 @@
|
||||
"""
|
||||
Spike helpers: GFM parse via markdown-it-py + line-level bullet probe.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
|
||||
from markdown_it import MarkdownIt
|
||||
from mdit_py_plugins.front_matter import front_matter_plugin
|
||||
|
||||
|
||||
def make_md() -> MarkdownIt:
|
||||
# Avoid gfm-like's linkify (needs optional linkify-it-py).
|
||||
md = MarkdownIt("commonmark", {"html": False, "linkify": False})
|
||||
md.enable(["table"])
|
||||
md.use(front_matter_plugin)
|
||||
return md
|
||||
|
||||
|
||||
@dataclass
|
||||
class BulletLine:
|
||||
line_no: int
|
||||
marker: str # "-" | "+" | "*" | "1."
|
||||
text: str
|
||||
|
||||
|
||||
_BULLET = re.compile(
|
||||
r"^(?P<indent>\s*)(?P<marker>[-+*]|\d+\.)\s+(?P<text>.*)$"
|
||||
)
|
||||
|
||||
|
||||
def scan_bullets(source: str) -> list[BulletLine]:
|
||||
"""Line-level scan: preserves '-' vs '+' regardless of MD library."""
|
||||
out: list[BulletLine] = []
|
||||
for i, line in enumerate(source.splitlines(), start=1):
|
||||
m = _BULLET.match(line)
|
||||
if not m:
|
||||
continue
|
||||
out.append(
|
||||
BulletLine(line_no=i, marker=m.group("marker"), text=m.group("text"))
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def parse_tokens(source: str):
|
||||
return make_md().parse(source)
|
||||
|
||||
|
||||
def split_front_matter(source: str) -> tuple[str | None, str]:
|
||||
"""
|
||||
Split leading --- ... --- front matter from body.
|
||||
Body-internal --- is left untouched.
|
||||
"""
|
||||
if not source.startswith("---"):
|
||||
return None, source
|
||||
lines = source.splitlines(keepends=True)
|
||||
if not lines or lines[0].strip() != "---":
|
||||
return None, source
|
||||
for i in range(1, len(lines)):
|
||||
if lines[i].strip() == "---":
|
||||
fm = "".join(lines[1:i])
|
||||
body = "".join(lines[i + 1 :])
|
||||
return fm, body
|
||||
return None, source
|
||||
|
||||
|
||||
_IMPORT = re.compile(r"^>\s*(?P<path>\S+\.mq\.md)\s*$")
|
||||
|
||||
|
||||
def front_matter_imports(fm: str) -> list[str]:
|
||||
paths: list[str] = []
|
||||
for line in fm.splitlines():
|
||||
m = _IMPORT.match(line.strip())
|
||||
if m:
|
||||
paths.append(m.group("path"))
|
||||
return paths
|
||||
@@ -0,0 +1,3 @@
|
||||
markdown-it-py>=3.0,<4
|
||||
mdit-py-plugins>=0.4,<1
|
||||
pytest>=8.0,<9
|
||||
@@ -0,0 +1,104 @@
|
||||
"""ADR 0001 Spike acceptance: S1–S5."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from marqdo_spike import (
|
||||
front_matter_imports,
|
||||
parse_tokens,
|
||||
scan_bullets,
|
||||
split_front_matter,
|
||||
)
|
||||
|
||||
|
||||
def test_s1_emphasis_vs_strong():
|
||||
"""S1: *a* vs **a** yield distinct inline marks."""
|
||||
src = "hello *em* and **strong** end\n"
|
||||
tokens = parse_tokens(src)
|
||||
inlines = [t for t in tokens if t.type == "inline"]
|
||||
assert inlines, "expected inline token"
|
||||
children = inlines[0].children or []
|
||||
types = [c.type for c in children]
|
||||
assert "em_open" in types and "em_close" in types
|
||||
assert "strong_open" in types and "strong_close" in types
|
||||
assert any(c.type == "text" and c.content == "em" for c in children)
|
||||
assert any(c.type == "text" and c.content == "strong" for c in children)
|
||||
|
||||
|
||||
def test_s2_bullet_minus_vs_plus_line_scan():
|
||||
"""S2: line scan distinguishes '-' (loop) vs '+' (branch)."""
|
||||
src = """
|
||||
# main
|
||||
|
||||
- loop item
|
||||
+ branch item
|
||||
"""
|
||||
bullets = scan_bullets(src)
|
||||
markers = [b.marker for b in bullets]
|
||||
assert "-" in markers
|
||||
assert "+" in markers
|
||||
assert bullets[0].marker == "-"
|
||||
assert bullets[1].marker == "+"
|
||||
|
||||
|
||||
def test_s2_library_list_tokens_exist():
|
||||
"""S2b: library at least parses both as lists (bullet char may need line scan)."""
|
||||
src = "- a\n\n+ b\n"
|
||||
tokens = parse_tokens(src)
|
||||
assert any(t.type == "bullet_list_open" for t in tokens)
|
||||
|
||||
|
||||
def test_s3_single_column_table():
|
||||
"""S3: GFM single-column table."""
|
||||
src = """
|
||||
| 果 |
|
||||
|----|
|
||||
| 苹果 |
|
||||
| 梨 |
|
||||
"""
|
||||
tokens = parse_tokens(src)
|
||||
assert any(t.type == "table_open" for t in tokens)
|
||||
texts = [t.content for t in tokens if t.type == "inline"]
|
||||
flat = " ".join(texts)
|
||||
assert "果" in flat
|
||||
assert "苹果" in flat
|
||||
assert "梨" in flat
|
||||
|
||||
|
||||
def test_s4_front_matter_vs_body_hr():
|
||||
"""S4: file-level --- front matter vs body --- frame."""
|
||||
src = """---
|
||||
title: demo
|
||||
---
|
||||
|
||||
# main
|
||||
|
||||
---
|
||||
+ `x` > 0
|
||||
> 输出 内容=正
|
||||
---
|
||||
"""
|
||||
fm, body = split_front_matter(src)
|
||||
assert fm is not None
|
||||
assert "title: demo" in fm
|
||||
assert body.lstrip().startswith("# main")
|
||||
assert "---" in body
|
||||
assert "+ `x` > 0" in body
|
||||
# Body still has its own ---; must not be eaten by FM splitter
|
||||
assert body.count("---") >= 2
|
||||
|
||||
|
||||
def test_s5_front_matter_mq_imports():
|
||||
"""S5: `> file.mq.md` lines inside front matter."""
|
||||
src = """---
|
||||
title: main
|
||||
> utils.mq.md
|
||||
> lib/math.mq.md
|
||||
---
|
||||
|
||||
# main
|
||||
"""
|
||||
fm, body = split_front_matter(src)
|
||||
assert fm is not None
|
||||
imports = front_matter_imports(fm)
|
||||
assert imports == ["utils.mq.md", "lib/math.mq.md"]
|
||||
assert "# main" in body
|
||||
Reference in new issue
Block a user