Add Python spike that passes S1–S5 parsing checks.

Document line-scan for +/- bullets and hybrid frontmatter imports; defer choosing the full reference-implementation language.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
cflmyandCursor committed 2026-08-04 11:00:56 +08:00
1 parent c1a4bb47b6
commit 23fa4284a8
8 files changed
+253 -21

No files matched your search

+1 -1
View File
@@ -20,7 +20,7 @@
| [design/return-hr-and-code-surface.md](design/return-hr-and-code-surface.md) | 语句/返回/输出/分割线决议记录 |
| [design/generated-yaml-manifest.md](design/generated-yaml-manifest.md) | 生成式 OKF 清单 |
| [design/tech-stack.md](design/tech-stack.md) | 实现栈对比分析 |
| [adr/0001-implementation-language.md](adr/0001-implementation-language.md) | **选型 ADR(提议中)** |
| [adr/0001-implementation-language.md](adr/0001-implementation-language.md) | 选型 ADR(Python Spike **已通过**) |
| [research/okf-and-marqdo.md](research/okf-and-marqdo.md) | OKF 调研 |
示例程序见仓库 [`examples/`](../examples/)。
+14 -20
View File
@@ -2,7 +2,7 @@
| | |
|---|---|
| 状态 | **提议中(Proposed)** — 待确认后 Accepted |
| 状态 | **Spike 通过(Python)** — 参考实现语言待勾选 |
| 日期 | 2026-08-04 |
| 决策者 | chaungming + 协作 |
| 相关 | [tech-stack.md](../design/tech-stack.md) · [markdown-mapping.md](../design/markdown-mapping.md) |
@@ -48,13 +48,15 @@
### 验证期(现在 → 跑通 examples)
**采用选项 A:TypeScript + remark-gfm + mdast + Vitest + Node ≥ 20。**
**Spike 优先采用 Python(快速验证),不默认上 npm/TypeScript 脚手架。**
理由:
| 步骤 | 选择 |
|------|------|
| **Spike(进行中)** | **Python 3.11+** + 轻量 Markdown 库(如 `markdown-it-py`)+ `pytest` |
| 验证通过后的参考实现 | 另议:可继续 Python,或再评 TypeScript / Rust(见原选项 A/B) |
1. 与 GitHub 预览最接近,减少「能渲染不能跑」。
2. mdast 保留 list 的 `spread` / 子节点;需 Spike 确认 **bullet 字符 `-`/`+` 可取**(取不到则自研行扫描补一层)。
3. 迭代宪法快,符合「设计刚定、语义还在磨」的阶段。
理由:npm 安装过慢会阻塞 Spike;Python 足以验收 S1–S5 与「行分类 / bullet 区分」风险。
**原选项 A(TypeScript + remark)降为验证通过后的候选实现,不再作为 Spike 必经之路。**
### 长期(宪法稳定后)
@@ -77,22 +79,14 @@
| S4 | 文件头 `---` + 正文内 `---` | 能分开 frontmatter 与可选框线 |
| S5 | frontmatter 行 `> x.mq.md` | 不被 YAML 解析掐死,可抽出导入 |
**提议流程:** 先做 **Spike A(≤1 日)** → 若 S2 失败再评估自研行扫描或转 Spike B → 将本 ADR 标为 **Accepted** → 再允许添加 `package.json` 与 `src/`。
---
## 后果
- Accepted 后:仓库可建 `src/`、`tests/`,CLI `marqdo run`。
- 生成 OKF 清单、LSP 等不在本 ADR 范围。
- 实现须遵守:先做行分类,再求值([code-vs-comment.md](../design/code-vs-comment.md))。
**提议流程:** 先做 **Python Spike(本目录 `spike/`,≤1 日)** → 记录 S1–S5 通过/补救 → 再决定参考实现语言(Python 延续 / TS / Rust)→ Accepted 后才建正式 `src/`。
---
## 确认栏
- [ ] 同意验证期 = TypeScript + remark
- [ ] 改选:验证期 = Rust(先 Spike B)
- [ ] 改选:其它:________
确认后把状态改为 **Accepted**,并开始 Spike(不是直接铺全量解释器)。
- [x] Spike 使用 Python(进行中)
- [ ] Spike 通过后:参考实现继续 Python
- [ ] Spike 通过后:改选 TypeScript
- [ ] Spike 通过后:改选 Rust
- [ ] 其它:________
+4
View File
@@ -0,0 +1,4 @@
.venv/
__pycache__/
*.pyc
.pytest_cache/
+14
View File
@@ -0,0 +1,14 @@
# Marqdo Spike(Python)
快速验收 ADR 0001 中的 S1–S5,**不是**完整解释器。
```bash
cd spike
python -m venv .venv
# Windows:
.\.venv\Scripts\activate
pip install -r requirements.txt
pytest -q
```
若需代理:`pip install -r requirements.txt --proxy http://127.0.0.1:7897`
+35
View File
@@ -0,0 +1,35 @@
# Spike 报告:Python + markdown-it-py(S1–S5)
| | |
|---|---|
| 日期 | 2026-08-04 |
| 状态 | **通过** |
| 命令 | `cd spike && pytest -q` → `6 passed` |
---
## 结果
| ID | 项 | 结果 | 备注 |
|----|----|------|------|
| S1 | `*` vs `**` | 通过 | `em_*` / `strong_*` 在 inline children 中 |
| S2 | `-` vs `+` | 通过 | **行扫描** `scan_bullets` 保留项目符;库只保证有 `bullet_list` |
| S3 | 单列表格 | 通过 | `table` 规则开启后可解析 |
| S4 | FM `---` vs 正文 `---` | 通过 | 自研 `split_front_matter` |
| S5 | FM 内 `> *.mq.md` | 通过 | 行正则抽取,不整块丢给严格 YAML |
---
## 对实现的含义
1. **行扫描是必需层**:`-`/`+` 消歧、frontmatter 导入、注释行分类,不宜只靠 MD 库 AST。
2. **markdown-it-py** 足够做块/inline 结构(标题、表、强调、列表);避免 `gfm-like` 默认 `linkify`(缺可选依赖会炸)。
3. Spike **不**等于选定长期实现语言;仅证明 Python 路径可验证宪法。参考实现可继续 Python,或再评 TS/Rust。
---
## 建议下一步
1. 将 ADR 0001 中 Spike 标为通过。
2. 拍板:参考实现是否 **继续 Python**(推荐,与 Spike 同栈)或换栈。
3. 再搭建正式包(`src/marqdo`)与 `marqdo run`,按 examples 金样例推进。
+78
View File
@@ -0,0 +1,78 @@
"""
Spike helpers: GFM parse via markdown-it-py + line-level bullet probe.
"""
from __future__ import annotations
import re
from dataclasses import dataclass
from markdown_it import MarkdownIt
from mdit_py_plugins.front_matter import front_matter_plugin
def make_md() -> MarkdownIt:
# Avoid gfm-like's linkify (needs optional linkify-it-py).
md = MarkdownIt("commonmark", {"html": False, "linkify": False})
md.enable(["table"])
md.use(front_matter_plugin)
return md
@dataclass
class BulletLine:
line_no: int
marker: str # "-" | "+" | "*" | "1."
text: str
_BULLET = re.compile(
r"^(?P<indent>\s*)(?P<marker>[-+*]|\d+\.)\s+(?P<text>.*)$"
)
def scan_bullets(source: str) -> list[BulletLine]:
"""Line-level scan: preserves '-' vs '+' regardless of MD library."""
out: list[BulletLine] = []
for i, line in enumerate(source.splitlines(), start=1):
m = _BULLET.match(line)
if not m:
continue
out.append(
BulletLine(line_no=i, marker=m.group("marker"), text=m.group("text"))
)
return out
def parse_tokens(source: str):
return make_md().parse(source)
def split_front_matter(source: str) -> tuple[str | None, str]:
"""
Split leading --- ... --- front matter from body.
Body-internal --- is left untouched.
"""
if not source.startswith("---"):
return None, source
lines = source.splitlines(keepends=True)
if not lines or lines[0].strip() != "---":
return None, source
for i in range(1, len(lines)):
if lines[i].strip() == "---":
fm = "".join(lines[1:i])
body = "".join(lines[i + 1 :])
return fm, body
return None, source
_IMPORT = re.compile(r"^>\s*(?P<path>\S+\.mq\.md)\s*$")
def front_matter_imports(fm: str) -> list[str]:
paths: list[str] = []
for line in fm.splitlines():
m = _IMPORT.match(line.strip())
if m:
paths.append(m.group("path"))
return paths
+3
View File
@@ -0,0 +1,3 @@
markdown-it-py>=3.0,<4
mdit-py-plugins>=0.4,<1
pytest>=8.0,<9
+104
View File
@@ -0,0 +1,104 @@
"""ADR 0001 Spike acceptance: S1–S5."""
from __future__ import annotations
from marqdo_spike import (
front_matter_imports,
parse_tokens,
scan_bullets,
split_front_matter,
)
def test_s1_emphasis_vs_strong():
"""S1: *a* vs **a** yield distinct inline marks."""
src = "hello *em* and **strong** end\n"
tokens = parse_tokens(src)
inlines = [t for t in tokens if t.type == "inline"]
assert inlines, "expected inline token"
children = inlines[0].children or []
types = [c.type for c in children]
assert "em_open" in types and "em_close" in types
assert "strong_open" in types and "strong_close" in types
assert any(c.type == "text" and c.content == "em" for c in children)
assert any(c.type == "text" and c.content == "strong" for c in children)
def test_s2_bullet_minus_vs_plus_line_scan():
"""S2: line scan distinguishes '-' (loop) vs '+' (branch)."""
src = """
# main
- loop item
+ branch item
"""
bullets = scan_bullets(src)
markers = [b.marker for b in bullets]
assert "-" in markers
assert "+" in markers
assert bullets[0].marker == "-"
assert bullets[1].marker == "+"
def test_s2_library_list_tokens_exist():
"""S2b: library at least parses both as lists (bullet char may need line scan)."""
src = "- a\n\n+ b\n"
tokens = parse_tokens(src)
assert any(t.type == "bullet_list_open" for t in tokens)
def test_s3_single_column_table():
"""S3: GFM single-column table."""
src = """
| 果 |
|----|
| 苹果 |
| 梨 |
"""
tokens = parse_tokens(src)
assert any(t.type == "table_open" for t in tokens)
texts = [t.content for t in tokens if t.type == "inline"]
flat = " ".join(texts)
assert "果" in flat
assert "苹果" in flat
assert "梨" in flat
def test_s4_front_matter_vs_body_hr():
"""S4: file-level --- front matter vs body --- frame."""
src = """---
title: demo
---
# main
---
+ `x` > 0
> 输出 内容=正
---
"""
fm, body = split_front_matter(src)
assert fm is not None
assert "title: demo" in fm
assert body.lstrip().startswith("# main")
assert "---" in body
assert "+ `x` > 0" in body
# Body still has its own ---; must not be eaten by FM splitter
assert body.count("---") >= 2
def test_s5_front_matter_mq_imports():
"""S5: `> file.mq.md` lines inside front matter."""
src = """---
title: main
> utils.mq.md
> lib/math.mq.md
---
# main
"""
fm, body = split_front_matter(src)
assert fm is not None
imports = front_matter_imports(fm)
assert imports == ["utils.mq.md", "lib/math.mq.md"]
assert "# main" in body