feat(rfc-cm): baseline P0-P3 compute machine, stdlib and resource slices
Implements the first vertical slices of RFC-CM-0001/0002 on top of a113d89:
- P1: opaque Memory regions (64 KiB pages, budgets, grow-only), byte
read/write, Memory-index place assignment, wire/host-boundary rejection.
- CM-A04/A05: value-semantics List/Map/Text/Table primitives and the
recoverable Result map convention.
- CM-A06: opaque Resource handles, resource.open/read/write/close, stdout
write, granted project-file read and buffered write, strict fail-closed.
- CM-A07 Compat-A: explicit .mq.md format/utf8/io chain matching builtin
print for the covered value types; builtin remains default.
- P2 fixtures: allocators, list/tree/graph/hash, integer codecs, UTF-8
encode/decode, CSV/JSON subsets, table filter/sort/join, bytecode VM.
- ADRs 0022-0027, spec updates, evidence reports and 68 semantic goldens.
Gate: cargo fmt --check, clippy -D warnings, 407 tests passed.
This commit is contained in:
1 parent
a113d89496
commit
92bfbf2568
127 files changed
+11351
-141
No files matched your search
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,498 @@
|
||||
# RFC-CM-0001:Marqdo 自主计算机器、可寻址内存与系统交互边界
|
||||
|
||||
> **状态**:设计提案(Draft,未实施,未批准)
|
||||
> **日期**:2026-11-20
|
||||
> **建议归档路径**:`docs/architecture/RFC-CM-0001-compute-machine.md`
|
||||
> **验证基线**:`cflmy/marqdo-cli`,Git 提交 `a113d894962d8c00eb02efb42e0e73f8b084a1fa`(检索于 2026-11-20)
|
||||
> **兼容性**:涉及 `Value`、表达式赋值、运行时、主机能力和线协议的变更,必须通过 ADR、规范版本与相应测试;本 RFC 不声明任何新接口目前可用。
|
||||
|
||||
---
|
||||
|
||||
## 摘要
|
||||
|
||||
Marqdo 的优势不应只是用 Markdown 排版代码,而应是:在不更换 Markdown 语法角色的前提下,以**自身定义的计算语义**构造算法、数据结构、运行时工具和知识制品。现有解释器已提供整数、浮点、文本、列表、映射、控制流和函数调用,但尚未提供**由 Marqdo 程序控制的可寻址字节存储**;其赋值左侧仅支持标识符,系统交互集中在一组固定 `HostOp` 与外部插件能力中。这妨碍了用 `.mq.md` 自己编写内存分配器、二进制协议、共享节点结构、虚拟机器等低层算法。
|
||||
|
||||
**提议:以“托管值 + 多个可增长的线性内存区域 + 不透明系统资源”为计算机器的三个基础领域。**
|
||||
|
||||
1. 保留现有 `Value` 计算体系,不强迫一切数据都变成字节地址。
|
||||
2. 引入一个拥有独立身份、按字节索引、可增长、具有边界的 `Memory` 值;每个区域内部线性寻址,物理实现可以是 Vec、分页或其他表示。
|
||||
3. 扩展现有“位置赋值”:允许对适当的索引目标写入,如 `m[8] = 42`,而不是另创 Markdown 语法或几十个关键字。
|
||||
4. 为创建、测量和增长内存定义**极少量计算原语**;让 `malloc/free`、内存池、链表、编码、缓冲区等**首先在 `.mq.md` 标准库**中实现。
|
||||
5. 将终端、文件、网络、进程、设备定义为**外部资源**,经稳定、可授权、可替换的底层适配接口交互;Marqdo 自身可以实现高级系统库与协议,但系统调用或硬件访问的**最后一步不能凭空消失**。
|
||||
6. CPU 寄存器由解释器/编译后端管理;裸机器码、GPU/QPU、共享内存原子操作暂不进入本 RFC 的基础语义。
|
||||
|
||||
这是一种**可编程的抽象机器**,而不是“在每种新需求前追加 Rust Host 函数”的框架,也不是把 x86/RISC-V 指令直接变成 Markdown 语法。
|
||||
|
||||
## 1. 设计动机:究竟希望解决什么
|
||||
|
||||
### 1.1 真实诉求
|
||||
|
||||
Marqdo 必须能在不借助 Python、JavaScript 或宿主语言提供专门算法函数的前提下,**通过自己的控制流和基础存储机制**构造原先不存在的高级能力。例如:实现内存分配器、二叉树、哈希表、字节码解释器、文件格式解析器、数值算法。对于 OS 和硬件,要求的是**高级功能主要由 Marqdo 实现,底层权限与系统调用接口保持稳定**,而不是“所有跨越内核边界的指令都神奇地由 Markdown 自己完成”。
|
||||
|
||||
### 1.2 非目标
|
||||
|
||||
- 不追求复刻 CPU 的真实寄存器、指令流水线、页表与异常处理体系。
|
||||
- 不把所有 Markdown `[]()` 链接重新定义为动态指针。链接继续表示文档/符号寻址,不伪装运行时对象地址。
|
||||
- 不把 GFM 表格当成实际运行内存。表格可以用于数据/状态的可读表达与快照,但不是存储管理器。
|
||||
- 不要求标准库全部使用 `.mq.md`;原生加速可以有,但**不是实现一般算法的必要条件**。
|
||||
- 不在第一期承诺用户代码内存安全、数据竞争自由、GPU 加速或跨平台无差异系统调用。
|
||||
- 不支持在无权限环境中直接执行任意机器码或任意 syscall。
|
||||
|
||||
### 1.3 设计原则
|
||||
|
||||
**P1 语义最小而充分**:首先增加无法由现有表达式自然实现的基础机制,不先增加高级 API 或新关键词。
|
||||
**P2 高级能力在语言中生长**:新增算法/数据结构不修改解释器、不修改 `HostOp`、不需要重新编译 CLI。
|
||||
**P3 读写位置统一**:索引读取和索引赋值遵循同一类“位置”语义;`Memory` 的读取/写入是基础例子,普通列表的值语义不得无声改变。
|
||||
**P4 内外边界诚实**:内存操作是语言计算;操作系统 I/O 依赖权限、平台接口和底层实现。
|
||||
**P5 可读性**:普通 Markdown 文档仍然可读;`##`/`**`/`*`/列表/blockquote 的现有角色不因本 RFC 改变。
|
||||
**P6 安全与可验证性**:将可验证的保证限定在区域边界和权限边界;不对自定义分配器的块级错误作虚假保证。
|
||||
**P7 零隐式权限**:能够导入模块、持有字符串或解析文档,不代表拥有文件、网络或设备访问权。
|
||||
|
||||
## 2. 现有仓库事实及需要保留的设计
|
||||
|
||||
依据提交 `a113d894...` 的源代码和规范:
|
||||
|
||||
| 项目 | 现状 | 对本 RFC 的约束 |
|
||||
|---|---|---|
|
||||
| `marqdo-value` | `Unit/None/Bool/Int/Float/Text/List/Map/Table/Formula/Code/FunctionRef/PluginHandle`,没有普通可寻址 `Memory` 类型 | 应新增专门的存储引用值,而非把现有 List 改成默认共享对象 |
|
||||
| `marqdo-expression` | 索引 `a[i]`/成员 `a.x` 可读取;`assignment = identifier = expression` | 最小语法修改是赋值目标扩展,不是新 Markdown 节点 |
|
||||
| `marqdo-runtime` | `Frame.locals: BTreeMap<String, Value>`;`Assign` 以 `locals.insert` 重绑定 | 局部变量重新赋值应保持既有含义;共享可变性仅属于显式 `Memory` |
|
||||
| `marqdo-frontend` | Markdown AST 是语法角色唯一权威;GFM Table 惰性 | 不更改表格、链接的执行语义;表达式子语言可增补语义 |
|
||||
| `marqdo-ir` | 内置函数元数据固定且需更新规范 | 新增基础内存原语需要小型且审慎的语言变更;高层库不应无限扩展 Builtin 表 |
|
||||
| `marqdo-host` | 固定 `HostOp`,包括 Print、Math、Foreign、FsRead/FsWrite、PluginInvoke | 不应将每一个新的标准库函数继续塞入固定 HostOp |
|
||||
| 插件 ABI v1 | C ABI、worker、能力请求已有雏形 | 可复用函数注册与 Worker,不能错误地把 C ABI 当沙箱 |
|
||||
| `pure/project/strict` | `pure` 禁止外部 I/O grant;`strict` Linux 探测成功后只支持 L1 范围,其他平台 fail closed | 新系统资源功能首先在项目模式开发;strict 须经新的隔离验证,不得默许开启 |
|
||||
| 包管理 | path/Git 依赖与 lock v3 已实现;版本范围部分尚未强制 | 独立标准库可利用现有依赖体系,但兼容版本需要明确验证 |
|
||||
|
||||
**保留的价值**:现有整数运算、函数调用、条件和循环不必推翻;现有 AST 映射、资源边界和可追踪执行事件是 Marqdo 与脚本转译器的根本区别之一。
|
||||
|
||||
源码(固定版本):
|
||||
- [`spec/expression.ebnf`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/expression.ebnf)
|
||||
- [`spec/mapping.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/mapping.md)
|
||||
- [`spec/evaluation.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/evaluation.md)
|
||||
- [`crates/marqdo-value/src/value.rs`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/crates/marqdo-value/src/value.rs)
|
||||
- [`crates/marqdo-runtime/src/interpreter.rs`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/crates/marqdo-runtime/src/interpreter.rs)
|
||||
- [`spec/security.md`](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/security.md)
|
||||
|
||||
## 3. 总体架构与唯一的语义边界
|
||||
|
||||
```text
|
||||
.mq.md 文档 GFM Table/Link/标题/正文仍按 Markdown 正常渲染
|
||||
│
|
||||
▼
|
||||
Canonical Markdown AST ──▶ 已有 Frontend ──▶ Marqdo IR
|
||||
│
|
||||
┌───────────────────┤
|
||||
▼ ▼
|
||||
值与控制 存储与位置
|
||||
Int/List/Map MemoryRegion
|
||||
函数/赋值/循环 字节读写/增长
|
||||
└─────────┬─────────┘
|
||||
│
|
||||
Marqdo 计算核心
|
||||
│
|
||||
需要访问外部世界时才穿越边界
|
||||
▼
|
||||
Resource capability gate
|
||||
│ │ │
|
||||
console file network/device
|
||||
│ │ │
|
||||
Platform Adapter / OS ABI
|
||||
```
|
||||
|
||||
逻辑上的执行状态可以表述为:
|
||||
|
||||
`State = (Control, Frames, ManagedValues, RegionStore, ResourceTable, Budget)`。
|
||||
|
||||
- `Control`:当前 IR 节点和执行顺序;保留已有控制流。
|
||||
- `Frames/ManagedValues`:现有 `Value`、局部变量和调用语义。
|
||||
- `RegionStore`:一组可增长字节区域,每个区域按 `(region identity, offset)` 寻址。
|
||||
- `ResourceTable`:操作系统资源的不透明句柄与授权信息,**不得与内存地址混用**。
|
||||
- `Budget`:fuel、调用深度、内存限额、I/O 限额、截止时间等。
|
||||
|
||||
此设计**不规定**必须先创建自定义字节码 VM。现有树遍历解释器可直接执行新内存语义;将来引入字节码、JIT、LLVM/Cranelift 等属于可选执行方式,必须保持可观察语义。
|
||||
|
||||
## 4. 三种基础领域:值、内存、资源
|
||||
|
||||
### 4.1 托管值:维持已有语义
|
||||
|
||||
`Int/Text/List/Map/Table/Formula/Code` 等继续是运行时管理的值。普通 `b = a` 不得偷偷将普通列表变成共享可变指针。现有对象可以由 Rust 分配内存——**实现语言不等于语言语义**。这个领域负责易用性、文档处理、不可变数据等。
|
||||
|
||||
### 4.2 Memory:拥有身份的字节区域
|
||||
|
||||
新增 `Value::Memory`(名称暂定),含不可伪造的运行时区域引用。逻辑区域内部是一串字节,下标为非负整数;不同区域即使下标相同也不是同一个位置。
|
||||
|
||||
- **区域身份**:由本次运行中的内存实例签发;`Memory` 复制/传参时保留指向相同区域的身份(共享,不复制所有字节)。
|
||||
- **地址**:`(MemoryHandle, offset)`;`offset` 只在所属区域内有意义;整数本身不是获得其他区域权限的凭据。
|
||||
- **内容**:每个元素是 `0..255` 的一个字节;初始字节为 0。
|
||||
- **大小**:区域长度是页大小的整数倍;建议初期采用 `64 KiB` 页,以简化语义与工具链对照,不要求底层真的按页分配。
|
||||
- **增长**:只增长,不缩小;旧地址偏移不变;新加入的字节全部为零;不得突破区域/运行全局限额。
|
||||
- **读取**:`m[i] -> Int(0..255)`;越界、非法整数或已失效区域均是类型/存储错误。
|
||||
- **写入**:`m[i] = b`,`b` 必须是 0..255 的整数,不隐式截断;越界拒绝。
|
||||
- **相等性**:两个 Memory 引用的相等比较区域身份;不比较全部字节;Memory 不定义大小排序。
|
||||
- **序列化**:不允许将活动 `Memory` 句柄伪装成数值或跨进程永久有效地址。允许显式**复制字节内容**为快照,需限额;跨进程传输需要显式复制或另立经过授权的共享内存协议。
|
||||
- **生命周期**:第一版只在单次执行内存在;由 Runtime 保持活动句柄可达性,引用消失后可回收,至少在运行结束时回收全部区域。**不增加用户可见的 `free(region)`**,以免引入悬空区域句柄。用户的 `malloc/free` 管理的是**区域内部的块**。
|
||||
- **内存安全范围**:Runtime 保证单次操作不访问该 Memory 区域之外;不保证用户自编分配器不会越块、重复 free、释放后复用错误。
|
||||
|
||||
此设计不要求新增独立的 `Pointer` 原始类型。**首版指针可由 `Memory` 引用和 `offset` 组合表达**;是否将这一对优化为一等 `Pointer`,由实际性能与调用需求决定,而非现在先引入语法。
|
||||
|
||||
### 4.3 Resource:与外部世界交互的独立对象
|
||||
|
||||
`ResourceHandle` 是由可信平台/Provider 签发的不透明句柄,代表文件、控制台流、网络连接或专门设备;含授权访问模式与生命周期。**它不是 `Memory`,不能索引成任意 RAM,也不能通过伪造整数创建。**
|
||||
|
||||
其常见操作是 `open/read/write/close`,但具体合法操作取决于资源接口。操作系统错误、EOF、部分读写、超时、取消都需要保留独立结果和错误语义。不得假定一次 `read` 总能填满缓冲区、一次 `write` 总能输出所有字节。
|
||||
|
||||
### 4.4 为什么要划分,而不是单一“万能地址空间”
|
||||
|
||||
`Memory` 代表运行时可管理的存储;`Resource` 代表外部效果。普通内存读写是局部状态改变,终端输入/网络读取可能随外部世界变化。把二者统一为可以随意解引用的指针,会隐藏权限和不可重复性,也会使知识证明与回放语义失真。
|
||||
|
||||
## 5. 可编程动态内存:到底哪些必须内置
|
||||
|
||||
### 5.1 最小计算内核操作
|
||||
|
||||
这里定义**能力**,不冻结最终函数名:
|
||||
|
||||
| 基本能力 | 候选用户接口 | 运行时职责 |
|
||||
|---|---|---|
|
||||
| 创建区域 | `memory.new(initial_pages, max_pages)` | 签发不可伪造引用,检查预算并分配零初始化区域 |
|
||||
| 查询大小 | `memory.size(m)` | 返回页数,不泄露真实宿主指针 |
|
||||
| 增长区域 | `memory.grow(m, pages)` | 原有字节与偏移不变,新字节为零;成功返回原页数,失败返回可区分错误 |
|
||||
| 读取一字节 | `m[i]` | 边界检查,读取 `0..255` |
|
||||
| 写入一字节 | `m[i] = b` | 边界和值域检查,更新同一 Memory 区域 |
|
||||
|
||||
`memory.new/size/grow` 是操作而非关键词;`m[i]` 利用当前已有的读取表达式,`m[i] = b` 只扩展现有赋值目标语义。这是此 RFC 要求的**唯一用户可见表达式语法增量**。
|
||||
|
||||
`len(m)` 可在另一个 ADR 中统一成字节长度而非另设 `memory.size`;这里暂不混淆“页数”和“字节数”。是否以页或字节为 `grow` 单位也应通过 ADR 固定,本文的原型推荐页。
|
||||
|
||||
### 5.2 位置赋值(Place)如何实现
|
||||
|
||||
对左值 `x` 与 `m[i]`,**赋值含义不同但遵循一个语法规则**:
|
||||
|
||||
- `x = value`:仍重新绑定当前函数帧局部变量,现有语义不变。
|
||||
- `m[i] = byte`:先按既有从左到右顺序求值 `m` 与 `i`,检查目标为 `Memory` 并检查偏移和值,然后更新同一内存实例。
|
||||
- `list[i] = value`:**不在第一个阶段承诺**。是否将列表修改为值更新/可变对象更新,要另行定义,不能顺便改变现有列表复制与索引语义。
|
||||
- `obj.field = value`:首版也不必实现。只有在明确记录值/对象语义后再拓展。
|
||||
|
||||
表达式编译结构的最小增量是 `Assign { target: PlaceExpr, value: Expr }`,其中 `PlaceExpr` 初期只有 `Local(name)`、`Index(base,index)`;索引目标仅接受 Memory,其他索引仍只读。无需新 Markdown 节点。
|
||||
|
||||
### 5.3 自定义指针与分配器
|
||||
|
||||
`malloc(n)` 在 `.mq.md` 标准库里持有一个区域 `m` 和元数据指针 `next`,向调用方返回区域内偏移(或 `{region:m, offset:p}` 组合值)。`free` 仅将已分配块放回空闲链表;若空间不足,调用 `memory.grow`。对齐、大小、空闲块链表、合并策略都由 `.mq.md` 编写,**不要求 Rust 为每一种分配算法做特殊 HostOp**。
|
||||
|
||||
例:块格式 `[size:u32][next:u32][flags:u8][payload...]`。首版只支持按字节读取,`u32` 的编码/解码可以使用四次索引、`//` 和 `%` 在 `.mq.md` 中实现。若该路径确实过慢,后来可增加 `load_u32` 等**语义等价、可验证的加速实现**,不能因此把完整分配器写到 Host。
|
||||
|
||||
分配器可建立:Arena、first-fit、buddy、slab、链表/图、字符串字节编码、编译器符号表、虚拟 CPU 的寄存器文件等。**内存块的逻辑安全由标准库分配器定义**;调用者拿到任意偏移后仍有越块写的能力,这属于明确的低层风险。
|
||||
|
||||
### 5.4 内存模型形式定义(草案)
|
||||
|
||||
令 `R` 是区域身份集合,`M[r]` 是 r 区域的字节序列,`len(M[r])` 是其有效长度。
|
||||
|
||||
- `read(r,i) = M[r][i]`,前提 `0 ≤ i < len(M[r])`。
|
||||
- `write(r,i,b)` 将 `M[r][i]` 更新为 `b`,前提 `0 ≤ b < 256` 且 `i` 合法;其余字节不变。
|
||||
- `grow(r,k)` 要求 `k ≥ 0`,若预算允许,尾部追加 `k × PAGE` 个零字节,旧位置的字节值不变;否则返回显式失败且区域不变。
|
||||
- 不同区域互不别名:`r1 != r2` 意味着修改 `M[r1]` 不影响 `M[r2]`。
|
||||
- `m2 = m1` 若 m1 为 Memory,则二者指向同一 `r`;若为普通 `List`,原有值语义不变。
|
||||
|
||||
实现必须防止巨大 BigInt 偏移到 `usize` 转换时溢出、`offset + length` 整数溢出、预算计算溢出。任何错误不得 panic、越界读写或产生非确定性内存破坏。
|
||||
|
||||
## 6. 非线性存储、寄存器与加速的恰当位置
|
||||
|
||||
### 6.1 非线性内存不需要独立关键字
|
||||
|
||||
- **逻辑非线性结构**(链表、树、图):在一个或多个字节区域中存储偏移/引用即可;不要求新类型。
|
||||
- **物理非连续内存**:运行时可以用分页、稀疏映射或 Vec 实现同一线性语义;用户无需知道物理页布局。
|
||||
- **多设备内存**:未来可能带有设备特定权限与同步约束;不要将 CPU 内存随意解释成 GPU/QPU 地址。优先经专用受控资源/后端接入。
|
||||
|
||||
### 6.2 不加入真实 CPU 寄存器接口
|
||||
|
||||
当前局部变量与表达式临时值已经构成“虚拟寄存器”意义上的计算槽。编译到机器码时,可以由后端分配硬件寄存器。用户要模拟寄存器堆,可以用 Memory 自己实现;特殊硬件寄存器则是设备 I/O,而不是普通变量。
|
||||
|
||||
### 6.3 并行和原子操作
|
||||
|
||||
本 RFC 的内存规则初始定义为**单线程或按明确顺序执行**。绝不能声称 `m[i] = m[i]+1` 在多线程之间自动原子。未来若允许多个执行任务共享 Memory,必须另行设计同步/原子操作、数据竞争行为和访问序;也可以先采用无共享可变内存的任务隔离。新增原语应依据不可由当前操作正确组合而来的能力,而非先堆锁/屏障关键字。
|
||||
|
||||
### 6.4 GPU/QPU 不由普通 Memory 自动获得
|
||||
|
||||
Memory 支持字节缓冲区与数据布局,但 GPU 计算仍需要执行后端与数据传输,真实量子资源则不是可随意读取的字节地址。Marqdo 可用独立后端/服务处理硬件特有语义,内核应保持硬件无关。
|
||||
|
||||
## 7. 系统交互:能否不依赖 Rust/Python 实现高级功能?
|
||||
|
||||
### 7.1 必须给出的诚实答案
|
||||
|
||||
**可以**:大量系统功能由 Marqdo 标准库直接编写,包括文本与字节编码、目录遍历算法、协议解析、HTTP 报文、部分网络协议逻辑、缓存、日志格式、CLI 交互逻辑、文件格式读写、数据库协议客户端中的高层逻辑。程序运行时无需执行 Python 或 JavaScript,也不需要为每个高级库函数增加 Rust 分支。
|
||||
|
||||
**不能无条件做到**:如果当前运行方式是一个 Rust 编写的解释器,那么跨越用户态与操作系统的**最后一步**必须由解释器内建的系统适配器、已授权插件/worker、或其他可信底层机器代码执行。Markdown 源码不能在无相应运行环境、系统调用 ABI 和权限的情况下直接控制硬件。即使将来 Marqdo 编译成独立原生二进制,它也仍通过操作系统 ABI 或设备驱动与外界交互,只是**不再需要 Rust 解释器常驻运行**。
|
||||
|
||||
**不可混淆的四种独立性:**
|
||||
|
||||
| 层面 | 能否达到 | 含义 |
|
||||
|---|---|---|
|
||||
| A. 算法独立 | **可以,近期目标** | 排序、内存分配、解析器等算法均在 `.mq.md` 中 |
|
||||
| B. 标准库独立 | **可以,近期目标** | 新标准库函数不修改 Rust Runtime/HostOp,单独版本发布 |
|
||||
| C. 系统接口独立 | **有条件可以** | 已有的 OS 原语/权限可组合构造新功能;新增设备/OS 原语需相应适配器,但不改语言核心 |
|
||||
| D. 运行时/宿主实现独立 | **中长期目标** | 独立编译成机器码/Wasm、或自举解释器,不依赖 Rust 解释器运行;仍依赖 OS ABI/硬件 |
|
||||
|
||||
**这里的“无需修改宿主”不是“无底层实现”。** 应将“高级逻辑用 Marqdo 实现”与“最后一跳由可信 OS adapter 执行”区分清楚。
|
||||
|
||||
### 7.2 最小系统资源操作模型
|
||||
|
||||
建议**不增加任何 I/O 关键字**,而以可注册、版本化的 `Resource` 接口提供 OS 能力。候选调用形式(未实施):
|
||||
|
||||
- `resource.open(kind, descriptor, rights) -> ResourceHandle`
|
||||
- `resource.read(handle, memory, offset, max_bytes) -> count | EOF/error`
|
||||
- `resource.write(handle, memory, offset, bytes) -> count | error`
|
||||
- `resource.close(handle) -> Result`
|
||||
|
||||
其中:
|
||||
|
||||
1. `kind` 例如 `stdio`、`file`、`socket`,必须由可信平台适配器或已安装的 Provider 明确注册,**不能因为传入任意字符串就自动获得能力**。
|
||||
2. `descriptor` 是声明请求的结构化参数,例如项目内文件名、网络目标,必须验证并授权;绝非任意可解引用真实指针。
|
||||
3. `rights` 是请求的权能;实际授权为“请求 ∩ 操作者 grant ∩ Provider 支持”,拒绝必须显式报错。
|
||||
4. `read/write` 通过 Memory 区域交换原始字节,边界检查和缓冲区借用由 Runtime/Adapter 保证;不得允许 I/O Adapter 长期保存未固定的宿主 Rust 指针。
|
||||
5. 区分 EOF/0 字节暂时无进展/取消/系统错误;支持部分读写,标准库负责重试与缓冲。
|
||||
6. `close` 撤销对应资源句柄;已关闭句柄、重用 ID 必须借助 generation/注册表检查,不能错误访问新资源。
|
||||
7. 对设备特殊操作,不强行滥用 `read/write`:必要时可由具体 Provider 暴露专门的**版本化、授权操作**;不在通用内核中新增一个无条件允许任意 syscall 的 `control` 后门。
|
||||
|
||||
`resource.*` 属于**平台接口层**而不是纯计算内核;即使调用看起来是普通函数,执行前必须检查权限。现有 Host 能力检查可成为迁移入口,但不应为每个新高层标准库函数创建新的 `HostOp` 变体。
|
||||
|
||||
### 7.3 应该怎样真正避免“每扩展一次就修改 Host”?
|
||||
|
||||
将目前的固定分派进一步分成:
|
||||
|
||||
```text
|
||||
普通 Marqdo 标准库函数
|
||||
├── 调用 Marqdo 语言计算原语(无需 Host)
|
||||
└── 仅在外部交互时使用稳定的 Resource ABI
|
||||
│
|
||||
Capability Gate
|
||||
│
|
||||
Provider/Platform Registry
|
||||
│
|
||||
┌────────────────┴───────────────┐
|
||||
▼ ▼
|
||||
标准平台适配器 独立第三方 Provider
|
||||
文件/终端/网络 新设备/专用系统能力
|
||||
```
|
||||
|
||||
关键是**一次性稳定可扩展的外部操作契约**,而不是一张不断增长的 Host 功能目录。
|
||||
|
||||
现有 C ABI v1、worker 与 `CallableRegistry` 应先尽量复用;新接口可在不破坏 v1 的条件下增补版本化注册信息。如需升级,必须有兼容矩阵与迁移机制。**Provider 负责它新增的原始能力;`.mq.md` 标准库负责高级功能。**
|
||||
|
||||
### 7.4 两种系统访问路线及取舍
|
||||
|
||||
| 路线 | 优点 | 风险/限制 | 建议 |
|
||||
|---|---|---|---|
|
||||
| 能力化 `Resource` ABI | 统一权限与跨平台行为;适合安全分层 | 新的真正 OS 原语仍需要 Adapter/Provider | **默认且必须支持** |
|
||||
| 直接的原生 syscall/FFI 边界 | 能较快覆盖大量平台特性 | 平台/ABI 依赖、地址搬运、权限绕过、维护和可验证性问题 | 仅可信、本机、显式授权的可选高级接口;默认 pure/strict 禁止 |
|
||||
|
||||
在 Linux 等平台,即使“直接 syscall”,也要处理目标架构调用约定、数字差异、结构体布局、指针与长度、errno 以及 OS 策略;Windows/macOS 并不提供与 Linux 一模一样的 syscall ABI。不得将其伪装成普适的语言核心。
|
||||
|
||||
**无悖论地满足用户目标的办法**:向 Marqdo 暴露一个小而稳定的**操作系统能力边界**,然后让标准库以 Marqdo 自己的程序逻辑组合出系统高级功能。这是可实现的强独立性;完全消灭系统边界不是现实目标。
|
||||
|
||||
### 7.5 端到端例子 A:完全用 Marqdo 构建交互式 CLI
|
||||
|
||||
**先决条件**:可信平台适配器提供 stdin/stdout 的已授权资源,Marqdo 运行时支持 Memory 操作。
|
||||
|
||||
1. 标准库 `io.read_line` 在 `.mq.md` 中申请/增长缓冲区,循环调用资源 `read`,查找换行字节。
|
||||
2. 标准库 `text.utf8_decode` 用 `.mq.md` 处理 UTF-8(正确处理非法序列;编码策略可配置)。
|
||||
3. 用户程序将文本解析为数字,用现有 `Int` 算术和条件循环处理。
|
||||
4. 标准库 `io.write_text` 使用 `.mq.md` 的 UTF-8 编码,再经 `write` 输出字节。
|
||||
|
||||
**全流程无需 Python/Node,也不需要 Host 特地实现 `input()`。** 然而最底层 stdin/stdout 仍由 OS 适配器调用操作系统接口。
|
||||
|
||||
### 7.6 端到端例子 B:新增 HTTP 客户端能力
|
||||
|
||||
**先决条件**:系统适配器已经提供经授权的网络连接资源、DNS/地址解析的可用机制,以及必要的加密能力或可插拔 TLS Provider。
|
||||
|
||||
1. 标准库构造 HTTP 请求行、首部和消息体的字节序列。
|
||||
2. 使用 `resource.write` 处理部分发送;使用 `resource.read` 实现缓冲接收。
|
||||
3. `.mq.md` 解析状态行、首部、chunked 编码等;实现重试、缓存、超时逻辑。
|
||||
4. 测试 HTTPS 时,必须使用可靠 TLS 实现(可由已审计 Provider 提供;“不依赖外部语言写协议逻辑”不意味着应该自己手写未经验证的 TLS)。
|
||||
|
||||
网络的“最后一跳”必须存在,且不同平台的 TCP/TLS 能力可能需要不同 Provider;不能只靠 Memory 推导出 socket。
|
||||
|
||||
### 7.7 端到端例子 C:科研加速与 QPU
|
||||
|
||||
- 经典数值模拟:`.mq.md` 可以在 Memory 中构造矩阵、压缩存储、数据布局算法,后续由独立计算后端加速。
|
||||
- QPU:量子设备是具有权限、调度与门集约束的资源,不是普通 Memory;通过专用 Provider/Backend 提交计算、接收结果。
|
||||
- 这种接口可保持标准库和上层代理/知识文档不绑定 Rust 宿主,但并不能保证算法自动获得 GPU/QPU 性能或精确的量子物理语义。
|
||||
|
||||
## 8. 安全、权限与错误模型
|
||||
|
||||
| 保护目标 | 首版保证 | 明确不能保证 |
|
||||
|---|---|---|
|
||||
| 虚拟 Memory | 区域越界、非法索引、增长限额、无效句柄失败 | 区域内 malloc/free 的块级内存安全 |
|
||||
| 不透明 Resource | 授权身份、访问模式、资源关闭和 generation 检查 | 平台适配器/不可信原生插件本身绝对安全 |
|
||||
| pure | 内部算术、内存与受预算控制的逻辑计算;不获得外部 I/O | 允许直接 syscall 或任意外部访问 |
|
||||
| project | 对批准的文件/终端/网络资源进行操作,使用能力化 API | 等价于 OS 级进程沙箱 |
|
||||
| strict | 维持现有 fail-closed 策略;新 I/O Provider 必须另行经过 OS 级安全验收 | 当前版本自动支持所有外部能力 |
|
||||
|
||||
进一步要求:
|
||||
|
||||
- 全局/单区域内存预算、增长次数与 fuel 限额、操作可取消性;内存失败不得半成功。
|
||||
- 路径资源应使用受控根路径和可验证的 OS 打开策略,避免字符串检查后再打开造成 TOCTOU;网络权限需要处理名称解析/重绑定等问题。
|
||||
- 插件/worker 不因拥有 API 名字自动获得权限;原生进程内插件不能被当作不可信沙盒。
|
||||
- Memory/Resource 活动句柄不应直接跨线协议序列化;跨边界传输使用字节复制、快照或显式授权的资源传递协议。
|
||||
- 诊断保持来源 Markdown 节点位置;可选追踪读取/写入的逻辑区域、偏移和长度,不能将外部文件秘密或原始敏感字节无条件写日志。
|
||||
|
||||
## 9. 与当前 Marqdo 语法和知识编译的结合
|
||||
|
||||
这一方案故意**只修改最少的表达式语义**。
|
||||
|
||||
- `##` 函数、`**` 执行、`*` 返回、Markdown 列表与引用规则继续维持规范。
|
||||
- `[]()` 是**文档/知识的静态寻址**;动态 Memory 的字节偏移属于执行期位置,两者不强行共享 URI 语法。
|
||||
- 普通 GFM 表格是文档值;可用于初始化/展示内存快照,但不会自动变成内存分配器。
|
||||
- `memory.new(...)` 与 `resource.open(...)` 都可按普通函数调用表达;不新增 `memory`、`pointer`、`asm`、`register` 这样的语法关键字。
|
||||
- 内存状态变化可选输出可读 GFM Table 供 View/Proof 使用;证明必须以抽象执行语义为依据,而不是假设表格文本本身就是执行存储。
|
||||
|
||||
示例(**候选语法,当前尚不可运行**):
|
||||
|
||||
````markdown
|
||||
## 研究一个字节缓冲区
|
||||
|
||||
此函数演示“可引用的存储区域”与普通 Markdown 叙述并存。
|
||||
|
||||
**区域 = memory.new(1, 4)**
|
||||
|
||||
**`区域[8] = 42`**
|
||||
|
||||
**另一个名字 = 区域**
|
||||
|
||||
**`另一个名字[9] = 99`**
|
||||
|
||||
*区域[8] + 区域[9]*
|
||||
````
|
||||
|
||||
预期结果 141。注意这里 `另一个名字 = 区域` 分享 Memory 身份;一般列表/表格并不因此被改造成共享字节缓冲区。
|
||||
|
||||
## 10. 可行性和备选方案评定
|
||||
|
||||
| 设计 | 语言自主性 | 高层开发便利性 | 性能潜力 | 安全/复杂度 | 结论 |
|
||||
|---|---|---|---|---|---|
|
||||
| 继续只扩充 HostOp | 低 | 表面高 | 取决于 Rust 实现 | Host 随功能膨胀 | 否决为主要路线 |
|
||||
| 只有托管对象堆/GC | 中 | 高 | 对一般数据不错 | 需要身份/生命周期语义 | 保留现有托管值,但不足以满足低层目标 |
|
||||
| **多区域字节内存 + 托管值 + Resource** | **高** | **高** | 后端可优化,字节操作起初较慢 | 明确的区域边界;块级风险可见 | **推荐** |
|
||||
| 任意裸指针 + 直接机器码 | 极低平台独立性 | 低 | 最高上限 | 高风险、难验证 | 只可选的可信原生扩展 |
|
||||
| 纯 GFM 表格作堆内存 | 低至中 | 对展示很高 | 非常低 | 动态状态/身份混乱 | 否决 |
|
||||
|
||||
该推荐不是宣称“最优”已经被数学证明,而是在现有语法、实现成本、扩展性与安全边界下的**工程选择**。
|
||||
|
||||
## 11. CLI/Runtime/标准库仓库的具体职责
|
||||
|
||||
### 11.1 `marqdo-cli`:保留小而稳定的内核
|
||||
|
||||
- `marqdo-value`:新增不可伪造的 Memory 值表示;必要时添加 ResourceHandle,注意现有 PluginHandle 不能随意直接复用为 OS 资源句柄。
|
||||
- `marqdo-expression`:`AssignTarget` 的 Local / Index 变体;保留原有表达式读取。
|
||||
- `marqdo-frontend`:更新表达式 AST 对应 IR 和定位信息;不改变 Markdown 语义映射。
|
||||
- `marqdo-runtime`:Memory 实例生命周期/增长/边界检查/预算;执行位置写入;参考解释器。
|
||||
- `marqdo-host`:逐步提供可注册资源操作的能力边界;必须保持旧 `HostOp` 兼容窗口,不能一次砍断所有旧调用。
|
||||
- `marqdo-package` / CLI:发布标准库兼容信息,继续利用已存在的 path/Git pinned lock;资源授权仍由真实操作者提供。
|
||||
|
||||
### 11.2 独立 `marqdo-stdlib`:以 Marqdo 实现通用功能
|
||||
|
||||
建议独立仓库 `cflmy/marqdo-stdlib`,首期目录:
|
||||
|
||||
```text
|
||||
marqdo-stdlib/
|
||||
core/ # 字节编码、整数工具、可恢复错误封装
|
||||
memory/ # arena, allocator, free-list, binary codecs
|
||||
collections/ # list/tree/graph/hashmap 等数据结构算法
|
||||
text/ # UTF-8, parsing
|
||||
io/ # buffered readers/writers, read_line
|
||||
data/ # csv, json, 自定义格式
|
||||
tests/ # 以 marqdo run/call 执行的标准库回归程序
|
||||
```
|
||||
|
||||
高层函数应在 `.mq.md` 里,按普通模块导入。发行版携带一个经测试的 stdlib 版本,支持离线运行;单独发布更新时,要求明确的 Runtime 兼容范围与锁定。标准库可有原生加速 Provider,但不能把它作为常规功能的唯一实现路径。
|
||||
|
||||
### 11.3 独立 Platform Providers
|
||||
|
||||
系统 Adapter 可以由 Rust、C、汇编或将来的 Marqdo 原生编译代码实现。当前 CLI 不应为普通库函数增加 HostOp;对真正的新 OS 原语,若标准 Provider 不支持,用户必须安装/编写扩展 Provider,并获得相关权限。**不是每个新设备都能只靠修改 `.mq.md` 获得访问能力。**
|
||||
|
||||
## 12. 分阶段实施(范围严格受控)
|
||||
|
||||
### R0:冻结设计与当前基线(无新功能)
|
||||
|
||||
- 此 RFC 和对应 ADR 经审阅,确认 `Memory` 值语义、`index assignment`、错误类别、线协议边界及安全模式。
|
||||
- 修复已有跨平台 CI/安全收敛问题;固定解释器行为的回归样例。
|
||||
- 建立一页对照表:新增语义是否需要 `HostOp`、新关键字、新 Markdown 节点。首版目标均为“不需要”。
|
||||
|
||||
### R1:最小可寻址存储(核心里程碑)
|
||||
|
||||
- `Memory` 区域:创建、大小、增长、读取一字节、写入一字节;实现总/单区域预算、错误恢复与 trace。
|
||||
- 支持 `**\`m[8] = 42\`**` 这类现有 Markdown 负载中的索引赋值。
|
||||
- 不开放裸指针、不引入新寄存器、不改旧 List/Map/Table 的赋值与复制行为。
|
||||
- 运行跨区域隔离与别名测试;报错不能导致原有区域被污染。
|
||||
|
||||
### R2:Marqdo 自举的内存与算法库
|
||||
|
||||
- `.mq.md` 顺序分配器;随后 free-list 可复用分配器。
|
||||
- 不修改 Runtime/Host 编写并执行:可增长链表、二叉树、简单虚拟 CPU、UTF-8/二进制解析器。
|
||||
- 提供内存利用率、分配吞吐、碎片率、执行时间及解释器 fuel 的测量;避免只用“功能跑通”证明成熟度。
|
||||
|
||||
### R3:最小系统资源闭环
|
||||
|
||||
- 在项目模式先实现 `stdio` 与受控项目文件的 `open/read/write/close`;**功能可用性不代表 strict 安全已通过**。
|
||||
- `.mq.md` 标准库实现 `read_line`、`write_text`、文件二进制复制与 UTF-8 编码。
|
||||
- 资源注册层允许 Provider 扩展;验证一个第三方资源端点不修改解释器编译即可接入(是否需要实现新的系统适配器需明示)。
|
||||
- 对当前 `HostOp` 逐步适配,保留兼容期,不能强行把所有第三方都迁移到新的不成熟 ABI。
|
||||
|
||||
### R4:可选增强(不阻断基础版本)
|
||||
|
||||
- 索引/字段赋值的一般化、字节多宽度原语及位运算、快照、增量编译、优化执行器。
|
||||
- 需要共享内存并行时,再设计原子/同步与数据竞争语义。
|
||||
- 独立网络/TLS、设备、GPU、QPU 后端与原生编译目标。
|
||||
|
||||
**必须遵循的顺序**:R1 完整正确 → R2 验证计算自主性 → R3 证明外部交互可组合;不先把 R4 的设备能力和并发类型系统塞入 CLI。
|
||||
|
||||
## 13. 端到端验收清单
|
||||
|
||||
| 编号 | 验收项目 | 判定标准 |
|
||||
|---|---|---|
|
||||
| T1 | 语义一致性 | 旧的局部赋值与 List/Map/Table 行为完全不变,旧 Markdown AST golden 通过 |
|
||||
| T2 | Memory 隔离 | `m1[0]=x` 不影响 `m2[0]`;`m2=m1` 后写入可相互观察 |
|
||||
| T3 | 增长与错误 | 成功增长保留旧字节且新字节为 0;负索引、越界、超限增长、巨大 BigInt 均明确失败且不破坏状态 |
|
||||
| T4 | 自建分配器 | `.mq.md` 实现 malloc/free,释放块重复利用;高低地址和碎片测试通过 |
|
||||
| T5 | 高级数据结构 | 不改 Rust 核心,仅新增 `.mq.md` 实现可增长链表、树、二进制协议解析器 |
|
||||
| T6 | 标准库独立升级 | 旧 CLI 二进制保持不变,安装一个与之兼容的新 `.mq.md` 库并成功使用 |
|
||||
| T7 | 终端闭环 | `marqdo run` 仅使用标准库加基础 OS Adapter,完成多轮输入/输出,无 Python/Node 调用 |
|
||||
| T8 | 系统资源授权 | 无 grant 无法 open;已关闭/伪造 handle 无法访问;项目路径无法越权;所有拒绝有诊断和必要审计 |
|
||||
| T9 | 安全模式 | `pure` 不允许外部 I/O;当前 `strict` 不自动放开新 Provider;真实隔离与失败路径有测试 |
|
||||
| T10 | 扩展接口 | 新高层 I/O/协议函数不增加任何 HostOp;新 Provider 通过约定的独立注册/版本机制接入 |
|
||||
| T11 | 兼容/性能 | Windows/macOS/Linux 的支持矩阵真实披露;可定量报告字节访问吞吐、内存增长和 Python/Rust baseline 差距 |
|
||||
| T12 | 文档与证明 | Memory 状态可导出可读快照;执行状态引用源 Markdown span;不将数据表格误认成内存本体 |
|
||||
|
||||
## 14. 需要 ADR 决断而不是隐含假设的开放问题
|
||||
|
||||
本 RFC 采用一条**可实施的推荐路线**,但下列决策会改变公开语义,须单独冻结:
|
||||
|
||||
1. **页语义**:推荐 64 KiB 页、只增长;是否与外部目标标准保持一致,预算如何计入已保留与已提交物理内存。
|
||||
2. **Memory 的值/调用传递**:推荐复制引用保持身份,区域可达则存活;禁止隐式跨线协议序列化。
|
||||
3. **Memory 引用寿命**:推荐首版只在一次执行期间有效,采用自动回收或至少结束时回收;内部块分配是 stdlib 职责。
|
||||
4. **Place AST**:推荐首版只增 Local + Memory Index;其他集合的索引赋值须另行评议。
|
||||
5. **OS Resource 最小 ABI**:是否先使用现有插件注册机制包装,而不立即新增第二套并行插件协议。
|
||||
6. **跨平台支持范围**:纯计算 API 保持统一;系统资源能力按平台明示 availability,不能假定所有设备/系统调用均存在。
|
||||
7. **实验数据**:R2 执行效率可能较低,是否加入 32/64 位宽读写及优化编译由实测结果决定,而非预设几十种关键字。
|
||||
|
||||
## 15. 最终建议与决策摘要
|
||||
|
||||
**建议通过的架构选择:**
|
||||
|
||||
- **语言核心**:保留现有托管值与控制流,新增**显式、多区域、可增长的字节 Memory**;用现有索引表达式扩展可写位置,不创建新的 Markdown 语法层。
|
||||
- **内存管理能力**:`Memory` 区域由 Runtime 提供并保护区域边界;`malloc/free`、块布局、链表、树、二进制编码等由 `.mq.md` 标准库自己构造。
|
||||
- **系统能力**:独立、授权、可替换的 `Resource` 边界;标准库在 Marqdo 中组合 I/O,OS 的最后一跳仍由可信本地 Adapter/Provider/编译后端承担。
|
||||
- **扩展性**:不为每个库函数增加 Rust `HostOp`;只有新增*真正不可由现有原语表达*的硬件/OS 能力,才需要扩展适配器。
|
||||
- **安全性**:区分区域边界安全、块级内存安全、进程隔离和 OS 权限;不能用“沙盒”一词掩盖缺少的保证。
|
||||
- **长期演进**:以此作为实现自主数据结构、字节码 VM、优化执行器与量子/科学计算后端的基础,但不许声称三个内存原语就能自动产生 GPU/QPU 加速。
|
||||
|
||||
**核心结论:** 如果“摆脱宿主语言依赖”指的是**程序逻辑、标准库算法和高级系统功能不依赖宿主语言来实现**,这条路线可以达成;如果指的是**无需任何底层可信机器代码和操作系统接口即可访问硬件**,则不可能。对真正的独立编程语言而言,恰当目标是**自己定义计算、自己构建高层库、清楚且可扩展地跨越不可消除的系统边界**。
|
||||
|
||||
---
|
||||
|
||||
## 参考资料(外部设计依据,不是 Marqdo 已实现功能的证据)
|
||||
|
||||
1. [WebAssembly Core Specification 3.0(2026-09-11)](https://www.w3.org/TR/wasm-core-2/):线性内存、内存增长、抽象机器、核心与环境 API 的分离;注意其保证为线性内存边界安全,不保证客户程序自己设计的内部分配器没有逻辑错误。
|
||||
2. [Rust `core` / `alloc` / `std` 文档](https://doc.rust-lang.org/std/):计算基础、分配与平台库的分层思路;并不意味着必须用 Rust 编写 Marqdo 的标准库算法。
|
||||
3. [Linux Kernel ABI documentation](https://docs.kernel.org/admin-guide/abi.html):OS 用户态接口的稳定性、适用范围和版本边界。
|
||||
4. [Linux Kernel ioctl interfaces](https://docs.kernel.org/driver-api/ioctl.html):设备特定能力并不都能归约为普通 RAM 读写,ABI 应明确数据布局与操作语义。
|
||||
5. [Marqdo 当前安全模型](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/security.md)及[插件 ABI v1](https://github.com/cflmy/marqdo-cli/blob/a113d894962d8c00eb02efb42e0e73f8b084a1fa/spec/plugin-abi.md)。
|
||||
+66
-7
@@ -71,7 +71,7 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
| 2 | Cross-platform strict `cfg` | **Fixed locally; native CI UNVERIFIED** | `#[cfg]` dispatch + non-Linux regression test |
|
||||
| 3 | Strict probe→launch race | **Fixed locally; GitHub Linux CI UNVERIFIED** | fail-closed CLI contract + probe-OK/launch-EAGAIN seam test |
|
||||
| 4 | Parallel security matrix isolation | **Fixed** | unique paths + helper `OnceLock`; 7/7 in parallel workspace run |
|
||||
| 5 | Markdown/power/control-flow corpus | **Done locally** | 160 AST, 44 semantic, 52 compile-negative, 12 runtime-negative fixtures |
|
||||
| 5 | Markdown/power/control-flow corpus | **Done locally** | 160 AST, 68 semantic, 52 compile-negative, 21 runtime-negative fixtures |
|
||||
| 6 | Condition header + source spans | **Fixed** | executable tails rejected; literal-inside-backtick mapping tests |
|
||||
| 7 | Independent Tool consumers | **Done locally** | `sample-view`/`sample-knowledge`/`sample-proof` E2E test |
|
||||
| 8 | lock v3 reproducibility | **Implemented; tamper test added** | `package_env` PASS |
|
||||
@@ -86,7 +86,7 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
|
||||
- Frozen Markdown profile `marqdo.md.v1` with 160 AST goldens and explicit
|
||||
CRLF byte semantics.
|
||||
- Native Rust interpreter with 44 semantic goldens, 52 compile-error fixtures,
|
||||
- Native Rust interpreter with 68 semantic goldens, 52 compile-error fixtures,
|
||||
and runtime error source attribution.
|
||||
- Native `**` power is pinned end-to-end: right associativity, unary-minus
|
||||
precedence, parentheses, float negative exponents, branch/loop conditions,
|
||||
@@ -109,6 +109,31 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
version verification.
|
||||
- Linux strict L1: Bubblewrap namespace supervisor, runtime probe, fail-closed
|
||||
launch, child self-verification, 7/7 adversarial matrix.
|
||||
- Addressable memory (RFC-CM P1, ADR-0022): opaque `Memory` values shared by
|
||||
handle identity, region isolation, zero-filled 64 KiB page growth, bounded
|
||||
byte read/write, `memory.new/size/grow`, `m[i] = byte`, run and per-region
|
||||
budgets, host/wire rejection, Analysis v2 target projection.
|
||||
- Recoverable `Result` convention (RFC-CM CM-A05, ADR-0024): success is
|
||||
`{ok:true,value:...}`, failure is `{ok:false,code,message[,details]}` as an
|
||||
ordinary Map; expected conditions return it, invariant violations stay
|
||||
fatal. A `.mq.md` first-fit/free-list allocator reuses blocks, splits free
|
||||
blocks, coalesces address-adjacent blocks and returns `alloc.no_space`
|
||||
without aborting.
|
||||
- CM-A07 Compat-A (ADR-0027): explicit `.mq.md` `io.写值` chain
|
||||
(`format.显示` → UTF-8 encode → `Memory` → `resource.write(stdout)`) matches
|
||||
builtin `打印` for the covered value set; builtin remains default and no
|
||||
alias is routed.
|
||||
- Resource streams (RFC-CM CM-A06, ADR-0025): opaque host-issued `Resource`
|
||||
handles with `resource.open/read/write/close`; bytes move only through
|
||||
`Memory`; stdout write is capability/output-budget checked and project-file
|
||||
read/write is path/grant checked; file writes buffer until `close` and
|
||||
never flush on drop; a `.mq.md` chunked copy loop reproduces a file without a
|
||||
Rust `copy_file`; strict refuses every resource open until a provider is
|
||||
verified. Semantic fixture 66 and CLI/host tests.
|
||||
- Minimal structure observation/construction (RFC-CM CM-A04, ADR-0023):
|
||||
value-semantics `list.append/concat`, `map.keys/has/get/set/remove`,
|
||||
Unicode scalar `text.codepoints`/`text.from_codepoint`, and non-negative
|
||||
`table.columns/row/cell`. Pure builtins; no mutation, no new syntax.
|
||||
|
||||
## 4. Open blockers and limits
|
||||
|
||||
@@ -121,25 +146,59 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
fixture lane, Unix-gated worker tests).
|
||||
4. `fs.write`, `net.request`, `process`, `clock`, `random` are not executed
|
||||
broker services; unsupported requests must return `capability.unavailable`.
|
||||
5. uv/npm/venv provisioning is not implemented. Only a pre-provisioned,
|
||||
5. Resource streams are a **P3 first slice** only: project-file read and
|
||||
buffered write plus stdout, but no directories, network, process, device,
|
||||
generic provider or stdlib `print` migration; strict refuses every
|
||||
`resource.open` until the provider passes the isolation lane.
|
||||
6. uv/npm/venv provisioning is not implemented. Only a pre-provisioned,
|
||||
digest-pinned directory with a regular-file executable is supported.
|
||||
6. No Windows/macOS RC archive exists from this host. The release-candidate
|
||||
7. No Windows/macOS RC archive exists from this host. The release-candidate
|
||||
workflow can build macOS on GitHub, but it has not run for this SHA.
|
||||
7. No release tag/binary publication. The release decision is **DO NOT
|
||||
8. No release tag/binary publication. The release decision is **DO NOT
|
||||
RELEASE** until the native CI matrix and a security-lane CI URL exist.
|
||||
|
||||
## 5. Evidence files
|
||||
## 5. RFC-CM compute-machine status
|
||||
|
||||
The two 2026-11-20 architecture RFCs are design input, not implemented
|
||||
behavior. This table records only what is in this tree now.
|
||||
|
||||
| RFC phase | State | Evidence / limit |
|
||||
|---|---|---|
|
||||
| P0 freeze (ADRs, spec deltas, baseline) | **P1 + partial P2 slice done** | ADR-0022 freezes CM-A01/CM-A02/CM-A03; ADR-0023 freezes CM-A04 (minimal List/Map/Text/Table observation and construction); ADR-0024 freezes CM-A05 (recoverable `Result` map convention); `spec/expression.ebnf`, `spec/evaluation.md`, `spec/value-wire.md`, `spec/analysis-v2.md`, `spec/security.md` updated; existing AST/value goldens unchanged |
|
||||
| P1 Memory + place assignment | **Implemented locally** | `Value::Memory`, run-scoped `MemoryStore` (64 KiB pages, 64 MiB total / 16 MiB region), `memory.new/size/grow`, `m[i]` read, `m[i] = byte`; semantic fixtures 45–47, runtime negatives 13–18, `examples/memory-buffer.mq.md`; evidence [`evidence/2026-11-20-memory-p1.md`](evidence/2026-11-20-memory-p1.md) |
|
||||
| P2 Marqdo-written allocator/data structures + minimal structure observation | **PARTIAL, gate NOT passed** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), and a quoted/escaped CSV row parser (68). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). Still missing: separate `marqdo-stdlib` repo, JSON floats, table group, EOF/partial stream contract, and a frozen-CLI closure lane. |
|
||||
| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict refuses all opens. Fixture 66 + CLI/host tests. CM-A07 is **Compat-A only**: `examples/stdlib` provides an explicitly imported `.mq.md` `format/utf8/io` chain whose output matches builtin `打印` for Unit/None/Bool/Int/Float/Text/List/Map/Table; builtin `打印` remains default, no alias routing, and Formula/Code/FunctionRef/PluginHandle formatting is still unobservable. Still missing: Compat-B/C, directories, network, generic provider registry. |
|
||||
| P4 Table logical/physical migration | **NOT IMPLEMENTED** | `Value::Table` remains `Vec<Vec<Value>>`; no stdlib table algorithms |
|
||||
| P5 third-party Provider / TCP / HTTP | **NOT IMPLEMENTED** | `fs.write`, `net.request`, `process`, `clock`, `random` remain unavailable broker services (see §4) |
|
||||
| P6 performance / native backends | **NOT IMPLEMENTED** | no bytecode/JIT/GPU/QPU work |
|
||||
|
||||
The P1 memory domain is pure computation: it needs no grant and is available in
|
||||
`pure`, `project` and `strict` (bounded by the memory budgets). It adds no
|
||||
`HostOp`, no Markdown role and no CLI command. `Memory` handles cannot cross
|
||||
the wire or a host/plugin/foreign boundary. Block-level safety of a
|
||||
user-written allocator is explicitly **not** claimed.
|
||||
|
||||
## 6. Evidence files
|
||||
|
||||
- [`evidence/2026-11-20-local-gate.md`](evidence/2026-11-20-local-gate.md) —
|
||||
earlier local gate and Windows autocrlf reproduction.
|
||||
- [`evidence/2026-11-20-strict-gc-8af7262.md`](evidence/2026-11-20-strict-gc-8af7262.md)
|
||||
— current VM lane, environment, full log and workspace test log.
|
||||
- [`evidence/2026-11-20-resource-stream-p3.md`](evidence/2026-11-20-resource-stream-p3.md)
|
||||
— CM-A06 first slice: stdout write, project-file chunked read, strict refusal,
|
||||
wire/host boundary rejection and test commands.
|
||||
- [`evidence/2026-11-20-resource-file-write-p3.md`](evidence/2026-11-20-resource-file-write-p3.md)
|
||||
— buffered project-file write, flush-on-close, buffer ceiling, grants and
|
||||
CLI write-then-read evidence.
|
||||
- [`evidence/2026-11-20-stdlib-print-compat-a.md`](evidence/2026-11-20-stdlib-print-compat-a.md)
|
||||
— CM-A07 Compat-A `.mq.md` print prototype, side-by-side output comparison,
|
||||
covered/uncovered value types and the Compat-B blocker.
|
||||
- [`evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md`](evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md)
|
||||
— G0..G12 matrix and RC decision.
|
||||
- [`security-gate.md`](security-gate.md) — mechanism, attack matrix and strict
|
||||
L1/L2 limits.
|
||||
|
||||
## 6. Reproduce
|
||||
## 7. Reproduce
|
||||
|
||||
```bash
|
||||
export PATH="$HOME/.cargo/bin:$PATH"
|
||||
|
||||
@@ -0,0 +1,183 @@
|
||||
# ADR-0022: Addressable `Memory` and Memory-index place assignment (RFC-CM P1)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md` (CM-A01/CM-A02/CM-A03),
|
||||
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (P0/P1), ADR-0007,
|
||||
ADR-0011, ADR-0017, ADR-0021
|
||||
Supersedes: the v1 rule that assignment targets are bare identifiers only
|
||||
Specs: `spec/expression.ebnf`, `spec/evaluation.md`, `spec/value-wire.md`,
|
||||
`spec/analysis-v2.md`, `spec/security.md`
|
||||
Evidence: `docs/evidence/2026-11-20-memory-p1.md`
|
||||
|
||||
## Context
|
||||
|
||||
RFC-CM-0001/0002 define three domains: managed values, addressable regions and
|
||||
capability resources. The first two are a language change; the third is an OS
|
||||
boundary and is deliberately **not** part of this ADR.
|
||||
|
||||
Before this decision the interpreter had no writable place other than a local
|
||||
slot. `Value` had no byte region, `a[i]` was read-only, and assignment was
|
||||
parsed only as `identifier = expression`. A `.mq.md` program therefore could
|
||||
not express an allocator, an encoded record, a free list or a byte protocol
|
||||
without a new Rust builtin for every algorithm. That is the closure gap
|
||||
RFC-CM-0002 §2 identifies.
|
||||
|
||||
The decision must not:
|
||||
|
||||
* turn managed `List`/`Map`/`Table` values into shared mutable objects;
|
||||
* expose a host pointer, `usize` address or procedural `free(region)`;
|
||||
* let an active handle cross the value wire or an ABI boundary;
|
||||
* add a Markdown node, keyword or `HostOp` for memory;
|
||||
* claim block-level memory safety for user allocators;
|
||||
* weaken `strict` or grant any external I/O.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. `Value::Memory` is an opaque, run-scoped identity
|
||||
|
||||
`MemoryHandle` (`crates/marqdo-value/src/value.rs`) is
|
||||
`(run_id, region_id, generation)` with private fields and a runtime-only
|
||||
constructor. It is not `Serialize`/`Deserialize`, is never built from user
|
||||
input, and carries no host address.
|
||||
|
||||
Copying the value, passing it as an argument and storing it in a list or map
|
||||
all copy the handle and preserve region identity. `==`/`!=` compare identity
|
||||
only; `Memory` has no ordering comparison. `display()`/`repr()` render the
|
||||
literal `Memory`, never the internal identity.
|
||||
|
||||
### 2. Regions are grow-only, bounded byte arrays
|
||||
|
||||
Frozen semantics (CM-A01/CM-A02):
|
||||
|
||||
| Property | Decision |
|
||||
|---|---|
|
||||
| Page size | 64 KiB logical page (`PAGE_SIZE`) |
|
||||
| Initial content | Every byte is `0` |
|
||||
| Growth | `memory.grow` only increases; old offsets/bytes unchanged; new bytes zero |
|
||||
| Failure atomicity | A rejected create/grow allocates, commits and mutates nothing |
|
||||
| Shrink/free | No shrink and no user-visible region free; regions are dropped with the run |
|
||||
| Address | `(region identity, non-negative offset)`; an integer alone is never a credential |
|
||||
| Read | `m[i] -> Int(0..=255)` |
|
||||
| Write | `m[i] = byte`, `byte` must be `Int(0..=255)`, no truncation |
|
||||
| Isolation | Different regions never alias; handle copies of one region do |
|
||||
| Visibility | `memory.size` returns committed pages, not host capacity or pointers |
|
||||
|
||||
Limits in `MemoryLimits`:
|
||||
|
||||
* `DEFAULT_MAX_TOTAL_BYTES = 64 MiB` committed across the run;
|
||||
* `DEFAULT_MAX_REGION_BYTES = 16 MiB` per region, which also caps declared
|
||||
`max_pages` at 256;
|
||||
* budget arithmetic uses checked `u64`/`usize` operations; allocation uses
|
||||
`try_reserve` so a host OOM is a structured `resource.exhausted` instead of
|
||||
a panic.
|
||||
|
||||
Limits are a runtime API (`marqdo_runtime::MemoryLimits`, `run_with_limits`)
|
||||
and are overridable on `run`/`call` with `--max-memory <BYTES>` (total) and
|
||||
`--max-region-memory <BYTES>` (per region). The defaults are stated in
|
||||
`spec/evaluation.md`. Both flags apply in `pure`, `project` and `strict`; they
|
||||
grant no external capability.
|
||||
|
||||
### 3. The only new writable place is a `Memory` index
|
||||
|
||||
`AssignTarget` gains `Index { base, index, span }` in the expression AST and
|
||||
`PlaceTarget` mirrors it in the IR. The surface syntax remains a `strong`
|
||||
node containing `target = expression`; no Markdown role, heading or list
|
||||
meaning changes.
|
||||
|
||||
* `x = value` still rebinds the current frame's local slot.
|
||||
* `m[i] = byte` evaluates **base, then index, then RHS**, exactly once each.
|
||||
All three checks (base is `Memory`, offset in range, byte in `0..=255`)
|
||||
happen before the single byte write, so a failed statement does not write.
|
||||
* `list[i] = value`, `map[key] = value` and `obj.field = value` are not
|
||||
defined. Member targets are a parse error; a non-`Memory` index base is
|
||||
`runtime.type`. No managed collection changes value semantics.
|
||||
* Nested index targets are accepted syntactically; the innermost base must
|
||||
still evaluate to a `Memory` value at run time.
|
||||
* Assignment remains statement-only; `x = y = 1` is still a parse error.
|
||||
|
||||
This is CM-A03.
|
||||
|
||||
### 4. Handles cannot cross the wire or the host boundary
|
||||
|
||||
* `to_wire(Memory)` emits only `{"type":"memory"}` so an accidental transfer
|
||||
is visible. `from_wire` rejects that tag with `WireError::Invalid`; a
|
||||
round-trip is intentionally impossible (no `wire_version` bump: the value
|
||||
shape is rejected, not newly accepted).
|
||||
* `Host::invoke` rejects any argument tree containing `Memory` before dispatch
|
||||
(`runtime.type`), so plugins, foreign kernels, math, file and print adapters
|
||||
never see a handle. A future explicit byte-snapshot operation must copy bytes
|
||||
into a managed value and is a separate ADR.
|
||||
* `HostOp` is unchanged. `memory.new/size/grow` and index read/write are
|
||||
interpreter operations, not capability-gated host effects.
|
||||
|
||||
### 5. Error categories are stable and distinct
|
||||
|
||||
| Situation | Code |
|
||||
|---|---|
|
||||
| Negative/unrepresentable offset, offset >= region length | `runtime.index` |
|
||||
| Unknown, stale or cross-run handle; invalid page count (negative, > budget declaration arithmetic) | `runtime.memory` |
|
||||
| Wrong operand type; byte outside `0..=255`; non-Memory index-assignment base | `runtime.type` |
|
||||
| Declared max, region budget or total budget exceeded | `resource.exhausted` |
|
||||
|
||||
No operation panics on malformed input or forged handles. `expect` is used only
|
||||
after the same handle has been validated in the same call frame.
|
||||
|
||||
### 6. Builtins are part of the language surface, not `HostOp`s
|
||||
|
||||
`memory.new(initial_pages, max_pages) -> Memory`,
|
||||
`memory.size(memory) -> Int`, `memory.grow(memory, pages) -> Int` are added to
|
||||
`marqdo-ir::builtins::BUILTINS` and `spec/evaluation.md`. They are pure
|
||||
(`effectful: false`): `pure` mode may use them, `strict` needs no new provider,
|
||||
and no capability grant is consulted.
|
||||
|
||||
### 7. Compatibility
|
||||
|
||||
* Local assignment IR keeps its `target` string in Analysis v2; an index
|
||||
target is an additive object (`{"kind":"index",...}`). Consumers that only
|
||||
handle locals continue to work.
|
||||
* Expression AST/statement serde gains an enum tag; this is a pre-1.0 internal
|
||||
contract documented in `spec/expression.ebnf`.
|
||||
* Existing Markdown AST goldens, value semantics, `List`/`Map`/`Table`
|
||||
behaviour and all old builtins are unchanged.
|
||||
* `Memory` is additive to the value type set. Old programs never produce it.
|
||||
ABI wire consumers see a new rejected tag, not a renumbered existing tag.
|
||||
|
||||
## Non-goals
|
||||
|
||||
* No `Pointer` value, raw pointer, `free(region)`, registers or machine code.
|
||||
* No index/field assignment for managed collections; no implicit mutation.
|
||||
* No multi-threaded sharing, atomics or data-race guarantees.
|
||||
* No serialization of live regions; no cross-process shared memory.
|
||||
* No `ResourceHandle`, `Stream`, files, network or OS syscalls. Those are
|
||||
CM-A05/CM-A06/CM-A09/CM-A11 and later RFC phases.
|
||||
* No block-level safety claim for a user-written allocator; the runtime only
|
||||
bounds a single access to its region.
|
||||
* No allocator or stdlib API in P1; `--max-memory` only bounds the runtime.
|
||||
|
||||
## Consequences
|
||||
|
||||
* A `.mq.md` program can now build byte encoders, arenas, free lists and
|
||||
virtual CPU register files without new Rust code. P2 (the Marqdo-written
|
||||
stdlib and closure fixtures) becomes testable.
|
||||
* The interpreter owns one `MemoryStore` per run; `run_with_limits` exposes
|
||||
the budget. This is runtime state, not a second `HostOp` table.
|
||||
* Trace gains `memory.new`, `memory.grow` and `memory.write` semantic events
|
||||
without exposing handles or byte secrets.
|
||||
* The remaining RFC decisions (CM-A04 and later) stay separate. In particular,
|
||||
generic structural observation/construction, recoverable `Result`, table
|
||||
algorithm migration and the resource/Stream ABI are **not** implemented by
|
||||
this ADR.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-memory-p1.md` records:
|
||||
|
||||
* parser/AST tests for index and rejected member targets;
|
||||
* runtime store tests for isolation, aliasing, zero-fill, atomic failed grow,
|
||||
negative/huge offsets, forged/stale handles, byte range and budgets;
|
||||
* semantic goldens `45`–`47` and runtime negatives `13`–`18`;
|
||||
* wire rejection test and host-boundary rejection test;
|
||||
* CLI example `examples/memory-buffer.mq.md` returning `141`;
|
||||
* Analysis v2 index-target projection;
|
||||
* `cargo fmt --check`, `cargo clippy -- -D warnings`, `cargo test --workspace`.
|
||||
@@ -0,0 +1,147 @@
|
||||
# ADR-0023: Minimal structural observation and construction (RFC-CM CM-A04)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`,
|
||||
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A04, P2),
|
||||
ADR-0007, ADR-0022
|
||||
Specs: `spec/evaluation.md`, `spec/expression.ebnf`, `spec/analysis-v2.md`
|
||||
Evidence: `docs/evidence/2026-11-20-structure-api-p2.md`
|
||||
|
||||
## Context
|
||||
|
||||
RFC-CM-0002 §3.5 identifies a closure gap: a `.mq.md` program can read a
|
||||
`List`/`Map`/`Table` and print it through the host, but it cannot generally
|
||||
**construct** the same structures dynamically or **observe** all of their
|
||||
logical parts. That makes `format`, JSON/CSV parsers and table algorithms
|
||||
impossible to write in `.mq.md` without adding a task-specific Rust function
|
||||
for each algorithm.
|
||||
|
||||
The current core already has:
|
||||
|
||||
* `List` literals, index reads (including negative offsets), `长度`;
|
||||
* `Map` record literals, `.field` reads, `长度`;
|
||||
* `Text` concatenation, character indexing and `长度`;
|
||||
* `Table` construction through `表(header, rows)`, `长度` (rows), display and
|
||||
wire round-trip.
|
||||
|
||||
It lacks: append/concat, dynamic map update/enumeration, Unicode scalar
|
||||
conversion, and table column/row/cell projection. These are observation and
|
||||
construction primitives, not algorithms. Sorting, filtering, joining,
|
||||
serialising and parsing remain standard-library work.
|
||||
|
||||
## Decision
|
||||
|
||||
Add a small, namespaced, **pure** builtin set. No new Markdown node, keyword,
|
||||
`HostOp`, capability or mutation semantics is introduced. All constructors
|
||||
return **new** values; an input collection is never modified in place. This
|
||||
keeps INV-02 and preserves existing `List`/`Map`/`Table` copy semantics.
|
||||
|
||||
### 1. `List`
|
||||
|
||||
| Signature | Semantics |
|
||||
|---|---|
|
||||
| `list.append(list: List, value) -> List` | New list with `value` at the end; the input list is unchanged. |
|
||||
| `list.concat(left: List, right: List) -> List` | New list containing `left` then `right`; both inputs unchanged. |
|
||||
|
||||
Neither aliases the other list's element storage: values are copied according
|
||||
to their normal value semantics (`Memory` handles keep region identity).
|
||||
|
||||
### 2. `Map`
|
||||
|
||||
| Signature | Semantics |
|
||||
|---|---|
|
||||
| `map.keys(map: Map) -> List` | `Text` keys in canonical sorted order (`BTreeMap` order, the order used by `display`). |
|
||||
| `map.has(map: Map, key: Text) -> Bool` | Whether the key exists. |
|
||||
| `map.get(map: Map, key: Text) -> Value` | Stored value, or `None` when absent. Use `map.has` when `None` is a legal stored value. |
|
||||
| `map.set(map: Map, key: Text, value) -> Map` | New map with the key set; the input map is unchanged. |
|
||||
| `map.remove(map: Map, key: Text) -> Map` | New map without the key (missing key is a no-op); the input map is unchanged. |
|
||||
|
||||
Keys are always `Text`; arbitrary-value keys are not introduced. `map.get`
|
||||
returning `None` is a *lookup result*, not the language's recoverable-error
|
||||
contract; a full `Result` convention remains CM-A05 and is not frozen here.
|
||||
|
||||
### 3. `Text`
|
||||
|
||||
`Text` remains a Unicode scalar-value sequence, not a byte array. These
|
||||
primitives make the existing representation observable without introducing a
|
||||
second text type or implicit UTF-8 conversion.
|
||||
|
||||
| Signature | Semantics |
|
||||
|---|---|
|
||||
| `text.codepoints(text: Text) -> List` | Unicode scalar values (`Int`) in code-point order. |
|
||||
| `text.from_codepoint(codepoint: Int) -> Text or None` | One-scalar `Text`; `None` for a surrogate or a value outside `0..=0x10FFFF`. |
|
||||
|
||||
UTF-8 encoding/decoding is **not** a core primitive. It can be written in
|
||||
`.mq.md` using bytes, `text.codepoints` and `text.from_codepoint`; this keeps
|
||||
the byte/code-point distinction explicit.
|
||||
|
||||
### 4. `Table`
|
||||
|
||||
`Value::Table` stays the logical value; its physical `Vec<Vec<Value>>` layout
|
||||
is not part of this contract. Indices are non-negative row/column coordinates
|
||||
(unlike `List` negative indexing) because a table cell is a document
|
||||
coordinate. Bounds failures are `runtime.index`.
|
||||
|
||||
| Signature | Semantics |
|
||||
|---|---|
|
||||
| `table.columns(table: Table) -> List` | Header names (`Text`) in column order. |
|
||||
| `table.row(table: Table, row: Int) -> List` | A new list of the row's cells in column order. |
|
||||
| `table.cell(table: Table, row: Int, column: Int) -> Value` | The cell value. |
|
||||
|
||||
No `table.set_cell`/`append_row` builtin is added. Rebuilding a table is a
|
||||
value-level operation that can be written with `表(header, rows)` once list
|
||||
construction exists; adding a Rust table algorithm would violate the closure
|
||||
goal.
|
||||
|
||||
### 5. Error and compatibility rules
|
||||
|
||||
* Wrong collection/text/table type or key type → `runtime.type`.
|
||||
* Table row/column outside the logical shape → `runtime.index`.
|
||||
* `text.from_codepoint` never throws for an invalid scalar; it returns `None`.
|
||||
* `map.get` never throws for a missing key; it returns `None`.
|
||||
* Existing `List`/`Map`/`Table` behavior, display, wire encoding and Markdown
|
||||
laziness are unchanged.
|
||||
* The builtins are `effectful: false`, available in `pure`, `project` and
|
||||
`strict`, and add no capability. They do not consume output budget.
|
||||
* Names are frozen as written; aliases are not added. Namespaced calls use the
|
||||
existing `Expr::Member` resolution path.
|
||||
|
||||
## Non-goals
|
||||
|
||||
* No `set_cell`, `append_row`, `sort`, `filter`, `join`, `group`, JSON/CSV or
|
||||
other algorithm builtins; those belong in `.mq.md`.
|
||||
* No mutable/aliased List or Map, no in-place mutation, no shared structural
|
||||
sharing protocol.
|
||||
* No arbitrary-value map keys, no ordered-map insertion semantics, no
|
||||
duplicate-key policy change.
|
||||
* No UTF-8/UTF-16 conversion primitive, no byte-array text type.
|
||||
* No `Result`/`try`/exception syntax; CM-A05 stays open.
|
||||
* No change to Table storage, GFM parsing, source spans or `ref` semantics.
|
||||
|
||||
## Consequences
|
||||
|
||||
* A `.mq.md` program can build maps and lists dynamically, enumerate keys,
|
||||
project table cells and encode Unicode scalars. This unblocks a Marqdo-
|
||||
written formatter, table filter/projection and a JSON/CSV parser without
|
||||
per-algorithm Rust.
|
||||
* The builtin table grows by 12 pure entries. The migration rule remains:
|
||||
only observation/construction primitives may enter core; algorithms do not.
|
||||
* Because constructors are value-returning, repeated `map.set` in a loop is
|
||||
O(n²) for a BTreeMap clone. That is acceptable for P2 semantics; a
|
||||
persistent/structural-sharing representation is a P6 performance decision,
|
||||
not a semantic one.
|
||||
* `map.get` returning `None` for absence is deliberate and documented; a
|
||||
future CM-A05 `Result` convention must not silently reinterpret existing
|
||||
programs.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-structure-api-p2.md` records:
|
||||
|
||||
* ADR/spec deltas and builtin metadata;
|
||||
* semantic fixtures for value-semantics list/map operations, code-point
|
||||
round-trips, table projection and invalid code points;
|
||||
* negative fixtures for wrong types and table bounds;
|
||||
* unit tests, `cargo fmt --check`, `cargo clippy -- -D warnings` and
|
||||
`cargo test --workspace`.
|
||||
@@ -0,0 +1,162 @@
|
||||
# ADR-0024: Recoverable result convention (RFC-CM CM-A05)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`,
|
||||
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A05, P2/P3),
|
||||
ADR-0023
|
||||
Specs: `spec/evaluation.md`
|
||||
Evidence: `docs/evidence/2026-11-20-result-and-free-list-p2.md`
|
||||
|
||||
## Context
|
||||
|
||||
A standard library cannot be written in `.mq.md` if every expected condition
|
||||
(no free block, invalid input, missing field, later end-of-file or a short
|
||||
read) must abort the whole run. The core has structured `RuntimeError`s with
|
||||
stable codes, but no way for an ordinary function to return a failure that the
|
||||
caller can inspect and handle in a normal branch. RFC-CM-0002 §3.6 recommends a
|
||||
`Result` data contract using existing `Map` values and control flow, without
|
||||
`try`/`catch` or exception syntax.
|
||||
|
||||
This ADR freezes the **data convention**, not a new runtime type. A separate
|
||||
core `Result` value, algebraic types or automatic propagation would be a
|
||||
larger change and is deliberately not introduced.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Shape
|
||||
|
||||
A recoverable result is an ordinary `Map`:
|
||||
|
||||
```text
|
||||
success: {ok: true, value: <any Value>}
|
||||
failure: {ok: false, code: <Text>, message: <Text> [, details: <Map>]}
|
||||
```
|
||||
|
||||
* `ok` is always present and always `Bool`.
|
||||
* On success, `value` is present. It may itself be `None`, a `List`, a `Map`,
|
||||
a `Table`, a `Memory` handle or any other value.
|
||||
* On failure, `code` and `message` are present. `code` is a lowercase
|
||||
dot-separated stable identifier; `message` is human-readable text.
|
||||
* `details` is optional and, when present, is a `Map` of additional structured
|
||||
facts. It must not contain host pointers, resource handles or secret bytes.
|
||||
* No other fields are required. Consumers must tolerate additive fields.
|
||||
|
||||
The constructors are ordinary `.mq.md` functions, for example:
|
||||
|
||||
```text
|
||||
## 成功
|
||||
`(值)`
|
||||
*{ok: true, value: 值}*
|
||||
|
||||
## 失败
|
||||
`(代码, 消息)`
|
||||
*{ok: false, code: 代码, message: 消息}*
|
||||
```
|
||||
|
||||
They are **not** builtins. A library may choose different local function names;
|
||||
the map shape is the contract.
|
||||
|
||||
### 2. What is recoverable, what stays fatal
|
||||
|
||||
Recoverable (`Result`):
|
||||
|
||||
* a user allocator has no fitting block (`alloc.no_space`);
|
||||
* a parser rejects malformed input or hits an unexpected end
|
||||
(`parse.invalid_syntax`, `parse.unexpected_eof`);
|
||||
* a codec rejects an invalid sequence (`codec.invalid_sequence`);
|
||||
* a lookup that uses an explicit result because `None` is a legal value;
|
||||
* later, a resource reports EOF/partial/pending as a **success payload** with
|
||||
its own byte counts, not as a fatal error (P3 contract).
|
||||
|
||||
Fatal `RuntimeError` (unchanged):
|
||||
|
||||
* violating a language invariant: wrong operand type, out-of-bounds managed
|
||||
collection access, unknown/stale `Memory` handle, memory budget exhaustion,
|
||||
division by zero, call arity, unknown symbol;
|
||||
* a capability denial (authorization is not a recoverable business result);
|
||||
* an internal invariant failure.
|
||||
|
||||
A library must not convert a fatal invariant into an `Err` by catching it;
|
||||
there is no catch mechanism and none is proposed here. It must validate inputs
|
||||
before calling an operation whose failure is fatal, or rely on its own
|
||||
pre-checks.
|
||||
|
||||
### 3. Codes and namespaces
|
||||
|
||||
`code` is stable within the library that defines it. Recommended prefixes:
|
||||
|
||||
| Prefix | Owner |
|
||||
|---|---|
|
||||
| `alloc.*` | allocator libraries |
|
||||
| `parse.*` | parsers |
|
||||
| `codec.*` | byte/text codecs |
|
||||
| `table.*` | table algorithms |
|
||||
| `io.*` | stream wrappers (P3, when the resource contract exists) |
|
||||
|
||||
Codes are compared as `Text`; a caller must not depend on the `message` text.
|
||||
Libraries may add codes; removing or renaming one is a breaking change for that
|
||||
library.
|
||||
|
||||
### 4. EOF and partial I/O (forward contract)
|
||||
|
||||
When the Stream/Resource API arrives (CM-A06), a successful read must
|
||||
distinguish:
|
||||
|
||||
```text
|
||||
{ok: true, value: {count: Int, eof: Bool, pending: Bool}}
|
||||
```
|
||||
|
||||
EOF is not an `Err`; a short read is not an `Err`; a connection reset or a
|
||||
permission failure is. The exact stream payload is frozen by CM-A06, not by
|
||||
this ADR. This ADR only reserves the principle.
|
||||
|
||||
### 5. Why not a new type or syntax
|
||||
|
||||
* A dedicated `Value::Result` would change the value model, wire format,
|
||||
display, equality and every exhaustive `Value` match for a convention that
|
||||
existing `Map` plus `ok` already expresses.
|
||||
* `try`/`catch` would add a control-flow construct that bypasses ordinary
|
||||
branches, contradicts the RFC's preference for `Result` values and needs new
|
||||
grammar, scope and error rules.
|
||||
* Ordinary `Map` results are visible to `type`, `length`, `map.get` and wire
|
||||
encoding today, and can be built by `.mq.md` libraries without a new ABI.
|
||||
|
||||
## Compatibility
|
||||
|
||||
* Additive: existing builtins and programs are unchanged.
|
||||
* `map.get` returning `None` for a missing key remains a lookup convention,
|
||||
not a `Result`; callers that need to distinguish a stored `None` use
|
||||
`map.has` (ADR-0023).
|
||||
* Runtime error categories and codes are unchanged.
|
||||
* No wire version bump: a `Result` is an ordinary `Map` already covered by the
|
||||
wire format.
|
||||
|
||||
## Non-goals
|
||||
|
||||
* No automatic `?`/propagation, exceptions, panic recovery or stack unwinding.
|
||||
* No `Result` type, generics, pattern matching or exhaustive `Ok`/`Err`
|
||||
typing.
|
||||
* No change to `memory.grow` or other core operations from fatal errors to
|
||||
`Result`; a library may pre-check and return `Err` instead.
|
||||
* No resource/stream implementation; CM-A06 remains open.
|
||||
* No promise that every existing library will adopt the convention.
|
||||
|
||||
## Consequences
|
||||
|
||||
* `.mq.md` algorithms can return and branch on failure without aborting the
|
||||
run. This is a prerequisite for a Marqdo-written allocator, parser and
|
||||
eventually `io.read_line`.
|
||||
* Because the convention is untyped, a caller can inspect the wrong shape;
|
||||
`类型(result)`, `map.has(result, "ok")` and the frozen field names are the
|
||||
checks. A future typed-value ADR may improve this without breaking the data
|
||||
shape.
|
||||
* The allocator fixture in `54`/`55` demonstrates success and failure maps and
|
||||
ordinary branching, with no Rust algorithm.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-result-and-free-list-p2.md` records a `.mq.md`
|
||||
first-fit/free-list allocator that returns `Result` maps, reuses freed blocks,
|
||||
splits free blocks, coalesces address-adjacent free blocks and returns
|
||||
`alloc.no_space` without aborting the run.
|
||||
@@ -0,0 +1,173 @@
|
||||
# ADR-0025: Minimal Resource/Stream ABI and byte transfer through `Memory` (RFC-CM CM-A06)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md`,
|
||||
`docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md` (CM-A06, CM-A09,
|
||||
CM-A11, P3), ADR-0022, ADR-0024
|
||||
Specs: `spec/evaluation.md`, `spec/security.md`, `spec/value-wire.md`
|
||||
Evidence: `docs/evidence/2026-11-20-resource-stream-p3.md`
|
||||
|
||||
## Context
|
||||
|
||||
P1/P2 give `.mq.md` managed values, addressable `Memory`, structure primitives,
|
||||
a recoverable `Result` convention and a substantial set of in-language
|
||||
algorithms. Nothing yet lets a standard-library function perform a real byte
|
||||
transfer to the outside world without a dedicated `HostOp` per high-level
|
||||
function.
|
||||
|
||||
CM-A06 asks for a stable, authorized, versioned resource boundary: obtain a
|
||||
granted resource, call the operations it promises, release it. CM-A09 says to
|
||||
extend the existing host/ABI rather than build a second plugin system. CM-A11
|
||||
says strict must stay fail-closed until a provider is actually verified.
|
||||
|
||||
This ADR freezes a **first, deliberately small slice**:
|
||||
|
||||
* one opaque host-issued `ResourceHandle` value;
|
||||
* `resource.open/read/write/close` as pure language builtins, not new `HostOp`
|
||||
variants;
|
||||
* byte transfer through `Memory`, never through host pointers;
|
||||
* stdio `stdout` write and project-file **read** only;
|
||||
* `Result` maps for EOF/closed/write counts, fatal errors for invalid handle
|
||||
or capability denial.
|
||||
|
||||
File write, directories, network, processes, TLS, asynchronous I/O and a
|
||||
generic provider registry remain open.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Opaque handle
|
||||
|
||||
`Value::ResourceHandle(ResourceHandle)` is a host-issued opaque identity with
|
||||
private `(run_id, resource_id, generation)` fields and a hidden constructor.
|
||||
It is not `Serialize`/`Deserialize`, is rejected by the wire, is never
|
||||
synthesized from a number or a JSON value, and renders as `Resource` in
|
||||
`display`/`repr` without exposing the id.
|
||||
|
||||
It is a **separate domain** from `Memory`: a handle is not indexable, not
|
||||
byte-addressable and not a capability token the user can manufacture.
|
||||
|
||||
### 2. First resource kinds
|
||||
|
||||
| Kind | Descriptor | Rights | Availability |
|
||||
|---|---|---|---|
|
||||
| `stdio` | `stdout` | `write` | `pure` and `project` when the console grant is present; `strict` rejects until verified |
|
||||
| `file` | project-relative path | `read` | `project` with an `fs_read` grant |
|
||||
|
||||
`file` streams are opened by reading a bounded regular file (existing
|
||||
`ProjectFs::MAX_READ_BYTES = 8 MiB` limit) into host-owned bytes; `read`
|
||||
returns successive chunks and `eof` when the position reaches the end. This is
|
||||
honest about being a first provider: it does not expose a host file descriptor,
|
||||
does not support seeking, writing, directories, network, processes, devices or
|
||||
paths outside the project root.
|
||||
|
||||
Any other kind/descriptor/right combination is an explicit
|
||||
`resource.unsupported`/`permission.denied` diagnostic, never a silent fallback.
|
||||
|
||||
### 3. Builtins
|
||||
|
||||
```text
|
||||
resource.open(kind: Text, descriptor: Text, rights: List) -> Resource
|
||||
resource.read(handle: Resource, memory: Memory, offset: Int, max_bytes: Int)
|
||||
-> {ok:true, value:{count:Int, eof:Bool}} | {ok:false, code, message}
|
||||
resource.write(handle: Resource, memory: Memory, offset: Int, length: Int)
|
||||
-> {ok:true, value:{count:Int}} | {ok:false, code, message}
|
||||
resource.close(handle: Resource)
|
||||
-> {ok:true, value:None} | {ok:false, code, message}
|
||||
```
|
||||
|
||||
* `resource.open` is effectful and capability-checked at execution time.
|
||||
* `resource.read` validates `offset` and the remaining buffer **before** asking
|
||||
the provider, so a failed range check cannot consume stream data. It caps the
|
||||
request to the remaining region, writes the returned bytes into `Memory` and
|
||||
returns the actual count and EOF flag. EOF is a **success** with `eof:true`.
|
||||
* `resource.write` copies bytes out of `Memory` only for the duration of the
|
||||
call. A short write is reported as a count; the caller loops. It never
|
||||
stores a host pointer into a region that could later grow.
|
||||
* `resource.close` closes the host slot. Reading/writing/closing an already
|
||||
closed handle returns `{ok:false, code:"resource.closed", ...}` (recoverable).
|
||||
A forged, cross-run or stale handle is `runtime.resource` (fatal).
|
||||
* All four are pure interpreter operations over host trait methods; they add no
|
||||
`HostOp` variant. Invalid capacity/bounds remains `runtime.index`; wrong
|
||||
operand types remain `runtime.type`.
|
||||
|
||||
### 4. Authorization and modes
|
||||
|
||||
* Authorization is the intersection of the requested operation, the effective
|
||||
`Capabilities` grant and the mode:
|
||||
* `stdio/stdout/write` requires `console` (the same grant `打印` uses);
|
||||
* `file/.../read` requires an `fs_read` pattern match and project-root path
|
||||
validation (`..`, absolute paths and symlink escapes rejected);
|
||||
* `pure`: stdio stdout is allowed because console is already a pure-mode
|
||||
console grant; **file resources are refused by the mode check**, even if a
|
||||
caller constructs a `Capabilities` set with a matching `fs_read` pattern.
|
||||
Network/process resources do not exist in this slice.
|
||||
* `project`: stdio stdout and explicitly granted project-file reads.
|
||||
* `strict`: **every** `resource.open` is refused with `permission.denied`.
|
||||
Strict L1 is unchanged and no new provider is silently enabled. A future ADR
|
||||
must add real OS-level evidence before strict accepts streams.
|
||||
* Manifest capability requests are not grants; only the operator-provided
|
||||
effective set is consulted.
|
||||
|
||||
### 5. Errors, budgets and tracing
|
||||
|
||||
New stable categories:
|
||||
|
||||
| Code | Meaning |
|
||||
|---|---|
|
||||
| `runtime.resource` | forged/cross-run/stale handle, invalid resource state |
|
||||
| `resource.closed` | read/write/close on an already closed handle (recoverable) |
|
||||
| `resource.unsupported` | kind/descriptor/rights not implemented in this slice |
|
||||
|
||||
`resource.write` to stdout charges the existing output budget and appends to
|
||||
the host output buffer exactly like `打印`; capability denial is
|
||||
`permission.denied`. Trace records `resource.open`, `resource.read`,
|
||||
`resource.write` and `resource.close` with kind/count/eof metadata, never raw
|
||||
file bytes or handles.
|
||||
|
||||
### 6. Wire and boundary
|
||||
|
||||
* `to_wire(Resource)` emits `{"type":"resource"}` only; decoding it is a hard
|
||||
error (same rule as `Memory`).
|
||||
* The generic `Host::invoke` boundary rejects arguments containing a resource
|
||||
handle, so plugins/foreign kernels/file adapters never receive one.
|
||||
* Plugin output cannot mint a handle: the constructor is host-only and the
|
||||
value is not decodable from plugin wire.
|
||||
|
||||
### 7. Non-goals
|
||||
|
||||
* No `print` migration in this ADR; that is CM-A07 and a later increment.
|
||||
* File **write** is added by [ADR-0026](0026-buffered-project-file-write-streams.md),
|
||||
not this decision; this ADR remains read-only for files.
|
||||
* No file write, directory listing, rename, metadata, network, DNS, TLS,
|
||||
process, clock, random or device operations.
|
||||
* No generic third-party provider registration; CM-A09 remains open.
|
||||
* No zero-copy, async, deadlines per stream, seek or shared memory.
|
||||
* No strict-mode resource support and no claim that host-in-process I/O is a
|
||||
sandbox.
|
||||
|
||||
## Compatibility
|
||||
|
||||
* Additive value variant; old programs never produce `Resource`.
|
||||
* New builtins only; existing `fs.read/write`, `打印`, plugin and foreign
|
||||
behavior is unchanged.
|
||||
* No wire version bump: the new tag is rejected, not accepted.
|
||||
* New error categories are additive; existing categories/codes are unchanged.
|
||||
|
||||
## Consequences
|
||||
|
||||
* `.mq.md` can implement `write_text`-style output by encoding text into
|
||||
`Memory` (fixture 59) and calling `resource.write`, and can implement
|
||||
chunked file readers with `resource.read`/`Result`/EOF.
|
||||
* The remaining high-level I/O surface can grow in the standard library without
|
||||
a new `HostOp` per function.
|
||||
* The resource table is host state per run; closed slots are not reused in this
|
||||
slice, so generation checks are straightforward.
|
||||
* Strict remains conservative; `.mq.md` resource code runs in `pure` (stdout)
|
||||
or `project` (file read) only.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-resource-stream-p3.md` records unit tests for grants,
|
||||
mode refusal, traversal rejection, forged/closed handles, output budget,
|
||||
file chunking/EOF, wire rejection and a `.mq.md` stdout fixture.
|
||||
@@ -0,0 +1,115 @@
|
||||
# ADR-0026: Buffered project-file write streams (CM-A06 extension)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A06, CM-A09, CM-A11), ADR-0024, ADR-0025
|
||||
Specs: `spec/evaluation.md`, `spec/security.md`
|
||||
Evidence: `docs/evidence/2026-11-20-resource-file-write-p3.md`
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0025 froze the first resource ABI slice with `stdio/stdout` write and
|
||||
project-file **read**. A standard library cannot implement file copy, file
|
||||
format writers or "write then verify" without a write-capable file resource.
|
||||
The RFC's P3 target is explicit `open/read/write/close` for project files in
|
||||
`project` mode; strict must remain fail-closed.
|
||||
|
||||
Direct per-write file I/O would expose partial writes and OS error timing to
|
||||
every caller and would make it easy to leave a half-written project file. This
|
||||
ADR adds the smallest honest write path: an in-memory buffer per file writer,
|
||||
flushed through the existing project path policy on `close`.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Provider shape
|
||||
|
||||
Add a third stream shape to the host resource table (ADR-0025):
|
||||
|
||||
```text
|
||||
resource.open("file", "<project-relative path>", ["write"]) -> Resource
|
||||
```
|
||||
|
||||
* `project` mode only, with an `fs_write` glob grant matching the normalised
|
||||
relative path.
|
||||
* `pure` still refuses file resources by mode check; `strict` still refuses
|
||||
every `resource.open` (CM-A11 fail-closed).
|
||||
* Absolute paths, `..`, symlink escapes and non-regular targets are rejected by
|
||||
`ProjectFs` exactly like the existing `fs.write`.
|
||||
* A writer owns an in-memory byte buffer. `resource.write` appends bytes and
|
||||
returns `{ok:true, value:{count}}`; it does **not** touch the filesystem.
|
||||
* `resource.close` flushes the buffer through `ProjectFs::write_bytes` and then
|
||||
marks the handle closed. If the flush fails, the handle stays open with its
|
||||
buffer intact and the error is reported; the caller may retry or close again.
|
||||
* If the program exits without closing, no file is written. This is a
|
||||
deliberate "no flush on drop" rule: a crash or a missing `close` must not
|
||||
silently publish a partial file.
|
||||
* `resource.write` is rejected on a file reader; `resource.read` is rejected on
|
||||
a file writer; stdout remains write-only. Wrong-direction use is
|
||||
`resource.unsupported`.
|
||||
* A writer buffer is capped at the existing 8 MiB project read/write ceiling
|
||||
(`MAX_READ_BYTES`). Exceeding it is `resource.exhausted` and the append is
|
||||
atomic: the buffer and file are unchanged.
|
||||
|
||||
### 2. Rights and modes
|
||||
|
||||
| Kind | Descriptor | Rights | Mode | Grant |
|
||||
|---|---|---|---|---|
|
||||
| `file` | project-relative path | `["read"]` | project | `fs_read` glob |
|
||||
| `file` | project-relative path | `["write"]` | project | `fs_write` glob |
|
||||
| `stdio` | `stdout` | `["write"]` | pure/project | `console` |
|
||||
|
||||
Mixed rights (`["read","write"]`) and other kinds/rights are
|
||||
`resource.unsupported` in this slice. There is no truncate/append/seek flag:
|
||||
every writer starts empty and replaces the file on flush.
|
||||
|
||||
### 3. Errors, budgets and tracing
|
||||
|
||||
* `resource.closed` for read/write/close on a closed handle (recoverable).
|
||||
* `runtime.resource` for forged/cross-run/stale handles (fatal).
|
||||
* `resource.exhausted` for a buffer over the byte ceiling.
|
||||
* `permission.denied` for a missing mode/grant or a path policy failure.
|
||||
* `resource.unsupported` for a direction/rights combination not implemented.
|
||||
* `resource.write` on a file writer does not charge the program output budget;
|
||||
the buffer is bounded by the file ceiling. `resource.close` performs one
|
||||
`ProjectFs::write_bytes` call and records `resource.close` with the byte
|
||||
count, never the contents.
|
||||
* No partial file is visible on a failed close because the write is a single
|
||||
`std::fs::write` to the validated project path (the same atomicity level as
|
||||
the existing `fs.write`; no temp-file rename is claimed).
|
||||
|
||||
### 4. Explicit non-goals
|
||||
|
||||
* No append/truncate/seek flags, no random access, no directory or metadata
|
||||
operations.
|
||||
* No file write in pure or strict mode.
|
||||
* No network/process/device/clock/random resources.
|
||||
* No generic provider registry (CM-A09), no async/nonblocking I/O.
|
||||
* No claim that `ProjectFs::write_bytes` is crash-atomic; it is a single
|
||||
validated write, not a transactional replace.
|
||||
* No CM-A07 `print` migration in this ADR.
|
||||
|
||||
## Compatibility
|
||||
|
||||
* Additive to ADR-0025: existing `file` read and `stdio` write behavior is
|
||||
unchanged; `file` write was previously `resource.unsupported`.
|
||||
* No new `Value` variant, `HostOp`, Markdown node or wire shape.
|
||||
* Error categories/codes are unchanged; the new `resource.exhausted` diagnostic
|
||||
reuses the existing stable `resource.exhausted` code family.
|
||||
|
||||
## Consequences
|
||||
|
||||
* `.mq.md` can now implement file copy and bounded format writers by opening a
|
||||
read stream and a write stream, transferring bytes through `Memory`, and
|
||||
closing both. This is C09's core shape without a `copy_file` Rust builtin.
|
||||
* Buffered close gives deterministic "all or nothing within one write call"
|
||||
behavior and keeps OS write errors at a point where the caller still has a
|
||||
Result path.
|
||||
* Large files require chunked writes; the 8 MiB ceiling is the first-slice
|
||||
limit, not a permanent design.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-resource-file-write-p3.md` records host unit tests
|
||||
(grants, pure/strict refusal, traversal, buffer ceiling, retry-after-failure),
|
||||
CLI tests for write-then-read and missing grants, and the full workspace gate.
|
||||
@@ -0,0 +1,103 @@
|
||||
# ADR-0027: CM-A07 print-migration staging and the Compat-A `.mq.md` stdlib prototype
|
||||
|
||||
Status: **Accepted (Compat-A only; Compat-B/C pending)**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A07, CM-A10, P3), ADR-0023, ADR-0024, ADR-0025, ADR-0026
|
||||
Specs: `spec/evaluation.md`
|
||||
Evidence: `docs/evidence/2026-11-20-stdlib-print-compat-a.md`
|
||||
|
||||
## Context
|
||||
|
||||
RFC-CM-0002 §5.4 and CM-A07 require `打印` to stop being a hidden Rust
|
||||
`Value::display()` path. The migration has three compatibility stages:
|
||||
|
||||
* **Compat-A**: a real `.mq.md` print implementation exists and is callable
|
||||
through explicit imports; the legacy builtin remains the default.
|
||||
* **Compat-B**: the `print/打印` name routes to the stdlib implementation
|
||||
(or an alias table maps it), while old behavior is comparison-tested.
|
||||
* **Compat-C**: the legacy `HostOp::Print` formatting path is removed from the
|
||||
public builtin implementation.
|
||||
|
||||
This ADR freezes the staging policy and implements **Compat-A only**. It does
|
||||
not change `打印` resolution, does not remove `HostOp::Print`, and does not
|
||||
claim CM-A07 complete.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Stage policy
|
||||
|
||||
* A stage is only complete when its comparison evidence exists:
|
||||
* A: explicit stdlib functions exist and their output is compared with the
|
||||
builtin for the covered value set;
|
||||
* B: name resolution tests prove a single implementation is selected and old
|
||||
callers still behave;
|
||||
* C: all old goldens pass with no `HostOp::Print` formatting path.
|
||||
* Stages are additive; no stage may silently make a second implementation
|
||||
reachable for the same name.
|
||||
* `strict` behavior is unchanged by stdlib-only code; a stdlib print still
|
||||
needs the console/resource grant.
|
||||
|
||||
### 2. Compat-A implementation
|
||||
|
||||
Add a normal `.mq.md` module package under `examples/stdlib/`:
|
||||
|
||||
```text
|
||||
examples/stdlib/format.mq.md # 显示 / 列表显示 / 映射显示 / 表格显示 / 转义文本
|
||||
examples/stdlib/text.mq.md # utf8编码 (Memory bytes)
|
||||
examples/stdlib/io.mq.md # 写文本 / 写值 (resource.open stdout + write/close)
|
||||
examples/stdlib/demo.mq.md # side-by-side comparison with 打印
|
||||
```
|
||||
|
||||
`io.写值(值)` = `format.显示(值) + "\n"` → `text.utf8编码` → `Memory` →
|
||||
`resource.open("stdio","stdout",["write"])` → `resource.write` → `close`.
|
||||
This uses only the public core/resource contracts; it is ordinary imported
|
||||
`.mq.md` code, not a new builtin.
|
||||
|
||||
Covered value types: `Unit`, `None`, `Bool`, `Int`, `Float`, `Text`, `List`,
|
||||
`Map`, `Table`. Formatting matches the builtin `Value::display()` for the
|
||||
checked corpus, including Text quoting/escaping inside collections, canonical
|
||||
map key order and table layout.
|
||||
|
||||
**Not covered yet**: `Formula`, `Code`, `FunctionRef`, `PluginHandle`. The
|
||||
language currently has no public observation for the module/name/handle
|
||||
identity needed to reproduce `Value::display()`/`repr()` for those types.
|
||||
Adding such observation would be a CM-A04 extension and is out of scope.
|
||||
|
||||
### 3. Compatibility rules
|
||||
|
||||
* `打印` remains the default builtin in this slice. `io.写值` is only reachable
|
||||
through an explicit import alias; there is no implicit module, bundled stdlib
|
||||
resolution or global name shadowing.
|
||||
* No `HostOp` is added or removed. Both paths write through the console/output
|
||||
budget; the stdlib path additionally goes through the resource ABI.
|
||||
* Stdlib modules are ordinary Markdown documents: no special parser, no
|
||||
embedded Rust formatter, no direct formatting syscall.
|
||||
* A future Compat-B ADR must define name precedence, bundled module lookup,
|
||||
stdlib version/digest (CM-A10) and failure behavior; this ADR does not.
|
||||
|
||||
### 4. Non-goals
|
||||
|
||||
* No `print` alias routing, no removal of `HostOp::Print`.
|
||||
* No bundled stdlib install/version/digest or offline distribution.
|
||||
* No formatting of Formula/Code/FunctionRef/PluginHandle.
|
||||
* No locale-aware, width-aware or float-format changes.
|
||||
* No change to Markdown rendering, `strict`, or the wire/ABI.
|
||||
* No claim that the examples directory is the shipped standard library.
|
||||
|
||||
## Consequences
|
||||
|
||||
* The target call chain (`format.display` → `utf8_encode` → `write_all` →
|
||||
`resource.write`) is exercised end to end in `.mq.md` and can be compared to
|
||||
the builtin.
|
||||
* The missing observation for `Formula/Code/FunctionRef/PluginHandle` is now an
|
||||
explicit, documented blocker for Compat-B rather than an implicit assumption.
|
||||
* Because the stdlib is not bundled, ordinary users still get the builtin; the
|
||||
prototype does not yet reduce the runtime's formatting surface.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-stdlib-print-compat-a.md` records the side-by-side
|
||||
demo (`io.写值` vs `打印` for a nested list and a table), the CLI test asserting
|
||||
byte-identical lines, and the full workspace gate. It also records the covered
|
||||
and uncovered value types.
|
||||
@@ -0,0 +1,38 @@
|
||||
# Architecture RFCs: compute machine and stdlib migration
|
||||
|
||||
Status: **Draft RFCs, partially implemented**. This page is an index; the
|
||||
normative RFC text stays at its reviewable path under `docs/`.
|
||||
|
||||
| Document | Status | Implemented part |
|
||||
|---|---|---|
|
||||
| [RFC-CM-0001: compute machine, addressable memory and system boundary](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) | Draft / design input | R1 (P1): `Memory` value, `memory.new/size/grow`, byte read/write, Memory-index assignment, budgets, wire/host rejection. Implemented per [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md); see [STATUS §5](../STATUS.md). |
|
||||
| [RFC-CM-0002: minimal core, stdlib self-hosting and system capability migration](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | Draft / design input | P0/P1, a **partial P2**, and a **partial P3** first slice (ADR-0025/0026: stdout write, granted project-file read and buffered project-file write via `resource.*`; strict refuses all opens; ADR-0027 Compat-A `.mq.md` print prototype, builtin still default). P2: fixtures 48–50 (`.mq.md` bump allocator, u32 codec, linked list/traversal, failed-allocation atomicity), CM-A04 structure primitives for List/Map/Text/Table (ADR-0023, fixtures 51–54), and a `.mq.md` first-fit free-list allocator with block splitting, address-adjacent coalescing and `Result` maps (ADR-0024, fixture 55), plus a `Memory`-backed binary tree with recursive insert/in-order traversal (fixture 56), a `.mq.md` CSV row splitter (fixture 57), an open-addressing hash table with linear probing (fixture 58), a `.mq.md` UTF-8 encoder and validating decoder (`utf8.truncated`/`invalid_continuation`/`invalid_scalar`) (fixtures 59–60), a `Memory`-backed graph adjacency list (fixture 61), a recursive JSON parser for arrays, objects, strings with escapes, integers/booleans/null and error Results (fixture 62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (fixture 63), big-endian 16/32/64-bit plus LEB128 integer codecs (fixture 64), a `.mq.md` table sort rebuilding the table without mutating the input (fixture 65), a table join built from `table.row/columns/cell` (fixture 67), and a quoted/escaped CSV row parser (fixture 68). No separate stdlib repo, JSON floats, network/generic providers, P4 Table migration, P5 providers or P6 performance work. |
|
||||
|
||||
The RFCs are architecture constraints, not evidence that a feature exists. The
|
||||
implementation status lives in `docs/STATUS.md`; frozen behavior is in
|
||||
`spec/`; decisions are in `docs/adr/`.
|
||||
|
||||
## Decisions already frozen
|
||||
|
||||
| RFC item | ADR | State |
|
||||
|---|---|---|
|
||||
| CM-A01 Memory identity/lifetime | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
|
||||
| CM-A02 page size, growth, budgets | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
|
||||
| CM-A03 place assignment and evaluation order | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
|
||||
| CM-A04 minimal structure observation/construction | [0023](../adr/0023-minimal-structure-observation-and-construction.md) | Accepted |
|
||||
| CM-A05 recoverable `Result` convention | [0024](../adr/0024-recoverable-result-convention.md) | Accepted |
|
||||
| CM-A06 minimal Resource/Stream ABI | [0025](../adr/0025-minimal-resource-stream-abi.md) | Accepted (first slice) |
|
||||
| CM-A06 buffered project-file write | [0026](../adr/0026-buffered-project-file-write-streams.md) | Accepted |
|
||||
| CM-A07 print migration staging / Compat-A | [0027](../adr/0027-print-migration-compat-a.md) | Accepted (Compat-A only) |
|
||||
| CM-A07 Compat-B/C, stdlib packaging, Table storage, generic providers, strict resources | — | Not decided; do not implement by implication |
|
||||
|
||||
## What a future phase must not do
|
||||
|
||||
* add a top-level CLI command or Markdown keyword for memory, allocation or I/O;
|
||||
* add a `HostOp` for every new stdlib function;
|
||||
* turn managed `List`/`Map`/`Table` into aliased mutable storage;
|
||||
* expose a host pointer, let a handle cross the wire, or claim block-level
|
||||
safety for user allocators;
|
||||
* make `strict` accept an unverified provider by default;
|
||||
* claim an RFC phase is complete without the closure and negative evidence its
|
||||
phase gate requires.
|
||||
@@ -0,0 +1,130 @@
|
||||
# RFC-CM P1 evidence: addressable Memory and place assignment
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline commit: `a113d894962d8c00eb02efb42e0e73f8b084a1fa` (`docs: bind
|
||||
README/COMPATIBILITY/CHANGELOG to the convergence SHA`)
|
||||
Working tree: RFC-CM P1 changes, **not committed** at the time of this report.
|
||||
Decision record: [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md)
|
||||
Scope: RFC-CM-0001 R1 / RFC-CM-0002 P1 only. P2–P6 are **not** implemented.
|
||||
|
||||
## Environment
|
||||
|
||||
* Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
|
||||
* `PATH=/root/.cargo/bin:$PATH`
|
||||
* `CARGO_HOME=/root/work/marqdo-cli/.cache/cargo`
|
||||
* `CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target`
|
||||
* No GitHub credentials in this checkout; this is **local** evidence, not a
|
||||
GitHub Actions result. Windows/macOS lanes were not run.
|
||||
|
||||
## Static and workspace gate
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
47 test binaries, 389 passed, 0 failed, 2 ignored (doc-tests)
|
||||
```
|
||||
|
||||
The 2 ignored tests are the existing `marqdo-plugin-sdk` doc-tests, not new
|
||||
memory tests.
|
||||
|
||||
## Focused evidence
|
||||
|
||||
```text
|
||||
cargo test -p marqdo-runtime --lib memory
|
||||
8 passed; 0 failed
|
||||
regions_are_isolated_and_handles_share_identity
|
||||
create_is_zero_filled_and_reports_pages
|
||||
grow_keeps_old_bytes_and_zero_fills_new_pages
|
||||
failed_grow_does_not_change_region_or_budget
|
||||
invalid_offsets_are_structured_errors
|
||||
create_validates_page_counts_and_budget
|
||||
forged_and_stale_handles_are_rejected
|
||||
page_arithmetic_never_overflows
|
||||
|
||||
cargo test -p marqdo-runtime --test interpreter memory
|
||||
3 passed; 0 failed
|
||||
memory_region_is_shared_by_handle_identity
|
||||
memory_handle_cannot_reach_the_host_boundary
|
||||
memory_budget_is_enforced_without_changing_the_region
|
||||
|
||||
cargo test -p marqdo-value --lib memory
|
||||
2 passed; 0 failed
|
||||
memory_display_never_leaks_the_runtime_handle
|
||||
memory_handles_are_not_transferable
|
||||
|
||||
cargo test -p marqdo-expression --lib memory_index
|
||||
1 passed; 0 failed
|
||||
memory_index_assignment_parses_as_a_place
|
||||
|
||||
cargo test -p marqdo-cli \
|
||||
--test conformance_semantics --test negative \
|
||||
--test analysis_trace --test examples_r6
|
||||
semantic_goldens_match_end_to_end_execution ok
|
||||
runtime_negative_goldens ok
|
||||
analysis_v2_projects_memory_place_targets_additively ok
|
||||
addressable_memory_example_runs_through_the_cli ok
|
||||
cli_memory_budget_override_fails_closed ok
|
||||
(20 tests total in these four binaries, 0 failed)
|
||||
```
|
||||
|
||||
## End-to-end CLI
|
||||
|
||||
```text
|
||||
$ marqdo run examples/memory-buffer.mq.md --json
|
||||
{"success": true, "value": {"type": "int", "decimal": "141"}, ...}
|
||||
|
||||
$ marqdo run examples/memory-buffer.mq.md --max-memory 0 --json
|
||||
{"success": false, "error": {"code": "resource.exhausted", ...}, ...}
|
||||
```
|
||||
|
||||
The example creates one 64 KiB page with a declared maximum of four pages,
|
||||
writes two bytes through two names that share one region, and returns
|
||||
`42 + 99 = 141`.
|
||||
|
||||
## Fixtures
|
||||
|
||||
Semantic goldens:
|
||||
|
||||
* `tests/conformance/semantics/45-memory-buffer.mq.md` — shared identity
|
||||
(`另一个名字 = 区域`) and two byte writes.
|
||||
* `tests/conformance/semantics/46-memory-grow-zero-fill.mq.md` — zero-page
|
||||
create, `grow` returning the old count, old bytes preserved, new byte zero,
|
||||
`memory.size`.
|
||||
* `tests/conformance/semantics/47-memory-isolation-and-alias.mq.md` — two
|
||||
regions do not alias at the same offset; handle equality is identity.
|
||||
|
||||
Runtime negatives:
|
||||
|
||||
| Fixture | Expected code |
|
||||
|---|---|
|
||||
| `13-memory-out-of-bounds` | `runtime.index` |
|
||||
| `14-memory-byte-range` | `runtime.type` |
|
||||
| `15-index-assign-non-memory` | `runtime.type` |
|
||||
| `16-memory-huge-offset` | `runtime.index` |
|
||||
| `17-memory-grow-beyond-max` | `resource.exhausted` |
|
||||
| `18-memory-negative-pages` | `runtime.memory` |
|
||||
|
||||
## What this does not prove
|
||||
|
||||
* No `.mq.md` stdlib repository or allocator/closure fixture yet (RFC-CM P2).
|
||||
* No `Resource`, `Stream`, byte-transfer ABI, stdlib `print` migration or
|
||||
`Table` storage abstraction (P3–P5).
|
||||
* No Windows/macOS evidence and no remote CI URL.
|
||||
* No claim of block-level memory safety for a user-written allocator; the
|
||||
runtime only bounds each access to its region.
|
||||
* No claim that `strict` accepts new I/O: memory is pure computation and no
|
||||
`HostOp` or provider was added.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
cargo fmt --all -- --check
|
||||
cargo clippy --workspace --all-targets -- -D warnings
|
||||
cargo test --workspace --locked
|
||||
cargo run -p marqdo-cli -- run examples/memory-buffer.mq.md --json
|
||||
```
|
||||
@@ -0,0 +1,74 @@
|
||||
# RFC-CM P2 partial evidence: `.mq.md`-written codec, allocator and list
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `a113d89` plus the uncommitted RFC-CM P0/P1/P2 changes.
|
||||
Related: [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md),
|
||||
[P1 evidence](2026-11-20-memory-p1.md),
|
||||
[RFC-CM-0002 §7.2](../../docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md).
|
||||
Scope: **partial P2 only**. The RFC-CM-0002 P2 gate is **not** passed.
|
||||
|
||||
> Historical slice: this report predates ADR-0023 (CM-A04 structure API),
|
||||
> ADR-0024 (CM-A05 Result), the coalescing allocator (fixture 55) and the
|
||||
> binary tree/CSV fixtures (56–57). Its "what remains" list is superseded by
|
||||
> [`2026-11-20-structure-api-p2.md`](2026-11-20-structure-api-p2.md) and
|
||||
> [`2026-11-20-result-and-free-list-p2.md`](2026-11-20-result-and-free-list-p2.md);
|
||||
> the P2 gate itself is still not passed.
|
||||
|
||||
## What was added without Rust algorithm code
|
||||
|
||||
Three semantic fixtures are ordinary `.mq.md` documents. They call only
|
||||
language builtins (`memory.new/size/grow`), indexing, arithmetic, functions,
|
||||
branches and loops. No Rust crate was changed for the algorithms themselves.
|
||||
|
||||
| Fixture | `.mq.md` subject | Result |
|
||||
|---|---|---|
|
||||
| `tests/conformance/semantics/48-memory-bump-allocator.mq.md` | big-endian u32 encode/decode and a 4-byte-header bump allocator; allocates 8, 16, 8 bytes and writes a byte | `[4, 12, 28, 36, 255]` |
|
||||
| `tests/conformance/semantics/49-memory-linked-list.mq.md` | allocates three 8-byte nodes (`value:u8`, `next:u32`), links them and traverses the list | `[4, 12, 20, 60]` |
|
||||
| `tests/conformance/semantics/50-memory-allocation-failure-atomic.mq.md` | a too-large allocation returns `None` without moving the header; the next allocation still starts at 4 | `[None, 4, 4, 65536]` |
|
||||
|
||||
Observed CLI results (all `success: true`):
|
||||
|
||||
```text
|
||||
48: [4, 12, 28, 36, 255]
|
||||
49: [4, 12, 20, 60]
|
||||
50: [None, 4, 4, 65536]
|
||||
```
|
||||
|
||||
`cargo test -p marqdo-cli --test conformance_semantics` passes with these three
|
||||
fixtures included.
|
||||
|
||||
## Why this is not the P2 gate
|
||||
|
||||
RFC-CM-0002 §9/P2 requires C01–C06 and C11 to pass in an environment with
|
||||
Foreign/native plugins disabled, and requires a minimal API for dynamic
|
||||
List/Map/Table construction and observation plus recoverable errors. This
|
||||
slice does **not** provide:
|
||||
|
||||
* duplicate-free/allocator adversarial tests (C01 now has a first-fit
|
||||
free-list allocator with reuse, splitting and coalescing, fixture 55);
|
||||
* graph/hash-table structures (C02 now has a singly linked list and a binary
|
||||
search tree, fixtures 49 and 56, but no graph/hash table);
|
||||
* general List/Map/Table construction, mutation-free rebuilding or key
|
||||
enumeration (CM-A04 is undecided);
|
||||
* a `Result`/recoverable-error contract for parser/IO/EOF/partial conditions
|
||||
(CM-A05 is undecided);
|
||||
* an independent `marqdo-stdlib` repository, version/digest contract or
|
||||
frozen-CLI-binary closure lane (the RFC §10.5 experiment).
|
||||
|
||||
It also does not add a stdlib API to the CLI; the fixtures are conformance
|
||||
documents, not a shipped standard library.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
for f in 48-memory-bump-allocator 49-memory-linked-list \
|
||||
50-memory-allocation-failure-atomic; do
|
||||
cargo run -q -p marqdo-cli -- run \
|
||||
tests/conformance/semantics/$f.mq.md --json
|
||||
done
|
||||
cargo test -p marqdo-cli --test conformance_semantics
|
||||
```
|
||||
@@ -0,0 +1,104 @@
|
||||
# RFC-CM CM-A06 extension evidence: buffered project-file writes
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `a113d89` plus the uncommitted RFC-CM P0–P3 changes.
|
||||
Decision record: [ADR-0026](../adr/0026-buffered-project-file-write-streams.md)
|
||||
Related: [ADR-0025](2026-11-20-resource-stream-p3.md),
|
||||
[STATUS §5](../STATUS.md).
|
||||
Scope: project-file **write** extension only; no network/process/directory.
|
||||
|
||||
## Environment and gate
|
||||
|
||||
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
406 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
|
||||
```
|
||||
|
||||
No GitHub CI is available from this checkout; this is local evidence.
|
||||
|
||||
## Implemented behavior
|
||||
|
||||
`resource.open("file", "<project-relative path>", ["write"])`:
|
||||
|
||||
* project mode only; requires an `fs_write` glob grant and the same
|
||||
`ProjectFs` path policy (reject absolute, `..`, symlink escapes);
|
||||
* pure mode refuses file resources by mode; strict refuses every resource open;
|
||||
* the writer owns an in-memory buffer; `resource.write` appends and returns
|
||||
`{ok:true, value:{count}}` without touching the filesystem;
|
||||
* `resource.close` flushes through `ProjectFs::write_bytes` and then marks the
|
||||
handle closed; a failed flush leaves the handle open with its buffer;
|
||||
* an unclosed writer never creates the file (no flush on drop);
|
||||
* buffer ceiling is the existing 8 MiB file limit; an oversized append is
|
||||
`resource.exhausted` and leaves both buffer and file unchanged;
|
||||
* `resource.read` on a writer, and `resource.write` on a reader, are
|
||||
`resource.unsupported`;
|
||||
* no new `Value`, `HostOp`, Markdown node or wire shape.
|
||||
|
||||
## Tests
|
||||
|
||||
### Host integration (`crates/marqdo-host/tests/resource_streams.rs`, 9)
|
||||
|
||||
```text
|
||||
pure_mode_writes_stdout_through_a_resource
|
||||
stdout_write_obeys_the_output_budget_atomically
|
||||
strict_mode_refuses_every_resource_open
|
||||
resource_handles_cannot_cross_the_generic_host_boundary
|
||||
pure_mode_refuses_file_resources_even_with_a_grant
|
||||
project_file_writes_buffer_until_close_and_require_a_grant
|
||||
file_write_buffer_has_a_hard_ceiling
|
||||
project_file_reads_are_chunked_and_require_a_grant
|
||||
traversal_and_unsupported_rights_are_refused
|
||||
```
|
||||
|
||||
The write test asserts the file does not exist before `close`, appears with
|
||||
`abcdef` after `close`, closed read/close return `resource.closed`, and the
|
||||
same open without `fs_write` is `permission.denied`.
|
||||
|
||||
### CLI integration (`crates/cli/tests/resource_streams.rs`, 4)
|
||||
|
||||
```text
|
||||
project_file_stream_writes_on_close_then_reads_back
|
||||
run index.mq.md --allow-write out.bin --allow-read out.bin --json
|
||||
write count 3, close ok, read count 3 eof true, bytes 65/66/67
|
||||
out.bin on disk = "ABC"
|
||||
|
||||
file_copy_is_written_in_mqmd_through_chunked_streams
|
||||
.mq.md opens data.txt for read and out.bin for write,
|
||||
loops resource.read(16) / resource.write(count) until eof,
|
||||
closes both; total 6; out.bin on disk = "abcdef"
|
||||
(C09 file copy without a Rust copy_file builtin)
|
||||
|
||||
project_file_stream_reads_chunks_into_memory
|
||||
file_stream_requires_a_read_grant
|
||||
```
|
||||
|
||||
### Host table unit tests (`crates/marqdo-host/src/resource.rs`, 3)
|
||||
|
||||
Chunked EOF, forged/closed handles, stdout-not-readable and file-not-writable
|
||||
(read-only direction) remain covered; the writer uses the same handle checks.
|
||||
|
||||
## What this does not prove
|
||||
|
||||
* No append/truncate/seek, directories, metadata, network, process, clock,
|
||||
random, device or generic provider registry.
|
||||
* No crash-atomic replace: `ProjectFs::write_bytes` is one validated write, not
|
||||
a temp-file rename.
|
||||
* No file write in `pure` or `strict`.
|
||||
* No CM-A07 stdlib `print` migration.
|
||||
* No Windows/macOS evidence and no remote CI URL.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
cargo test -p marqdo-host --test resource_streams
|
||||
cargo test -p marqdo-cli --test resource_streams
|
||||
cargo test --workspace --locked
|
||||
```
|
||||
@@ -0,0 +1,144 @@
|
||||
# RFC-CM CM-A06 evidence: minimal Resource/Stream ABI
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `a113d89` plus the uncommitted RFC-CM P0–P3 changes.
|
||||
Decision record: [ADR-0025](../adr/0025-minimal-resource-stream-abi.md)
|
||||
Related: [P1](2026-11-20-memory-p1.md),
|
||||
[P2 result/allocator](2026-11-20-result-and-free-list-p2.md),
|
||||
[STATUS §5](../STATUS.md).
|
||||
Scope: **first CM-A06 slice only**. Not the full P3 migration.
|
||||
|
||||
## Environment and gate
|
||||
|
||||
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
402 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
|
||||
```
|
||||
|
||||
No GitHub CI is available from this checkout; this is local evidence.
|
||||
|
||||
## Implemented slice
|
||||
|
||||
### Value and wire
|
||||
|
||||
* `Value::Resource(ResourceHandle)` is an opaque host-issued handle; fields are
|
||||
private, the constructor is host-only, and `display`/`repr` render only
|
||||
`Resource`.
|
||||
* `to_wire(Resource)` emits only `{"type":"resource"}`; `from_wire` rejects the
|
||||
tag. A `Resource` inside a list/map/table is equally rejected on decode.
|
||||
* The generic `Host::invoke` boundary rejects resources before dispatch, like
|
||||
`Memory`. Plugins/foreign kernels never receive either handle kind.
|
||||
|
||||
### Builtins
|
||||
|
||||
```text
|
||||
resource.open(kind, descriptor, rights) -> Resource
|
||||
resource.read(handle, memory, offset, max_bytes) -> Result {count, eof}
|
||||
resource.write(handle, memory, offset, length) -> Result {count}
|
||||
resource.close(handle) -> Result
|
||||
```
|
||||
|
||||
* `read` validates the destination region **before** reading from the provider
|
||||
and caps the request to the remaining bytes.
|
||||
* EOF is `{ok:true, value:{count:0, eof:true}}`, not an error.
|
||||
* Closed handles return `{ok:false, code:"resource.closed"}`; forged/cross-run/
|
||||
stale handles are fatal `runtime.resource`; unsupported kind/rights are
|
||||
`resource.unsupported`; missing grant is `permission.denied`.
|
||||
* Bytes move only as copied `Memory` ranges for the duration of the call. No
|
||||
host pointer is stored in a region.
|
||||
|
||||
### Providers in this slice
|
||||
|
||||
| Kind | Descriptor | Rights | Mode | Behavior |
|
||||
|---|---|---|---|---|
|
||||
| `stdio` | `stdout` | `write` | pure/project | appends to the program output buffer after the output-budget check |
|
||||
| `file` | project-relative path | `read` | project | bounded regular file loaded at open; chunked reads with EOF |
|
||||
| anything else | — | — | — | explicit `resource.unsupported` |
|
||||
|
||||
* `strict` refuses every `resource.open` with `permission.denied` (CM-A11
|
||||
fail-closed); no provider is silently enabled.
|
||||
* `pure` allows only stdio stdout (console grant), exactly the existing
|
||||
console exception; file resources are refused by an explicit mode check,
|
||||
even if a caller hand-constructs a matching `fs_read` grant.
|
||||
* File paths still go through `ProjectFs::resolve` (absolute paths, `..`,
|
||||
symlink escapes rejected) and require an `fs_read` glob grant.
|
||||
* File size is bounded by the existing 8 MiB read limit.
|
||||
* `resource.write` to stdout charges the existing output budget; a budget
|
||||
failure produces no output. There is no separate I/O-byte budget yet.
|
||||
|
||||
## Tests
|
||||
|
||||
### Host integration (`crates/marqdo-host/tests/resource_streams.rs`, 7)
|
||||
|
||||
```text
|
||||
pure_mode_writes_stdout_through_a_resource
|
||||
stdout_write_obeys_the_output_budget_atomically
|
||||
strict_mode_refuses_every_resource_open
|
||||
resource_handles_cannot_cross_the_generic_host_boundary
|
||||
pure_mode_refuses_file_resources_even_with_a_grant
|
||||
project_file_reads_are_chunked_and_require_a_grant
|
||||
traversal_and_unsupported_rights_are_refused
|
||||
```
|
||||
|
||||
### Host table unit tests (`crates/marqdo-host/src/resource.rs`, 3)
|
||||
|
||||
```text
|
||||
file_reads_are_chunked_with_eof
|
||||
forged_and_closed_handles_are_rejected
|
||||
stdout_is_not_readable_and_file_is_not_writable
|
||||
```
|
||||
|
||||
### CLI integration (`crates/cli/tests/resource_streams.rs`, 2)
|
||||
|
||||
```text
|
||||
project_file_stream_reads_chunks_into_memory
|
||||
data.txt = "abcdef"
|
||||
run index.mq.md --allow-read data.txt --json
|
||||
results: counts 2/4/0, eof false/true/true, bytes 97..102
|
||||
resource.close => ok:true
|
||||
|
||||
file_stream_requires_a_read_grant
|
||||
same document without --allow-read
|
||||
exit 1, error.code = permission.denied, no bytes read
|
||||
```
|
||||
|
||||
### Semantic fixture 66
|
||||
|
||||
`tests/conformance/semantics/66-resource-stdout-write.mq.md`:
|
||||
|
||||
* `.mq.md` UTF-8 encoder writes `"你好,资源!\n"` into `Memory`;
|
||||
* `resource.open("stdio","stdout",["write"])`;
|
||||
* `resource.write` reports count 19;
|
||||
* `resource.close` succeeds;
|
||||
* a second `resource.read` on the closed handle returns
|
||||
`{ok:false,code:"resource.closed"}`;
|
||||
* captured program stdout is exactly `你好,资源!\n`.
|
||||
|
||||
## What this does not prove
|
||||
|
||||
* No file write, directory listing, rename, metadata, network, DNS, TLS,
|
||||
process, clock, random or device resources.
|
||||
* No generic third-party provider registry (CM-A09) and no Worker/ABI byte
|
||||
transfer.
|
||||
* No stdlib `print` migration (CM-A07); `打印` still uses `HostOp::Print`.
|
||||
* No strict-mode stream support. Strict L1 is unchanged and refuses opens.
|
||||
* No async/nonblocking/seek/zero-copy and no per-stream deadline.
|
||||
* No Windows/macOS native lane and no remote CI URL.
|
||||
* Host-in-process I/O is not an OS sandbox; project mode remains cooperative.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
cargo test -p marqdo-host
|
||||
cargo test -p marqdo-cli --test resource_streams
|
||||
cargo test -p marqdo-cli --test conformance_semantics
|
||||
cargo test --workspace --locked
|
||||
```
|
||||
@@ -0,0 +1,361 @@
|
||||
# RFC-CM CM-A05 evidence: `Result` convention and first-fit free-list allocator
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `a113d89` plus the uncommitted RFC-CM P0/P1/P2 changes.
|
||||
Decision records: [ADR-0024](../adr/0024-recoverable-result-convention.md),
|
||||
[ADR-0023](../adr/0023-minimal-structure-observation-and-construction.md)
|
||||
Related: [P1](2026-11-20-memory-p1.md),
|
||||
[P2 closure](2026-11-20-memory-p2-closure.md),
|
||||
[structure API](2026-11-20-structure-api-p2.md).
|
||||
Scope: CM-A05 convention, a `.mq.md` first-fit allocator, and the later P2
|
||||
fixtures 56–59 (binary tree, CSV splitter, hash table, UTF-8 encoder). Full P2
|
||||
gate still **not** passed.
|
||||
|
||||
## Environment and gate
|
||||
|
||||
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
47 test binaries, 389 passed, 0 failed, 2 ignored (doc-tests)
|
||||
```
|
||||
|
||||
No GitHub CI is available from this checkout; this is local evidence.
|
||||
|
||||
## `Result` convention (ADR-0024)
|
||||
|
||||
Success and failure are ordinary `Map` values:
|
||||
|
||||
```text
|
||||
success: {ok: true, value: <Value>}
|
||||
failure: {ok: false, code: <Text>, message: <Text>[, details: <Map>]}
|
||||
```
|
||||
|
||||
No `Result` value type, exception, `try`/`catch` or automatic propagation was
|
||||
added. Expected conditions return this shape; wrong types, out-of-bounds,
|
||||
invalid/stale handles, budget exhaustion and capability denial remain fatal
|
||||
`RuntimeError`s.
|
||||
|
||||
## Allocator fixture 55
|
||||
|
||||
`tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md` is a
|
||||
pure `.mq.md` first-fit allocator over `Memory`:
|
||||
|
||||
* 8-byte block header: total block size (`u32`) and free-list next (`u32`);
|
||||
* region header: free-list head at offset 0, bump pointer at offset 4;
|
||||
* allocation rounds the request up to an 8-byte multiple;
|
||||
* first fit walks the free list, splits a fitting block when the remainder is
|
||||
at least 16 bytes, otherwise unlinks it;
|
||||
* no suitable free block falls back to the bump pointer;
|
||||
* no space returns `{ok:false, code:"alloc.no_space", message:...}`;
|
||||
* the free list is address-sorted; `释放` inserts a block and coalesces it
|
||||
with an address-adjacent successor and predecessor without erasing payload
|
||||
bytes.
|
||||
|
||||
Observed result (compact wire values):
|
||||
|
||||
```text
|
||||
甲 (8 bytes) -> {ok:true, value:16}
|
||||
乙 (16 bytes) -> {ok:true, value:32}
|
||||
丙 (8 bytes) -> {ok:true, value:16} # reuses 甲's freed block
|
||||
丁 (4 bytes) -> {ok:true, value:16} # reuses the same block
|
||||
戊 (16 bytes) -> {ok:true, value:32} # reuses 乙's freed block
|
||||
己 (64 bytes) -> {ok:true, value:56}
|
||||
庚 (8 bytes) -> {ok:true, value:56} # split from 己
|
||||
辛 (16 bytes) -> {ok:true, value:72} # split remainder
|
||||
壬 (8 bytes) -> {ok:true, value:96} # split remainder
|
||||
癸 (8 bytes) -> {ok:true, value:112} # exact remainder
|
||||
failure -> {ok:false, code:"alloc.no_space",
|
||||
message:"没有足够的空闲空间"}
|
||||
payload bytes -> 44, 55, 66, 77 at 庚/辛/壬/癸
|
||||
```
|
||||
|
||||
A second region isolates coalescing from bump allocation:
|
||||
|
||||
```text
|
||||
甲2 8B, 乙2 8B, 丙2 8B -> payloads 16, 32, 48
|
||||
fill 65472B -> consumes the rest of the 64 KiB region
|
||||
free 甲2, 乙2, 丙2 -> address-adjacent blocks merge to one 48-byte block
|
||||
丁2 40B (need 48) -> {ok:true, value:16} # only the merged block fits
|
||||
戊2 8B -> {ok:false, alloc.no_space} # bump space exhausted
|
||||
```
|
||||
|
||||
Without coalescing, `丁2` would see only three 16-byte fragments and the
|
||||
exhausted bump pointer, so it would fail; the observed success at offset 16
|
||||
proves the merge.
|
||||
|
||||
The payload addresses do not overlap:
|
||||
`庚=56..71`, `辛=72..95`, `壬=96..111`, `癸=112..127`. The earlier bytes
|
||||
`甲=16..23`, `乙=32..47`, `戊=32..47` show deliberate reuse after `释放`
|
||||
(the block is not zeroed; callers must initialise a block they allocate).
|
||||
|
||||
`cargo test -p marqdo-cli --test conformance_semantics` passes with fixture 55
|
||||
included. `cargo run -p marqdo-cli -- run
|
||||
tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md --json`
|
||||
returns `success: true` with the checked-in expected value.
|
||||
|
||||
## Binary tree fixture 56
|
||||
|
||||
`tests/conformance/semantics/56-memory-binary-tree.mq.md` is a second
|
||||
`.mq.md` structure over `Memory`:
|
||||
|
||||
* 16-byte nodes: one-byte value at offset 0, left child offset at 4, right
|
||||
child offset at 8; `0` is the empty child;
|
||||
* region header at offset 0 is the bump pointer, initialised to 8;
|
||||
* `插入` is recursive and writes the child link through its parent block;
|
||||
* `遍历` is recursive in-order and builds the result with `list.concat`;
|
||||
* inserted `[5, 3, 8, 1, 4, 7, 9]`, returned:
|
||||
|
||||
```text
|
||||
root=8
|
||||
in-order=[1,3,4,5,7,8,9]
|
||||
root.left=24
|
||||
root.right=40
|
||||
```
|
||||
|
||||
No Rust tree/container algorithm, new `HostOp`, `Value` variant or Markdown
|
||||
syntax was added. `cargo test -p marqdo-cli --test conformance_semantics`
|
||||
passes with fixture 56 included.
|
||||
|
||||
## CSV row splitter fixture 57
|
||||
|
||||
`tests/conformance/semantics/57-csv-row-splitter.mq.md` splits a single CSV row
|
||||
using only `.mq.md` and the CM-A04 primitives:
|
||||
|
||||
* `text.codepoints` enumerates Unicode scalar values;
|
||||
* `text.from_codepoint` reconstructs each character;
|
||||
* `list.append` builds the field list, `list.concat` appends the final field;
|
||||
* comma is the separator; empty fields are preserved.
|
||||
|
||||
Observed:
|
||||
|
||||
```text
|
||||
"甲,乙,,丙" -> ["甲","乙","","丙"]
|
||||
"" -> [""]
|
||||
"a,b" -> ["a","b"]
|
||||
```
|
||||
|
||||
Quoted fields, embedded commas, doubled quotes and line breaks are **not**
|
||||
implemented. This is a C05 demonstration, not a claim that CSV is complete.
|
||||
|
||||
## Hash table fixture 58
|
||||
|
||||
`tests/conformance/semantics/58-memory-hash-table.mq.md` implements an
|
||||
open-addressing hash table over `Memory`:
|
||||
|
||||
* 8-byte header: element count at 0, slot count at 4; slots from offset 8,
|
||||
8 bytes each (`key:u32`, `value:u32`);
|
||||
* key `0` means empty, so business keys start at 1;
|
||||
* hash is `key % capacity` with linear probing;
|
||||
* existing keys update in place without incrementing the count; a full table
|
||||
returns `{ok:false, code:"hash.full", ...}`.
|
||||
|
||||
Observed:
|
||||
|
||||
```text
|
||||
capacity 32: insert 5->50, 37->370 (collision at slot 5), 12->120, update 5->55
|
||||
lookup 5 => 55, 37 => 370, 12 => 120, 99 => None
|
||||
count => 3
|
||||
capacity 2: insert 1, 2, then 4 => hash.full
|
||||
```
|
||||
|
||||
No Rust hash container, `HostOp` or new `Value` type was added.
|
||||
|
||||
## UTF-8 encoder fixture 59
|
||||
|
||||
`tests/conformance/semantics/59-utf8-encoder.mq.md` writes UTF-8 bytes into
|
||||
`Memory` using `text.codepoints` and byte indexing:
|
||||
|
||||
* code points below `0x80` -> 1 byte;
|
||||
* below `0x800` -> 2 bytes; below `0x10000` -> 3 bytes; otherwise 4 bytes;
|
||||
* the fixture encodes `"Aé中𝄞"` to
|
||||
`[65, 195, 169, 228, 184, 173, 240, 157, 132, 158]` with length 10.
|
||||
|
||||
## UTF-8 decoder fixture 60
|
||||
|
||||
`tests/conformance/semantics/60-utf8-decoder.mq.md` decodes UTF-8 from
|
||||
`Memory` and validates every sequence:
|
||||
|
||||
* `utf8.truncated` when a multi-byte sequence runs past the region length;
|
||||
* `utf8.invalid_continuation` when a byte outside `0x80..0xBF` appears in a
|
||||
continuation position;
|
||||
* `utf8.invalid_scalar` for overlong 3-byte forms, surrogates and values above
|
||||
`0x10FFFF`;
|
||||
* `utf8.invalid_lead` for `0x80..0xC1` and `0xF5..0xFF` lead bytes.
|
||||
|
||||
Observed:
|
||||
|
||||
```text
|
||||
[65,195,169,228,184,173] -> ok "Aé中"
|
||||
[228,184] -> utf8.truncated
|
||||
[228,65,173] -> utf8.invalid_continuation
|
||||
[237,160,128] -> utf8.invalid_scalar
|
||||
[240,157,132,158] -> ok "𝄞"
|
||||
```
|
||||
|
||||
Together with fixture 59 this closes the C04 encode/decode demonstration using
|
||||
only `.mq.md`; no Rust UTF-8 codec is called.
|
||||
|
||||
## Graph adjacency fixture 61
|
||||
|
||||
`tests/conformance/semantics/61-memory-graph-adjacency.mq.md` stores a directed
|
||||
graph in `Memory`:
|
||||
|
||||
* 16-byte node: one-byte value at 0, edge-list head `u32` at 4;
|
||||
* 8-byte edge: target node offset at 0, next edge offset at 4;
|
||||
* edges are head-inserted, so traversal returns the most recently added edge
|
||||
first;
|
||||
* nodes `甲/乙/丙` at offsets 8/24/40, edges `甲→乙`, `甲→丙`, `乙→丙`.
|
||||
|
||||
Observed:
|
||||
|
||||
```text
|
||||
甲=8, 乙=24, 丙=40
|
||||
邻接(甲) => [40, 24] # 丙 then 乙 (head insertion)
|
||||
邻接(乙) => [40]
|
||||
邻接(丙) => []
|
||||
```
|
||||
|
||||
No Rust graph container, `HostOp` or new `Value` type was added.
|
||||
|
||||
## JSON subset parser fixture 62
|
||||
|
||||
`tests/conformance/semantics/62-json-array-parser.mq.md` is a recursive
|
||||
descent parser over `text.codepoints`:
|
||||
|
||||
* supports nested arrays and objects, strings with standard escapes including
|
||||
`\uXXXX`, non-negative and negative integers, `true`/`false`/`null`,
|
||||
whitespace, and returns `{ok,value,next}`;
|
||||
* errors are `Result` maps: `json.unexpected_eof`, `json.invalid_array`,
|
||||
`json.invalid_object`, `json.invalid_string`, `json.invalid_number`,
|
||||
`json.invalid_literal`, `json.unexpected_token`;
|
||||
* floats are explicitly out of scope.
|
||||
|
||||
Observed:
|
||||
|
||||
```text
|
||||
" [1, 23, [4, 5], -6] " -> ok [1,23,[4,5],-6], next=20
|
||||
"[1 2]" -> json.invalid_array
|
||||
"[1," -> json.unexpected_eof
|
||||
{"a": 1, "b": [true, null, "说\"好"]}
|
||||
-> ok {a:1, b:[true,null,"说\"好"]}, next=35
|
||||
"\"a\\nb\"" -> ok "a\nb", next=6
|
||||
```
|
||||
|
||||
No Rust JSON parser, new `Value` type or `HostOp` was added.
|
||||
|
||||
## Bytecode VM fixture 63
|
||||
|
||||
`tests/conformance/semantics/63-memory-bytecode-vm.mq.md` is a small stack
|
||||
bytecode machine written entirely in `.mq.md`:
|
||||
|
||||
* program bytes at offset 0; stack at a caller-provided `Memory` offset;
|
||||
* instruction set: `1=PUSH imm8`, `2=ADD`, `3=MUL`, `5=JMP`, `6=JZ`, `7=HALT`;
|
||||
* state (`pc`, `sp`, running, error) is ordinary local state; the stack and
|
||||
program live in `Memory`;
|
||||
* `(5 + 7) * 3` executes to `{ok:true, value:36}`;
|
||||
* a `JZ` program jumps over a `PUSH 99` and returns `{ok:true, value:42}`.
|
||||
|
||||
No Rust VM, opcode enum, `HostOp` or new `Value` type was added.
|
||||
|
||||
## Integer codecs fixture 64
|
||||
|
||||
`tests/conformance/semantics/64-memory-integer-codecs.mq.md` provides generic
|
||||
big-endian and variable-length integer codecs over `Memory`:
|
||||
|
||||
* `写大端(内存,基址,值,宽度)` writes the least significant byte last, so the
|
||||
result is big-endian for widths 2, 4 and 8;
|
||||
* `读大端(内存,基址,宽度)` rebuilds the integer with `value * 256 + byte`;
|
||||
* `写变长`/`读变长` are LEB128 (7-bit continuation) with the consumed byte
|
||||
count.
|
||||
|
||||
Observed:
|
||||
|
||||
```text
|
||||
u16 4660 -> 4660
|
||||
u32 305419896 -> 305419896
|
||||
u64 1311768467463790320 -> 1311768467463790320
|
||||
u64 18446744073709551615 -> 18446744073709551615
|
||||
varint 123456789 -> 4 bytes, decoded 123456789, consumed 4
|
||||
last two bytes of u64::MAX -> 255, 255
|
||||
```
|
||||
|
||||
No Rust codec builtin or new `Value` type was added.
|
||||
|
||||
## Table sort fixture 65
|
||||
|
||||
`tests/conformance/semantics/65-table-sort-stdlib.mq.md` sorts a table by a
|
||||
column using only CM-A04 primitives:
|
||||
|
||||
* `table.row` extracts rows, `table.cell` compares the sort column;
|
||||
* `插入有序` builds a new row list with `list.append` and binary condition
|
||||
branches (insertion sort);
|
||||
* `表(列名, 已排序)` rebuilds the result; the input table is unchanged.
|
||||
|
||||
Observed:
|
||||
|
||||
```text
|
||||
原表 rows: 甲87, 乙92, 丙71
|
||||
升序 rows: 丙71, 甲87, 乙92
|
||||
原表: unchanged
|
||||
```
|
||||
|
||||
No Rust table sort, `HostOp` or new `Value` type was added.
|
||||
|
||||
## Table join fixture 67
|
||||
|
||||
`tests/conformance/semantics/67-table-join-stdlib.mq.md` joins two tables on a
|
||||
column using only CM-A04 primitives:
|
||||
|
||||
* nested `table.row` / `table.cell` loops compare the join keys;
|
||||
* matches append `list.concat(left_row, right_row)` to the output rows;
|
||||
* `表(list.concat(headers...), rows)` rebuilds the result; both inputs are
|
||||
unchanged.
|
||||
|
||||
Observed: left `甲/乙/丙`, right `乙/甲/丁` on column 0 yields the two matched
|
||||
rows `甲 87 甲 B` and `乙 92 乙 A`. No Rust join or `HostOp` was added.
|
||||
|
||||
## Quoted CSV fixture 68
|
||||
|
||||
`tests/conformance/semantics/68-csv-quoted-row.mq.md` parses one CSV row with
|
||||
quoted fields:
|
||||
|
||||
* `"..."` quotes fields, `""` is a literal quote, commas inside quotes are
|
||||
preserved;
|
||||
* empty fields are preserved;
|
||||
* multi-line fields and whitespace trimming are out of scope.
|
||||
|
||||
Observed:
|
||||
|
||||
```text
|
||||
甲,"乙,丙","说""好" -> ["甲", "乙,丙", "说"好"]
|
||||
a,,c -> ["a", "", "c"]
|
||||
"" -> [""]
|
||||
"a""b",c -> ["a"b", "c"]
|
||||
```
|
||||
|
||||
No Rust CSV parser or new `Value` type was added.
|
||||
|
||||
## What remains open
|
||||
|
||||
* JSON floats, table group, and an independent `marqdo-stdlib` repository;
|
||||
* CM-A06 `Resource`/`Stream` byte-transfer ABI; EOF/partial semantics are
|
||||
reserved by ADR-0024 but not implemented;
|
||||
* CM-A07–CM-A11 (stdlib naming, Table storage, provider ABI, packaging,
|
||||
strict resource policy) and P3–P6;
|
||||
* the RFC §10.5 frozen-CLI closure lane and Windows/macOS evidence.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
cargo test -p marqdo-cli --test conformance_semantics
|
||||
cargo run -q -p marqdo-cli -- run \
|
||||
tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md --json
|
||||
cargo test --workspace --locked
|
||||
```
|
||||
@@ -0,0 +1,94 @@
|
||||
# RFC-CM CM-A07 Compat-A evidence: `.mq.md` print prototype
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `a113d89` plus the uncommitted RFC-CM P0–P3 changes.
|
||||
Decision record: [ADR-0027](../adr/0027-print-migration-compat-a.md)
|
||||
Related: [Resource P3](2026-11-20-resource-stream-p3.md),
|
||||
[file write](2026-11-20-resource-file-write-p3.md), [STATUS §5](../STATUS.md).
|
||||
Scope: **Compat-A only**. Builtin `打印` is not migrated; no alias routing and
|
||||
no removal of `HostOp::Print`.
|
||||
|
||||
## Environment and gate
|
||||
|
||||
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
407 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
|
||||
```
|
||||
|
||||
No GitHub CI is available from this checkout; this is local evidence.
|
||||
|
||||
## Prototype
|
||||
|
||||
Ordinary `.mq.md` modules under `examples/stdlib/`:
|
||||
|
||||
| File | Responsibility |
|
||||
|---|---|
|
||||
| `format.mq.md` | `显示` for Unit/None/Bool/Int/Float/Text/List/Map/Table, `值显示` (repr), `转义文本`, `列表显示`, `映射显示`, `表格显示`, `连接文本` |
|
||||
| `text.mq.md` | `utf8编码(文本, 内存, 基址)` — 1–4 byte UTF-8 into `Memory` |
|
||||
| `io.mq.md` | `写文本` (encode → `resource.write(stdout)` → close) and `写值` (format + newline) |
|
||||
| `demo.mq.md` | side-by-side `io.写值(x)` and builtin `打印(x)` for a nested list and a table |
|
||||
|
||||
The call chain is exactly the RFC target shape:
|
||||
|
||||
```text
|
||||
io.写值(value)
|
||||
-> format.显示(value) # .mq.md value formatting, no Rust display()
|
||||
-> text.utf8编码(text, memory) # .mq.md UTF-8 encoder
|
||||
-> resource.open(stdout)
|
||||
-> resource.write(memory, ...) # bytes through Memory
|
||||
-> resource.close(stdout)
|
||||
```
|
||||
|
||||
No `HostOp`, no new builtin, no parser change and no special stdlib dispatch.
|
||||
The module is loaded only through an explicit Markdown import alias.
|
||||
|
||||
## Output comparison
|
||||
|
||||
```text
|
||||
$ marqdo run examples/stdlib/demo.mq.md
|
||||
["文本", 2, 1.5, [3, true], {a: 1, b: "x"}, None]
|
||||
["文本", 2, 1.5, [3, true], {a: 1, b: "x"}, None]
|
||||
Table[名称 | 分数]{甲 | 87; 乙 | 92}
|
||||
Table[名称 | 分数]{甲 | 87; 乙 | 92}
|
||||
```
|
||||
|
||||
The first and third lines are `io.写值`; the second and fourth are builtin
|
||||
`打印`. They are byte-identical.
|
||||
|
||||
CLI test:
|
||||
`crates/cli/tests/examples_r6.rs::mqmd_stdlib_print_output_matches_builtin_print`
|
||||
asserts exactly four lines, pairwise equality, and the exact list/table text.
|
||||
|
||||
## Covered and uncovered values
|
||||
|
||||
Covered by Compat-A comparison: `Unit`, `None`, `Bool`, `Int`, `Float`, `Text`,
|
||||
`List`, `Map`, `Table`.
|
||||
|
||||
Not covered: `Formula`, `Code`, `FunctionRef`, `PluginHandle`. The language
|
||||
still has no public observation for the module/name/handle identity needed to
|
||||
reproduce `Value::display()`/`repr()` for those types; adding it is a CM-A04
|
||||
extension, not part of this slice. This is the explicit blocker for Compat-B.
|
||||
|
||||
## What is not migrated
|
||||
|
||||
* `打印`/`print` still resolves to the existing builtin and `HostOp::Print`.
|
||||
* No alias table, no bundled stdlib resolution, no `std` package.
|
||||
* No stdlib version/digest or offline distribution (CM-A10).
|
||||
* No Compat-B/C comparison lane or removal of the Rust `Value::display()` path.
|
||||
* The demo/stdlib is an `examples/` prototype, not a shipped standard library.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
cargo run -q -p marqdo-cli -- run examples/stdlib/demo.mq.md
|
||||
cargo test -p marqdo-cli --test examples_r6
|
||||
cargo test --workspace --locked
|
||||
```
|
||||
@@ -0,0 +1,110 @@
|
||||
# RFC-CM CM-A04 evidence: minimal structure observation and construction
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `a113d89` plus the uncommitted RFC-CM P0/P1/P2 changes.
|
||||
Decision record: [ADR-0023](../adr/0023-minimal-structure-observation-and-construction.md)
|
||||
Related: [P1 memory evidence](2026-11-20-memory-p1.md),
|
||||
[P2 closure evidence](2026-11-20-memory-p2-closure.md),
|
||||
[STATUS §5](../STATUS.md).
|
||||
Scope: CM-A04 only. The full RFC-CM-0002 P2 gate is **not** passed.
|
||||
|
||||
## Environment and gate
|
||||
|
||||
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
47 test binaries, 389 passed, 0 failed, 2 ignored (doc-tests)
|
||||
```
|
||||
|
||||
The 2 ignored tests are the pre-existing `marqdo-plugin-sdk` doc-tests. No
|
||||
GitHub CI was available from this checkout; this is local evidence.
|
||||
|
||||
## Builtins added (pure, no `HostOp`, no capability)
|
||||
|
||||
| Name | Semantics |
|
||||
|---|---|
|
||||
| `list.append` | new list with one appended value; input unchanged |
|
||||
| `list.concat` | new concatenated list; inputs unchanged |
|
||||
| `map.keys` | Text keys in canonical sorted order |
|
||||
| `map.has` | key membership |
|
||||
| `map.get` | stored value or `None`; `map.has` disambiguates a stored `None` |
|
||||
| `map.set` | new map with the key; input unchanged |
|
||||
| `map.remove` | new map without the key; missing key is a no-op |
|
||||
| `text.codepoints` | Unicode scalar values as `Int`, in order |
|
||||
| `text.from_codepoint` | one-scalar Text, or `None` for surrogate/out-of-range |
|
||||
| `table.columns` | header names in column order |
|
||||
| `table.row` | new list of row cells; non-negative coordinate |
|
||||
| `table.cell` | cell value; non-negative row/column |
|
||||
|
||||
No `sort`, `filter`, `join`, `group`, JSON/CSV, `table.set_cell`, mutable
|
||||
List/Map or new Markdown syntax was added. Constructors never mutate their
|
||||
arguments; this preserves INV-02.
|
||||
|
||||
## Semantic fixtures
|
||||
|
||||
```text
|
||||
51-list-map-value-semantics:
|
||||
original list [1,2] unchanged; appended [1,2,3]
|
||||
original map {a:1,b:2} unchanged; map.set c=3 then remove a
|
||||
keys after remove => ["b","c"]
|
||||
map.has(original, "c") => false
|
||||
map.get(original, "z") => None
|
||||
list.concat([1],[2,3]) => [1,2,3]
|
||||
|
||||
52-text-codepoints:
|
||||
text.codepoints("aé中") => [97, 233, 20013]
|
||||
text.from_codepoint(20013) => "中"
|
||||
text.from_codepoint(0xD800) => None
|
||||
text.from_codepoint(0x110000) => None
|
||||
|
||||
53-table-projection:
|
||||
columns => ["名称","分数"]
|
||||
row 1 => ["乙",92]
|
||||
cell(0,1) => 87
|
||||
长度(table) => 2
|
||||
|
||||
54-table-filter-stdlib:
|
||||
原表 3 rows, threshold 80
|
||||
结果 table has 2 rows (甲 87, 乙 92)
|
||||
原表 is still 3 rows and byte-for-byte the same logical table
|
||||
```
|
||||
|
||||
Fixture 54 is the first `.mq.md` algorithm over the new primitives: it uses
|
||||
`table.columns`, `table.cell`, `table.row`, `list.append` and `表` to rebuild a
|
||||
filtered table. It is a conformance document, not a shipped `table.filter`
|
||||
builtin; no Rust table algorithm was added.
|
||||
|
||||
CLI confirms all three return `success: true` with the checked-in wire values.
|
||||
|
||||
## Runtime negatives
|
||||
|
||||
| Fixture | Expected code | Observed |
|
||||
|---|---|---|
|
||||
| `19-list-append-wrong-type` | `runtime.type` | `runtime.type` |
|
||||
| `20-table-cell-out-of-bounds` | `runtime.index` | `runtime.index` |
|
||||
| `21-map-key-not-text` | `runtime.type` | `runtime.type` |
|
||||
|
||||
## Focused command
|
||||
|
||||
```text
|
||||
cargo test -p marqdo-cli --test conformance_semantics --test negative
|
||||
semantic_goldens_match_end_to_end_execution ... ok
|
||||
runtime_negative_goldens ... ok
|
||||
```
|
||||
|
||||
## What remains open
|
||||
|
||||
* CM-A05 recoverable `Result`/EOF/partial model. This report predates ADR-0024,
|
||||
which now freezes the `Result` map convention; the Stream EOF/partial part
|
||||
remains CM-A06. `map.get` returning `None` is a lookup convention, **not**
|
||||
that contract.
|
||||
* CM-A06 `Resource`/`Stream` ABI, CM-A07 stdlib naming/alias, CM-A08 Table
|
||||
storage/physical migration, CM-A09 provider ABI, CM-A10 stdlib packaging,
|
||||
CM-A11 strict resource policy.
|
||||
* A separate `marqdo-stdlib` repository, graph containers, a full JSON parser,
|
||||
quoted/escaped CSV and the RFC §10.5 frozen-CLI closure lane.
|
||||
* Windows/macOS native lanes. No remote CI URL exists for this working tree.
|
||||
Reference in new issue
Block a user