From 23e68cba5f5fd82be907e132513f0d43db960ab7 Mon Sep 17 00:00:00 2001 From: cflmy Date: Sat, 21 Nov 2026 00:58:34 +0800 Subject: [PATCH] docs(resource): record stdio stream endpoints and stdlib APIs --- ...est_Architecture_Audit_and_Refactor_RFC.md | 468 ++++++++++++++++++ docs/STATUS.md | 24 +- docs/adr/0030-print-compat-b-routing.md | 6 +- ...0-stdio-stdin-stderr-and-stdlib-streams.md | 60 +++ docs/architecture/README.md | 5 +- docs/evidence/2026-11-21-stdio-streams.md | 30 ++ spec/evaluation.md | 22 +- spec/security.md | 15 +- tests/README.md | 12 +- 9 files changed, 608 insertions(+), 34 deletions(-) create mode 100644 docs/Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md create mode 100644 docs/adr/0040-stdio-stdin-stderr-and-stdlib-streams.md create mode 100644 docs/evidence/2026-11-21-stdio-streams.md diff --git a/docs/Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md b/docs/Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md new file mode 100644 index 0000000..f53a535 --- /dev/null +++ b/docs/Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md @@ -0,0 +1,468 @@ +# Marqdo CLI 最新提交架构审查与重构 RFC + +> 状态:**审查报告 / 待讨论设计,不是已实施代码** +> 仓库:[`cflmy/marqdo-cli`](https://github.com/cflmy/marqdo-cli) +> 冻结审查提交:[`7fbe069196e61d06a00d64fe8d65049c327373eb`](https://github.com/cflmy/marqdo-cli/commit/7fbe069196e61d06a00d64fe8d65049c327373eb) +> 编写日期:2026-11-21;依据 GitHub 仓库树、源码、ADR、spec、stdlib、测试文件和状态记录,**没有在本环境重新构建或执行完整测试**。 +> 重要约束:**标准库继续保存在同一仓库**;保持 Markdown 原生语法、`std:` 导入、能力授权及既有稳定语义。 + +## 0. 审查结论 + +**不是计算模型需要再推倒重来,而是 `语言核心 → 运行时 → 平台边界 → 标准库 → CLI` 的职责未完成收敛。** + +- 已有 `Memory` 和 `resource.*`,意味着 RFC-CM-0001/0002 的重要骨架已落地;不能再把“缺少独立内存”“打印仍由 `HostOp::Print` 实现”当作现状。 +- 但标准库目前主要是 `format.mq.md`、`text.mq.md`、`io.mq.md`、`data/table.mq.md`、`net/http.mq.md`;`io` 仅提供 `写文本/写值`,缺乏输入、文件和面向用户的通用流操作。 +- 仓库有大量**证明可以写**的 `.mq.md` 测试,但它们不等于已经**作为标准库产品发布**:`JSON`、CSV、分配器、UTF-8 解码、字节码虚拟机等主要见于 `tests/conformance/semantics/`。 +- 现有 `resource.*` 与旧 `fs.read/write` 并存;`Host` 仍有数学、Foreign、Plugin 等操作;Provider、Plugin、Foreign Kernel、Tool 具有不同职责,但其注册、策略、诊断、输送方式的实现入口日益分散。 +- 物理文件组织尚可维护,不应以“crates 多”为由全面合并。真正要减少的是**同类逻辑的实现入口**、大文件承担多种职责、文档互相矛盾,以及把测试夹具当成用户标准库的错觉。 + +**本 RFC 的最高优先级:构建一个可靠的 `stdin/stdout/stderr + file + byte streams` 底层端点;基于它用 `.mq.md` 交付完整 `std:io` / `std:fs` / `std:data`;在此之后再拆分超大模块、收敛扩展注册面。** + +## 1. 事实依据与边界 + +### 1.1 已实现、但容易被忽略的能力 + +| 能力 | 最新代码中的事实 | 限制 | +|---|---|---| +| 自有计算 | Markdown/Expression/Frontend/IR/Interpreter 均已独立实现 | 仍为 tree-walk,未实现 native optimizer/JIT | +| 程序可寻址内存 | `marqdo-runtime/src/memory.rs`:独立区域、64 KiB 页、增长、字节读写、别名共享、运行级预算 | 区域在整次 run 中存在;无语言级释放整个 region;默认每区域16 MiB、每次运行64 MiB | +| 位置更新 | `m[i] = byte` 已实现 | 并非通用的 `list[i]=...` / `map[k]=...` 原地可变引用语义;普通集合仍为值语义 | +| 结构操作 | `list.append/concat`、`map.keys/has/get/set/remove`、`table.columns/row/cell`、`text.codepoints/from_codepoint` | 只是一组必需底层操作,不能替代完整标准库 | +| 标准库载入 | `stdlib/` 同仓库;`std:` 显式导入;`stdlib.json` 摘要校验 | 缺安装包一致性和 API 分层的完整验收;不要拆库 | +| 打印 | `print/打印` 通过加载 `stdlib/io.mq.md` 的 `写值` 执行;原 `HostOp::Print` 已移除 | stdlib 输出路径仍只写一次、固定一页缓冲、未逐项检查 Result | +| 底层文件 | `resource.open("file",..., ["read"]/["write"])`;`fs.read/write` 遗留接口 | 读取时整文件载入、有 8 MiB 限额;写入时缓冲到 close;没有完整 `std:fs` | +| TCP | `resource.open("tcp", IP:port,["read","write"])` | IP 字面量、同步、5 秒超时;缺 DNS、TLS、监听、UDP | +| 资源 Provider | 进程内、worker、C ABI 等实现已经存在 | 与 Plugin、Foreign 的注册和安全边界重叠,不能认为它们都是一种 API | +| 严格隔离 | Linux Bubblewrap 且探测成功时严格执行;本地 VM 有证据 | 不能称 Windows/macOS 或所有 Provider 已获相同保障 | + +### 1.2 标准库中哪些是真正的公开 API? + +此 HEAD 的正式 `stdlib` 函数源码集中于 `format.mq.md`、`text.mq.md`、`io.mq.md`、`data/table.mq.md`、`net/http.mq.md`,另有 `demo.mq.md` 和三个测试文档。现状**不是空标准库**,但明显没有达到日常 CLI 所需的覆盖范围。 + +已在 conformance 中证明,但仍需产品化的候选: + +- `tests/conformance/semantics/55-first-fit-free-list-allocator.mq.md`:空闲链表分配器; +- `59-utf8-encoder` / `60-utf8-decoder`:编码/校验解码; +- `62-json-array-parser`:JSON 解析相关能力; +- `63-memory-bytecode-vm`:虚拟 CPU/字节码; +- `68-csv-quoted-row`:引号转义 CSV; +- `70-matrix-multiply`、`71-http-parser`、`72-table-columnar`、`73-sparse-matrix`。 + +**结论:测试证明的是表达能力,标准库证明的是用户能依赖的 API 及错误、性能、版本契约;两者不能相互代替。** + +### 1.3 可以明确定位的结构偏移 + +1. `crates/cli/src/commands.rs` 约 1539 行,同时承担加载调用、运行环境组装、严格隔离启动、工具输出、依赖命令和文件写入;`cli/src/loader.rs` 也掺入特定 `print` 识别/重定向。CLI 过度承担执行策略。 +2. `crates/marqdo-host/src/host.rs` 约 1138 行,负责 `HostOp`、资源分发、权限与 Provider 注册;`resource.rs` 约 540 行混合 stdout、文件、TCP、Provider 句柄。资源能力正确,但组织已超出一个文件的合适边界。 +3. `marqdo-runtime` 依赖 `marqdo-host`,即语言运行内核直接依赖具体平台边界模块,尚未做到执行引擎与平台实现清楚的依赖方向;是否引入一个薄接口 crate 必须根据下一阶段的独立嵌入需求决定,禁止盲目增 crate。 +4. `HostOp` 同时拥有高层数学内核、Foreign、Plugin、遗留 `fs.*`,又存在新的 `resource.*`;**高级操作与基础系统操作在一个分发层并存**。 +5. `Plugin ABI` / `Resource Provider` / `Foreign Kernel` / `CLI Tool` 在功能上不同,但基础注册、授权、版本、签名、传输、诊断等通用机制缺少统一的架构地图。删除其中某一种协议会伤害已有消费者;应先定义所有权,后考虑传输层复用。 +6. 文档状态漂移:`docs/architecture/README.md` 仍称 P5 Provider 未实现,`docs/evidence/2026-11-20-rfc-cm-completion-audit.md` 却记录 Provider/P5 的本地验收;`docs/STATUS.md` 把较老的 `c9f01aa` 描述为 GitHub master,而此次实际最新为 `7fbe069`。**应把状态“生成与冻结”机制纳入重构,而非靠人手更新数十份计划。** +7. `stdlib/io.mq.md` 的“写文本”总是 `memory.new(1,1)`,把 Unicode 写入缓冲后只 `resource.write` 一次,不检查 `{ok, count}`;输入大于 64 KiB 会越界/失败,潜在部分写失败被吞掉。这个问题属于标准库正确性,不是“再加一个 Rust println”能解决的。 +8. `stdlib/net/http.mq.md` 以 Unicode 码点文本为基础解析响应和 chunked 数据,而 HTTP 的长度以**字节**为单位;对非 ASCII 正文、混合编码、大小写不敏感头字段、碎片读取和消息上限不能据此宣称完整 HTTP/1.1 实现。 + +## 2. 目标架构:五个清晰的责任层 + +```text + 用户 .mq.md 程序 / Agent / 科研工作流 + │ + stdlib/* (.mq.md,仓库内版本化与签名/摘要) + ├─ core/collections/text/encoding/data/math + ├─ io/console/stream/fs/net/process + └─ memory/alloc + 可移植纯算法 + │ + 语言语义层:Markdown → AST → Frontend → IR + │ + 执行层:Value / MemoryStore / evaluator / budgets + (只描述计算,不拥有 stdout 文件句柄) + │ + 受控平台接口:open/read/write/close + 非流式请求 + ├─ capability / budget / audit / tracing + ├─ stdio / file / tcp / OS 适配器 + └─ 显式注册的 Provider(可 in-process / worker / C) + │ + 操作系统 / 外部服务 / 设备 + + CLI 只做 Composition Root:参数、工程加载、装配、执行、输出 + Plugin callable / Foreign legacy / Tool protocol = 并列的扩展类别, + 由独立注册模块负责,不直接污染标准库或语言 IR。 +``` + +### 2.1 宪法性边界(应成为 CI 架构检查规则) + +- **语言语义内核**:表达式、控制流、局部变量、List/Map/Table 语义、数值和字节行为、内存区域、值与位置赋值、函数/文档引用、错误源位置。它不应知道 `stdout`、项目目录、TCP 超时或第三方插件路径。 +- **平台接口**:负责不可纯计算实现的 I/O、系统时钟、熵源、网络、文件句柄、权限校验;这些接口的宿主实现是必要的,但不能承载 JSON 解析、文本格式化、HTTP 语义或内存分配策略。 +- **标准库**:以 `.mq.md` 实现普通 API、协议、算法、容器管理。**普通能力新增时,只改 stdlib + tests + docs,不改 Rust Builtin。** +- **CLI**:只负责选择输入项目、解析命令、加载配置、装配 Host 和 Runtime、路由人类/机器输出;不是语义策略中心。 +- **扩展接口**:原生函数扩展和资源扩展都经同一授权审计机制,但函数调用与有状态流仍保留不同能力类型。 + +### 2.2 引擎与平台的依赖方向 + +当前 `marqdo-runtime → marqdo-host` 并非立即必须合并或新增接口 crate 的理由。分两步: + +**第一步**:把所有 I/O 分发和许可策略严格收敛到 `marqdo-host`,`runtime` 不再增加外部操作分支;建立可以注入 FakeHost 的语义测试。 + +**第二步(当支持嵌入式 Runtime 或第二个执行后端时)**:引入最薄、无平台依赖的 `ExecutionPort` 契约,并让 `marqdo-host` 实现它。若 Rust crate 会形成环依赖,才考虑一个不足数百行的共享接口 crate;**禁止为每个端口和 Provider 单独设 crate**。 + +## 3. 目标目录(同一仓库,不拆 stdlib) + +只展示重要目录;不代表一次性移动所有 crate: + +```text +marqdo-cli/ +├── Cargo.toml +├── crates/ +│ ├── marqdo-markdown/ # 只保留 Markdown profile、AST、source span +│ ├── marqdo-expression/ # 表达式词法、语法与 AST +│ ├── marqdo-frontend/ # 映射、名称分析、静态诊断 +│ ├── marqdo-ir/ # 稳定执行 IR;不积累高层 API +│ ├── marqdo-value/ # 值语义及 wire 合约 +│ ├── marqdo-runtime/ +│ │ └── src/ +│ │ ├── interpreter.rs +│ │ ├── memory.rs +│ │ ├── place.rs # 可赋值位置与统一读取/写入 +│ │ ├── primitives/ # 确实不可由 .mq.md 有效组合的计算原语 +│ │ └── lib.rs +│ ├── marqdo-host/ +│ │ └── src/ +│ │ ├── session.rs # 每次执行的 Host 状态、资源预算 +│ │ ├── resource/ +│ │ │ ├── table.rs # 句柄登记、生命周期、权限 +│ │ │ ├── stdio.rs +│ │ │ ├── file.rs +│ │ │ ├── tcp.rs +│ │ │ └── provider.rs +│ │ ├── providers/ # registry/native/worker/C 实现适配 +│ │ ├── effects/ # auditing、trace、capability +│ │ ├── legacy/ # 旧 fs / foreign 迁移包装 +│ │ └── lib.rs +│ ├── marqdo-package/ # import / lock / package identity +│ ├── marqdo-sandbox/ # OS 隔离证明,绝不移入 stdlib +│ ├── marqdo-foreign/ # 可选兼容执行器,非内核 +│ ├── marqdo-math/ # 临时可选原生数学实现,逐项确立归属 +│ ├── marqdo-plugin-abi/ # 原生可调用扩展协议 +│ ├── marqdo-plugin-sdk/ +│ ├── marqdo-provider-worker/ +│ └── cli/ +│ └── src/ +│ ├── main.rs +│ ├── args.rs +│ ├── commands/ # run, call, check, inspect, package, tooling +│ ├── project/ # loader, stdlib resolver, project context +│ ├── assembly.rs # RunSession 的惟一装配位置 +│ └── output.rs +├── stdlib/ # 保留在同一个 git 仓库 +│ ├── stdlib.json # 继续 digest 校验 +│ ├── core/result.mq.md +│ ├── collections/list.mq.md +│ ├── collections/map.mq.md +│ ├── text/utf8.mq.md +│ ├── text/format.mq.md +│ ├── text/scan.mq.md +│ ├── memory/alloc.mq.md +│ ├── io/stream.mq.md +│ ├── io/console.mq.md +│ ├── fs/file.mq.md +│ ├── fs/path.mq.md +│ ├── net/tcp.mq.md +│ ├── net/http1.mq.md +│ ├── data/json.mq.md +│ ├── data/csv.mq.md +│ ├── data/table.mq.md +│ ├── io.mq.md # 兼容门面,保留旧 std:io +│ ├── format.mq.md # 兼容门面,保留旧 std:format +│ ├── text.mq.md # 兼容门面,保留旧 std:text +│ └── tests/ # 对公共 std: API 做产品级测试 +├── tests/ +│ ├── conformance/ # 语义契约(无需变成文档产品) +│ ├── architecture/ # import graph / primitive-only tests +│ ├── e2e/ # 纯 Marqdo CLI,输入/文件/HTTP/JSON 等 +│ └── security/ +├── spec/ # 真正冻结的语言和 ABI 规范 +├── docs/ +│ ├── architecture/ # 唯一当前架构总览 + RFC 决策索引 +│ ├── adr/ # 只记录决策,不冒充实时状态 +│ ├── evidence/ # SHA 固定的事实快照 +│ ├── history/ # 已过时规划 +│ └── STATUS.md # 自动采集或引用当前状态来源 +└── tools/ # 打包、CI、文档一致性检查和 benchmark +``` + +**迁移原则:先移动职责,再移动路径。** 不因为希望目录好看就制造公共 API 大面积重命名;已公开的 `std:io`、`std:format`、`std:text`、`std:data/table`、`std:net/http` 必须有兼容导入门面和逐版本弃用协议。 + +## 4. 系统 I/O 的最小充分基座 + +### 4.1 必须补上的真正底层能力 + +当前 `resource.open("stdio","stdout",["write"])` 已有。**缺少 `stdin` 读端点**,这是不能只用已有资源来拼出的新 OS 交互能力。应扩展同一 `stdio` 资源类别,不增加语言关键字、`Input` builtin 或第二套控制台协议。 + +目标: + +```text +resource.open("stdio", "stdin", ["read"]) -> Resource +resource.open("stdio", "stdout", ["write"]) -> Resource +resource.open("stdio", "stderr", ["write"]) -> Resource +resource.read(handle, Memory, offset, max_bytes) -> Result({count,eof}) +resource.write(handle, Memory, offset, length) -> Result({count}) +resource.close(handle) -> Result +``` + +stdin 与 stdout 使用同一个 `ResourceTable`、同一个权限检查和同一 Trace 契约。`stderr` 必须区分**应用输出**与 CLI 自身的诊断,以免 `--json` 机器可读输出混合无关数据。所有端点均遵守可取消/超时与资源预算(第一版可采用同步阻塞,但规范明确等待策略与取消路径)。 + +**不能将 `read_line` 作为新 Rust Builtin。** 它应位于 `stdlib/io/console.mq.md`,由 `resource.read` + 有界缓冲 + 增量 UTF-8 解码 + 换行判定构建。 + +### 4.2 修复现有 `stdlib/io.mq.md` 的正确性缺陷 + +现状:固定 `memory.new(1,1)`,单次 `resource.write`,无 `{ok,count}` 校验。目标 `write_all` 必须: + +1. 将任意长度文本按**字节**分片编码,或按需增长 Memory;禁止超过区域上限时静默截断。 +2. `resource.write` 若 `{ok:false}`,返回带原因的 `Result`;若 `0 < count < requested`,继续写剩余字节;若 `count=0` 且有剩余,返回 `io.write_zero` 防止死循环。 +3. 不因 `> resource.write(...)` 语句返回值被忽略而掩盖错误;`打印` 的失败须有明确可观测结果。若要求完全兼容旧 `print` 的致命失败行为,需要规定 `Result` 到致命错误的桥接方式,不能在 `.mq.md` 中擅自吞错。 +4. 使用统一缓冲生命周期与容量预算;stdout、文件和 TCP 共享 `write_all`,只在入口处选择资源。 +5. 确保多字节 Unicode、嵌入 NUL、超过一页、短写、失效资源、无权限、EOF 等均有测试。 + +### 4.3 文件:不是没有底层能力,而是接口不完整 + +目前 `resource.open("file", path, ["read"]/["write"])` 可工作。但实现为:读取时整文件转入缓冲(8 MiB 限制),写入时缓存在 Host 并于 `close` 提交。这是当前阶段的安全简化,不是通用 OS 文件流。 + +**标准库立即可构造**:`read_all`、`read_text`、`write_text`、`write_bytes`、`copy_file`(限当前文件容量)、统一 close/error 检查。 + +**需要真实的底层新增**:开放大文件的增量读写;追加和 seek;stat/exists/listdir/mkdir/remove/rename 等非流式路径操作。后二类不能通过 `resource.read/write` 神奇推导出来。 + +推荐能力合同分两类: + +- `Resource Stream`:打开、读、写、关闭,负责字节流和文件位置。 +- `Resource Operation`:受版本协议和**逐操作授权**的元信息/目录修改请求,用于不能归结为流的系统命令;候选形态 `resource.request(scope, operation, arguments)`,**不是无授权的万能 syscall**。引入前需 ADR 固定操作命名、路径规范化、能力范围、返回错误、审计与默认拒绝。 + +第一版不扩充复杂的文件事件监听、mmap 及目录枚举细节;只实现能支撑真实 CLI 文件工具的最小操作组。旧 `fs.read/write` 保持一段时间的兼容 shim,最终统一到 Resource 路线,避免双路径权限规则。 + +### 4.4 网络:低层缺口与 stdlib 缺口分别处理 + +- 现在:IP-literal TCP、同步、5 秒超时、HTTP `.mq.md` 请求构造和部分响应解析。 +- 标准库:`tcp.connect/read_exact/write_all`、可增量的 HTTP/1.1 状态机、二进制 body、Content-Length 和 chunked 字节协议、错误 Result。 +- 新 OS/Provider 能力:DNS 解析、TLS(可通过经过审计的安全传输 Provider)、监听、UDP、系统证书存储和可信熵源。**不应宣称普通裸字节内存自动生成安全 TLS 或 DNS 能力。** +- 当前 `stdlib/net/http.mq.md` 不应以码点长度处理分块字节长度;在面向通用网络发布之前先改为字节流解析。 + +## 5. 功能归属:内核 / 平台 / 标准库 / 可选加速 + +| 能力 | 推荐归属 | 是否应新增 HostOp / Rust 算法 | +|---|---|---| +| `Int/Float/Bool` 算术与比较、函数控制流 | 内核 | 核心原语,保留 | +| `Memory` 区域创建、增长、边界检查、字节访问 | 内核 | 保留;由解释器维护抽象存储 | +| `List/Map/Table` 必要结构观察与构造 | 内核 | 保留最小闭包,不增排序等算法 | +| `type/len`、结构访问、值/位置语义 | 内核 | 保留契约,慎重扩展 | +| 动态分配器、free-list、内存池、链表和树 | `stdlib/memory`、`collections` | 不增加 Rust 算法 | +| UTF-8 解码、文本转义、数字格式化 | `stdlib/text` | 原生可作为等价加速,不拥有唯一语义 | +| `打印`、输入行、文件复制、日志格式化 | `stdlib/io` / `fs` | 不增高层 Builtin;底层系统操作必要 | +| JSON/CSV、GFM 表格变换、HTTP 解析 | `stdlib/data` / `net` | 纯 `.mq.md` 必须有参考实现 | +| `std:fs` 目录/元数据/重命名 | 标准库包装 + 平台 `Resource Operation` | 需要**通用能力端点**,不是每个函数单独 HostOp | +| OS 文件句柄、网络 socket、stdio、时间、熵源 | Host / OS adapter | 必须有少量可信系统调用实现 | +| 数学符号求值/微分、科学优化 | 可选 library 或加速层 | 逐项评估;不要放进语言语义基石 | +| Python/Node Foreign | 可选兼容扩展 | 不作为标准库存在性的依赖 | +| 原生插件导出 | Callable Plugin | 不能替代普通自举算法 | +| 资源 Provider | Resource Provider | 不能替代普通 `std:io` | +| `marqdo-` 工具 | CLI Tool Protocol | 与应用运行时 API 严格分离 | + +特别说明:当前的 `Value::Table` 使用 `Vec>` 作为存储并不意味着架构失败。先保证 GFM 源码、逻辑 Table、物理存储三层分离、确认普通 Table 的值语义,再按性能证据改存储;优先建设**能直接使用的表格 API**而不是立即上列存引擎。 + +## 6. 扩展体系如何收敛,而不推倒已有工作 + +**不要粗暴删除四种看起来类似的扩展方式**:它们的边界不同。 + +| 当前机制 | 保留的真正职责 | 应停止承担的职责 | +|---|---|---| +| Tool Protocol | `marqdo` CLI 自身的外部工具、inspect/verify/IDE 等命令 | 在用户程序运行过程中替代标准 I/O | +| Callable Plugin ABI | 可调用的原生函数扩展、受控实验加速 | 把普通 JSON/CSV/格式化都做成原生 ABI | +| Resource Provider | 新资源类型和设备/网络/文件端点 | 为每个高级算法创建一个 Provider | +| Foreign Kernel | 明确请求跨语言计算的兼容层 | Marqdo 实现常见算法的常规路径 | + +共用基础设施(**先集中到 `marqdo-host` 内部的统一 Registry/Policy 模块,暂不增加 crate**): + +- package/adapter identity、descriptor/version validation; +- 精确能力授予与每次请求审计; +- 统一超时、预算、错误分类和 trace metadata; +- 进程内/worker/C 等传输后端的记录方式; +- 严格模式拒绝未经证明的适配器,不把“worker 子进程”误称沙盒。 + +不应人为强迫函数调用与资源流共享一份万能二进制 ABI。共同的是**注册和权限元模型**,而不是计算语义。 + +## 7. CLI 模块治理与目录重构步骤 + +### P0:冻结最新行为、建立一张事实总表 + +- 建立单一 `docs/architecture/CURRENT.md`,固定审查 SHA、实现/测试/标准库发布/多平台验证四个状态维度。 +- 对照 `docs/STATUS.md`、`docs/architecture/README.md`、RFC 完成审计,消除旧 `master` SHA 与 P5 状态冲突;旧状态移动至 `docs/history/` 或加明显“历史快照”标志。 +- 建立源依赖图、公开 API/stdio/file/TCP 契约、现有扩展路径分类图;记录 baseline 而非先改文件名。 +- CI 加入简单文档一致性检查和标准库 manifest/digest 校验。 + +**验收**:任何对外状态声明都能指向固定 commit、真实测试或源码位置;不允许“测试有例子”冒充“stdlib 已发布”。 + +### P1:把标准输入与 I/O 正确性做成第一条闭环 + +1. Host 增加 `stdio/stdin`(可选 `stderr`)且复用 `resource.*`;保留现有 stdout 安全检查。 +2. 实现 `stdlib/io/stream`:`read_chunk`、`read_exact`、`read_to_end`、`write_all`、`close_checked`。 +3. 实现 `stdlib/io/console`:`read_line`、`prompt`、`print`、`println`,行缓冲、EOF 与 UTF-8 错误一致。 +4. 修复当前 `stdlib/io.mq.md`:复用这些函数,保持 `打印` 外部行为不变。 +5. 保持 `--json` 通道纯净,诊断/trace 与程序输出严格分流。 + +**验收**:通过管道将三行输入送入 `marqdo run`,程序按行响应;单行超过 64 KiB、短读/短写、空输入 EOF、无权限、错误 UTF-8 不崩溃、不丢字节;同一测试必须使用 `.mq.md` 实现 `read_line` 而非 Rust 新内置。 + +### P2:文件与完整数据标准库 + +- 发布 `std:fs/file` 基本 API:read/write/copy;迁移旧 `fs.read/write` 到兼容包装而不是形成两个长期安全边界。 +- 将 conformance 中成熟的 JSON、CSV、UTF-8 decode、整数编码、free-list allocator 迁为公共 stdlib API;移动逻辑后重写测试,确保测试导入**实际标准库模块**。 +- 文件增量读写与目录/元数据行为需要最小的底层 Resource 扩展 ADR;**不能将目标 API 列入已实现直到完成**。 +- `std:data/table` 保留可读 API,增加列名选择、空值/类型校验、按键访问、流式 CSV→Table;对稳定输出保留复制/值语义。 + +**验收**:纯 `.mq.md` 实现并测试文件复制、JSON 读改写回、CSV→Table→GFM、UTF-8 错误报告;追加一个新的数据格式适配器仅改 `.mq.md`。 + +### P3:分离 CLI/Host 中的多重职责 + +- 不改公开 API,先将 `cli/commands.rs` 按 run/call/check/inspect/package 分成内部模块;`stdlib.rs`、`loader.rs` 属于 project 加载路径。 +- 将 `host.rs` 的资源处理和 Provider 注册拆到 `resource/`、`providers/`,Policy 与 Trace 保留统一入口。 +- 将 StdHost 建立为一次性 `RunSession`;配置、权限、stdlib digest、资源句柄生命周期及输出路由由一个装配函数确定。 +- 外部 Tool Protocol 不参与 `marqdo run` 的普通标准库函数解析。 +- 不改变 `marqdo-runtime` 对外执行结果,不因“美观”合并所有小 crate。 + +**验收**:Public API/trace/ABI digest 不变;主 CLI 的 run/call/check 与 strict 安全测试全部维持;对 `src/commands.rs`、`src/host.rs` 做职责级拆分,而不是纯机械拆文件。 + +### P4:系统资源扩展与生产可用性 + +- 在稳定的授予模型上增加 `fs` 元数据与目录事务、DNS/TLS Provider(安全审计单列)、stderr、时间/熵源等;其适用顺序以真实应用需求和安全边界为依据。 +- 把 HTTP 从 Unicode 文本演示升级成二进制增量解析;协议处理仍为 `.mq.md`。 +- 标准库可在不修改冻结 CLI 二进制的情况下增加新模块/补丁版本,仍在同仓库发布和统一打包。 + +**验收**:能用标准库完成交互式 CLI(逐行读写)、大文件增量处理、文件清单与复制、HTTP 请求(在有授权的传输端点上);新增协议解析不引入 Rust 算法。 + +### P5:可选优化,而不是再次堆原语 + +- 测量 `format.显示` 文本拼接、表格值复制、List 结构变换、字节循环读写的成本。 +- 保留纯 `.mq.md` 的参照语义;仅对经基准证明的瓶颈设计内核优化(例如有界 bulk memory copy、结构共享、字符串 builder、编译后端),不得改变可观测语义。 +- 在确实需要长驻 Agent/并行的情况下,再讨论对象与资源生命周期、任务调度和原子语义;不提前推入本轮最小修复。 + +## 8. 能力缺陷与优先级清单 + +| 优先级 | 缺陷 | 性质 | 应由谁完成 | +|---|---|---|---| +| P0 | 无 stdio/stdin 可读端点 | 真正缺失的 OS 能力 | Host `Resource` adapter | +| P0 | `io.写文本` 固定一页、未检查短写/错误 | 标准库正确性 | stdlib/io | +| P0 | `stdlib` 输入、read_line/read_exact/write_all 缺失 | 标准库产品缺口 | stdlib/io | +| P0 | stdlib 测试夹具与公共 API 混淆 | 工程产品化 | stdlib + tests | +| P0 | 架构状态文档相互矛盾 | 文档治理 | docs/CI | +| P1 | `std:fs` 文件工具缺失 | 标准库产品缺口 | stdlib/fs | +| P1 | 文件 read 预读、write 关闭提交、8 MiB 上限 | 当前低层契约能力限制 | Host Resource + 安全审查 | +| P1 | 缺文件 stat/list/rename/remove/mkdir/append/seek | 新 OS 操作需要平台端点 | Resource Operation + stdlib | +| P1 | JSON/CSV/UTF-8 decode 主要停留在 tests | API 发布缺口 | stdlib/data,text | +| P1 | 大型 `host.rs`/`commands.rs` 的职责混杂 | 架构债 | Rust 模块重组 | +| P1 | 多协议注册/安全/追踪概念交叉 | 架构边界问题 | Provider/Plugin Registry | +| P2 | HTTP 解析基于字符,非通用字节流 | 语义与正确性限制 | stdlib/net | +| P2 | TCP 缺 DNS/TLS/监听/UDP | 平台端点能力缺失 | Provider/OS adapter | +| P2 | 标准库缺 path、regex、datetime、env、process、安全随机、压缩等 | 依赖具体 OS/算法的混合缺口 | stdlib 优先;必要时端点 | +| P2 | 当前每区域16MiB、run 64MiB 的限制 | 资源预算设置限制 | Runtime/CLI 配额策略 | +| P3 | 无并发任务、原子同步、可取消的长驻应用调度 | 高级执行语义缺口 | Runtime,另作 ADR | +| P3 | 无 native 优化器、表格高效后端 | 性能能力缺口 | 依据 benchmark 决定 | +| P3 | 多平台严格隔离与 CI 证据有限 | 验证/发布缺口 | CI/Sandbox | + +## 9. 计算闭包验收:防止又变成 Rust Builtin 项目 + +必须区分四种状态: + +- `S0` 原语可表达:语义核心拥有必需的计算/系统边界; +- `S1` 证明可写:测试文档写出过一个算法; +- `S2` 标准库已发布:用户可通过 `std:` 稳定导入; +- `S3` 工程可用:正确性、边界错误、权限、性能预算、文档、版本契约得到验收。 + +下表要求只在冻结 CLI 之上改 `.mq.md`(涉及新增 OS 端点的用例须先完成对应底层适配器): + +| 能力 | 核心闭包验收 | 标准库验收 | 防作弊条件 | +|---|---|---|---| +| 内存分配 | bump / free-list / splitting / coalescing | `std:memory/alloc` | 不加 Rust allocator builtin | +| 字节编码 | UTF-8 / endian / LEB128 | `std:text/utf8` | 不调用 Foreign | +| 数据格式 | JSON/CSV roundtrip、错误位置 | `std:data/json`/`csv` | 不增加 JSON/CSV HostOp | +| 表格 | filter/sort/join/group/GFM | `std:data/table` | 不加表格算法 Builtin | +| 输入输出 | stdin→read_line→stdout;多行 EOF | `std:io` | 只有底层字节读写由 Host 实现 | +| 文件 | read/copy/write/metadata/rename | `std:fs` | 新增 OS 端点须有精确授权,不加 `copy_file` Rust Builtin | +| HTTP | 解析分块/内容长度/UTF-8 body | `std:net/http1` | 不调用 Python HTTP 包 | +| 独立更新 | 仅替换 stdlib bundle 和摘要 | frozen-binary 测试 | CLI 二进制哈希不变 | + +**完成门槛**:不允许只在 `tests/conformance/semantics` 放一个 demo 就关闭 stdlib 的待办事项;测试必须从用户导入路径走完整执行流。 + +## 10. 重构的安全与兼容性验收 + +1. **纯模式**:任何文件或网络端点无授予则拒绝;控制台 stdin/stderr 是否允许由明确的新 ADR 决定,不能无意突破既有严格模式契约。 +2. **项目模式**:每种文件元数据/写入/删除操作单独授权;绝不能复用普通 read 授权执行目录删除;路径规范化防越界、符号链接竞态需要攻击性测试。 +3. **严格模式**:目前只明确支持受限 stdout,stdin、file、network、Provider 必须在隔离证明达到要求后才能逐个开放;拒绝时可诊断,不可静默降级。 +4. **资源句柄**:跨 run、伪造、关闭后使用均拒绝;标准库 `close_checked` 保证正常路径不遗漏 close;异常致命退出必须由 Host 在 run 结束时清理资源。 +5. **短读短写**:所有 `read/write` 返回实际字节数,标准库循环并在零进度时终止;不得认为一次调用即完整成功。 +6. **字节与 Unicode**:二进制 I/O 不隐式调用 `文本()`,协议长度以字节计算;UTF-8 解码错误返回有类型的 `Result`。 +7. **兼容**:`打印` 输出文本契约、`std:io` 名称、Result 结构和 `Resource` 权限语义不改变;若要调整已有接口,以兼容门面和重大版本演进。 +8. **测试**:最基本验收建议为 `cargo fmt --all -- --check`、`cargo clippy --workspace --all-targets -- -D warnings`、`cargo test --workspace --locked`、stdlib digest tests、frozen CLI tests、对应 strict VM 安全矩阵;**这些是目标命令,不代表本审查执行了它们**。 +9. **发布文档**:CI 的成功/失败、commit SHA、环境、精确测试命令由自动报告生成;不能用旧审查快照声称新提交已过关。 + +## 11. 禁止事项:防止再次架构偏移 + +- 不创建独立的 `marqdo-stdlib` 仓库,除非用户以后决定改变已接受的 ADR-0039。 +- 不为 `input()`、`read_line()`、`file_copy()`、JSON、CSV、表格排序分别新增 Rust Builtin。 +- 不把标准库失败伪装成成功;不能忽略 `resource.write(...).ok` 与短写。 +- 不为了“统一”把普通 `List/Map/Table` 直接改成可共享可变裸内存,破坏既有值语义。 +- 不将 Callable Plugin、Resource Provider、Foreign 和 Tool 合并成一个不可审查的万能入口。 +- 不在 `strict` 中默许新 native provider,或将 worker 进程隔离称作操作系统沙盒。 +- 不因为小 crate 数量多而一次性合并十几个 crate;先识别依赖方向和职责。 +- 不再把一次性测试示例宣称成已经交付的标准库产品。 +- 不用单纯文本/Unicode 码点算法假装处理 HTTP 的二进制长度和文件协议。 +- 不在第一阶段建设无需求支持的 GPU/QPU、JIT、裸机器码或复杂任务调度框架。 + +## 12. 按文件的最先修改计划(建议前两个 PR) + +### PR-1:Correct IO / stdin First + +- `crates/marqdo-host/src/resource.rs`:扩展 Stdin、Stderr stream 变体和读写权限;可拆 `resource/stdio.rs`。 +- `crates/marqdo-host/src/host.rs`:使 stdio 资源发现和授权支持新端点;保持严格模式 fail-closed。 +- `crates/cli/src/commands.rs`:确认真实 stdin 传入 Runtime Session 的路径与 CLI JSON 输出互不污染。 +- `stdlib/io.mq.md`:改为兼容门面,提供正确的 write_all + input line API。 +- 新增 `stdlib/io/stream.mq.md`、`stdlib/io/console.mq.md`、`stdlib/text/utf8.mq.md`。 +- `crates/cli/tests/resource_streams.rs`:新增 stdin/EOF/partial 语义与 stdout/stderr 区分测试。 +- 新增 `tests/e2e/interactive-cli`:冻结 CLI、不使用 foreign/plugin。 +- 更新 `spec/evaluation.md` 和 security ADR;更新 stdlib digest。 + +### PR-2:Stdlib Productization + CLI Module Boundaries + +- 新增 `stdlib/fs/file.mq.md`、`stdlib/data/json.mq.md`、`stdlib/data/csv.mq.md` 和公共 API 文档。 +- 将已验证的 conformance 示例改造为标准库测试,保留原有语义夹具作为后向回归。 +- 将 `crates/cli/src/commands.rs` 分为 `commands/`,`loader.rs/stdlib.rs` 归入 project 模块;不改变命令行接口。 +- 将 `crates/marqdo-host/src/host.rs` 的资源操作迁至 `resource/*`,将多 Provider 注册迁至 `providers/*`;对 ABI/Capability 策略固定回归。 +- 新增 CI 的 `stdlib-import-only`、`no-new-builtin-for-algorithm` 和文档 SHA 一致性门禁。 + +文件重排应遵循“先行为保护、后内部抽取、最后删冗余”的顺序。不要让大规模结构性 diff 与系统 I/O 新功能混在同一个 PR 里;若严格实施,应把 PR-2 再拆成 stdlib 产品化与纯 Rust 文件重排两个独立变更。 + +## 13. 与之前 RFC-CM-0001 / RFC-CM-0002 的关系 + +- 继续沿用 `Managed Value + Addressable Memory Region + Capability Resource`,**不再重启机器码或门电路讨论**。 +- 已有 `Memory`、`Resource`、`print` 迁移和表格算法是真实进展,不需要重复开发。 +- RFC-CM-0002 中“标准库独立仓库”的早期预设已经被 ADR-0039 改为**同仓库标准库**;应修改旧 RFC 顶部的状态注释并链接 ADR-0039,而不是保留两种相反的实施指导。 +- 以前以“证明计算闭包”为主;本次以“**闭包已证明的能力能否成为可靠的、可发行的用户 API**”为中心。 + +## 14. 审查来源(冻结到同一 commit) + +以下路径均对应 `7fbe069196e61d06a00d64fe8d65049c327373eb`: + +- [Cargo workspace](https://github.com/cflmy/marqdo-cli/blob/7fbe069196e61d06a00d64fe8d65049c327373eb/Cargo.toml) +- [CLI commands](https://github.com/cflmy/marqdo-cli/blob/7fbe069196e61d06a00d64fe8d65049c327373eb/crates/cli/src/commands.rs) +- [CLI project/print loader](https://github.com/cflmy/marqdo-cli/blob/7fbe069196e61d06a00d64fe8d65049c327373eb/crates/cli/src/loader.rs) +- [Stdlib resolver](https://github.com/cflmy/marqdo-cli/blob/7fbe069196e61d06a00d64fe8d65049c327373eb/crates/cli/src/stdlib.rs) +- [Runtime memory](https://github.com/cflmy/marqdo-cli/blob/7fbe069196e61d06a00d64fe8d65049c327373eb/crates/marqdo-runtime/src/memory.rs) +- [Builtins](https://github.com/cflmy/marqdo-cli/blob/7fbe069196e61d06a00d64fe8d65049c327373eb/crates/marqdo-ir/src/builtins.rs) +- [Host dispatch](https://github.com/cflmy/marqdo-cli/blob/7fbe069196e61d06a00d64fe8d65049c327373eb/crates/marqdo-host/src/host.rs) +- [Resource table](https://github.com/cflmy/marqdo-cli/blob/7fbe069196e61d06a00d64fe8d65049c327373eb/crates/marqdo-host/src/resource.rs) +- [Standard library IO](https://github.com/cflmy/marqdo-cli/blob/7fbe069196e61d06a00d64fe8d65049c327373eb/stdlib/io.mq.md) +- [Standard library HTTP](https://github.com/cflmy/marqdo-cli/blob/7fbe069196e61d06a00d64fe8d65049c327373eb/stdlib/net/http.mq.md) +- [Standard library Table](https://github.com/cflmy/marqdo-cli/blob/7fbe069196e61d06a00d64fe8d65049c327373eb/stdlib/data/table.mq.md) +- [Evaluation semantics](https://github.com/cflmy/marqdo-cli/blob/7fbe069196e61d06a00d64fe8d65049c327373eb/spec/evaluation.md) +- [Stdlib deferred split ADR](https://github.com/cflmy/marqdo-cli/blob/7fbe069196e61d06a00d64fe8d65049c327373eb/docs/adr/0039-stdlib-extraction-intent.md) +- [Architecture index](https://github.com/cflmy/marqdo-cli/blob/7fbe069196e61d06a00d64fe8d65049c327373eb/docs/architecture/README.md) +- [Completion audit](https://github.com/cflmy/marqdo-cli/blob/7fbe069196e61d06a00d64fe8d65049c327373eb/docs/evidence/2026-11-20-rfc-cm-completion-audit.md) + +--- + +**最终设计判断**:核心三域(托管值、可寻址内存区域、受控系统资源)是合理的;现在必须把“资源端点完整性、标准库产品化、CLI/Host 职责收敛”放到第一优先级。不要再为了扩展某个普通函数修改 Rust Builtin;但也不能假装 stdin、目录删除、TLS、系统时钟和熵源在没有 OS 适配器时能够自动涌现。 diff --git a/docs/STATUS.md b/docs/STATUS.md index 5b546f3..6499ade 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -133,7 +133,15 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL `打印/print` to the bundled `io.写值` when the program uses it; stdlib modules are injected lazily; resource operations are trace `effect` events and `HostOp::Print` no longer runs on that path. Strict permits only - stdio/stdout; file/network resources remain refused. + stdio/stdout; stdin/stderr/file/network resources remain refused. +- Stdio streams and console productization (ADR-0040): `stdio/stdin` is + read-only over one shared run cursor; `stdio/stdout`/`stdio/stderr` are + write-only with separate host buffers and output-budget accounting; the CLI + copies non-terminal process stdin at composition and embeds program stderr as a + separate JSON string (or writes it to CLI stderr). `stdlib/io/stream.mq.md` + checks short writes and close errors; `stdlib/io/console.mq.md` implements + line input and print through the resource ABI; `stdlib/text/utf8.mq.md` + provides a validating UTF-8 decoder. - Full-value stdlib formatting (ADR-0029): FunctionRef `module/name` and PluginHandle `plugin/export` are read-only observation members (host id private); `stdlib/format.mq.md` reproduces builtin `打印`/`repr` for all 15 @@ -181,13 +189,15 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL ([evidence](evidence/2026-11-20-stdlib-package.md)). The stdlib intentionally stays in this repository for now (ADR-0039); extraction is a future option when size/release cadence requires it, not a pending gate. -- Resource streams (RFC-CM CM-A06, ADR-0025): opaque host-issued `Resource` - handles with `resource.open/read/write/close`; bytes move only through - `Memory`; stdout write is capability/output-budget checked and project-file +- Resource streams (RFC-CM CM-A06, ADR-0025/0026/0040): opaque host-issued + `Resource` handles with `resource.open/read/write/close`; bytes move only + through `Memory`; stdin is read-only over one shared run cursor; stdout and + stderr are separate capability/output-budget checked buffers; project-file read/write is path/grant checked; file writes buffer until `close` and never flush on drop; a `.mq.md` chunked copy loop reproduces a file without a - Rust `copy_file`; strict permits only the stdio console channel and refuses - file/TCP/provider resources (CM-A11 lane PASS). Semantic fixture 66 and CLI/host tests. + Rust `copy_file`; strict permits only the stdio stdout console channel and + refuses stdin/stderr/file/TCP/provider resources (CM-A11 lane PASS). Semantic + fixture 66 and CLI/host tests. - Minimal structure observation/construction (RFC-CM CM-A04, ADR-0023): value-semantics `list.append/concat`, `map.keys/has/get/set/remove`, Unicode scalar `text.codepoints`/`text.from_codepoint`, and non-negative @@ -226,7 +236,7 @@ behavior. This table records only what is in this tree now. | P0 freeze (ADRs, spec deltas, baseline) | **P1 + partial P2 slice done** | ADR-0022 freezes CM-A01/CM-A02/CM-A03; ADR-0023 freezes CM-A04 (minimal List/Map/Text/Table observation and construction); ADR-0024 freezes CM-A05 (recoverable `Result` map convention); `spec/expression.ebnf`, `spec/evaluation.md`, `spec/value-wire.md`, `spec/analysis-v2.md`, `spec/security.md` updated; existing AST/value goldens unchanged | | P1 Memory + place assignment | **Implemented locally** | `Value::Memory`, run-scoped `MemoryStore` (64 KiB pages, 64 MiB total / 16 MiB region), `memory.new/size/grow`, `m[i]` read, `m[i] = byte`; semantic fixtures 45–47, runtime negatives 13–18, `examples/memory-buffer.mq.md`; evidence [`evidence/2026-11-20-memory-p1.md`](evidence/2026-11-20-memory-p1.md) | | P2 Marqdo-written allocator/data structures + minimal structure observation | **PASS (local pure lane); stdlib remains in-repo by design** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), a quoted/escaped CSV row parser (68), table group (69), an Int matrix multiply (70), and a sparse-matrix multiply (73); the first-fit allocator also detects double free (`alloc.double_free`). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). Local gate PASS: `crates/cli/tests/p2_closure.rs` runs 21 C01–C06/C11/C12 fixtures plus the stdlib table self-test in `--mode pure` (no foreign/plugin/import grants), scans sources for foreign/plugin/import shortcuts and compares value/stdout to goldens; evidence [`p2-closure-gate`](evidence/2026-11-20-p2-closure-gate.md). The bundled `stdlib/` tree, `std:` imports, digest verification, Analysis identity and frozen-CLI closure test are implemented (ADR-0028). The stdlib intentionally remains in-repo (ADR-0039); a separate repository is a future extraction option, not a gate. Linux Docker CI is green; a strict security runner and Windows/macOS lanes are still pending. | -| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict permits only stdio/stdout and refuses file/network resources. Fixture 66 + CLI/host tests. CM-A07 **Compat-C** is implemented: when a program calls `打印/print`, the loader injects the bundled `stdlib/io.mq.md` and the runtime routes to `io.写值` (all 15 value types, UTF-8 via `Memory`, `resource.write(stdout)`); `HostOp::Print` and the legacy `Value::display()` print path are removed, and a missing bundle fails with `kernel.unavailable`. Strict permits only the existing stdio console channel. Providers are registered through ADR-0033 (in-process) and ADR-0035 (worker helper). Still missing: directories, C ABI provider binding, provider packages, strict provider support. | +| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026/0040 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdin read, separate stdout/stderr write, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict permits only stdio/stdout and refuses stdin/stderr/file/network resources. `stdlib/io/stream.mq.md`, `stdlib/io/console.mq.md` and `stdlib/text/utf8.mq.md` are product-level `.mq.md` APIs; fixture 66 + CLI/host tests. CM-A07 **Compat-C** is implemented: when a program calls `打印/print`, the loader injects the bundled `stdlib/io.mq.md` and the runtime routes to `io.写值` (all 15 value types, UTF-8 via `Memory`, `resource.write(stdout)`); `HostOp::Print` and the legacy `Value::display()` print path are removed, and a missing bundle fails with `kernel.unavailable`. Strict permits only the existing stdio console channel. Providers are registered through ADR-0033 (in-process) and ADR-0035 (worker helper). Still missing: directories, C ABI provider binding, provider packages, strict provider support. | | P4 Table logical/physical migration | **PASS (local gate); T1/T2/T3 future** | ADR-0032/CM-A08: `Value::Table` semantics and GFM AST laziness unchanged; `stdlib/data/table.mq.md` provides select/filter/sort/append/set/join/group/`转GFM` as value-returning `.mq.md` functions with input immutability; zero Rust table algorithms were added. Cross-storage equivalence is pinned by fixture 72 (row Table vs `.mq.md` columnar store, identical headers/rows/cells). The optional T1/T2/T3 physical-storage/column-store migration was not required by the gate and is not implemented. | | P5 third-party Provider / TCP / HTTP | **PASS (local gate); external network/CI and plugin-Worker resource binding open** | C10: `stdlib/net/http.mq.md` + fixture 71 parse HTTP/1.1 status/headers/chunked bodies and build requests; `http_over_tcp.rs` runs a loopback HTTP exchange wholly in `.mq.md` over `resource.open("tcp",...)`. ADR-0034 IP-literal blocking TCP endpoint (exact IP grant, project-only, 5s timeouts) and ADR-0033 in-process `ResourceProvider` v1 (registration, protocol/kind checks, name grant, mock read/write provider) ADR-0035 host-managed worker provider transport (framed JSON helper, ping/open/read/write/close, trusted-only) and ADR-0036 trusted C ABI provider (shared-library symbols, version/load checks) ADR-0037 local provider-package manifest installation and ADR-0038 C ABI provider worker binding (separate trusted helper process) are tested. External DNS/TLS/UDP, host HTTP, provider packages/signatures and the plugin-Worker resource binding remain out of scope. | | P6 performance / native backends | **BASELINE ONLY** | reproducible profile harness `tools/perf/bench.py` and [`p6-profile`](evidence/2026-11-20-p6-profile.md) for 7 workloads; no bytecode/JIT/columnar numeric backend or equivalence/optimization gate yet | diff --git a/docs/adr/0030-print-compat-b-routing.md b/docs/adr/0030-print-compat-b-routing.md index 195c4ad..e0a47bb 100644 --- a/docs/adr/0030-print-compat-b-routing.md +++ b/docs/adr/0030-print-compat-b-routing.md @@ -61,9 +61,9 @@ legacy `HostOp::Print` effect does not appear in Compat-B runs. Strict L1 already permits the console channel. Compat-B needs stdout writes in strict so the same print implementation works there. Therefore strict mode -permits **only** `stdio/stdout` write with the console grant; project-file, -network, process and device resource opens remain refused, and no new provider -is enabled. This is the existing console capability reached through the +permits **only** `stdio/stdout` write with the console grant; stdin/stderr, +project-file, network, process and device resource opens remain refused, and no new +provider is enabled (ADR-0040 keeps the later stdin/stderr slice separate). This is the existing console capability reached through the resource ABI, not a new OS capability. File resources stay fail-closed. ### 5. Compatibility and fallback diff --git a/docs/adr/0040-stdio-stdin-stderr-and-stdlib-streams.md b/docs/adr/0040-stdio-stdin-stderr-and-stdlib-streams.md new file mode 100644 index 0000000..cadef53 --- /dev/null +++ b/docs/adr/0040-stdio-stdin-stderr-and-stdlib-streams.md @@ -0,0 +1,60 @@ +# ADR-0040: Stdio stdin/stderr endpoints and stdlib stream APIs + +Status: **Accepted** +Date: 2026-11-21 +Relates to: ADR-0025, ADR-0026, ADR-0027, ADR-0030, ADR-0031 +Specs: `spec/evaluation.md`, `spec/security.md` +Evidence: `docs/evidence/2026-11-21-stdio-streams.md` + +## Context + +ADR-0025 opened one `stdio/stdout` write endpoint. That left `stdin` and +`stderr` as hidden OS channels, and the bundled `io.写文本` ignored the +`resource.write` `{ok,count}` result. The audit RFC calls for a real resource +boundary, not a Rust `Input`/`Print` builtin. + +## Decision + +* Add `stdio/stdin` (rights `["read"]`) and `stdio/stderr` (rights + `["write"]`). `stdio/stdout` remains `["write"]`. Rights/kinds are exact; + mixing them is `resource.unsupported`. +* All stdin handles in one run share one cursor. `resource.read` consumes bytes and + reports `{count,eof}`; EOF is a successful zero-byte read, not an error. + Closing a handle does not reset or close the run cursor. There is no blocking + OS read inside the host table. +* stdout and stderr are distinct host buffers. Both charge the existing output + budget, but `output()`/`take_output()` never return stderr and + `error()`/`take_error()` never return stdout. A budget failure appends + nothing to either buffer. +* The CLI reads process stdin once at composition time only when stdin is not a + terminal. This keeps an interactive `marqdo run` from waiting for EOF. JSON run + output adds a separate `stderr` string; text mode writes captured program + stderr to process stderr. Diagnostics and traces remain CLI stderr and never mix + with the JSON stdout document. +* Strict mode remains fail-closed: only `stdio/stdout` with `["write"]` and the + console grant is allowed. stdin, stderr, file, TCP and provider streams are + `permission.denied`. +* The standard library keeps the product APIs in `.mq.md`: + `stdlib/io/stream.mq.md` owns `写全部`/`写文本` with short-write and close + checking; `stdlib/io/console.mq.md` owns `读取行`/`打印` over the resource + ABI; `stdlib/text/utf8.mq.md` owns a validating UTF-8 decoder. + `stdlib/io.mq.md` remains the compatibility facade and print target. + +## Non-goals + +No process, device, directory, DNS, TLS, UDP, seek, mmap, async or zero-copy +endpoint. The stdin source is the finite process input supplied by the +composition root, not a general interactive console driver. No new language +keyword, `Input` builtin, `HostOp::Print` or second Markdown parser. + +## Compatibility + +The resource ABI is additive. Existing stdout-only programs, `打印`, file +read/write and provider/tcp streams keep their prior behavior. A program that +used a closed handle still gets the recoverable `resource.closed` result. + +## Consequences + +CLI programs can now read piped stdin and write program stderr without leaking +either into JSON diagnostics. The remaining OS-shaped gaps are explicit and can be +added only through a separate resource operation decision. diff --git a/docs/architecture/README.md b/docs/architecture/README.md index 7ca738d..afa1d3f 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -6,7 +6,7 @@ normative RFC text stays at its reviewable path under `docs/`. | Document | Status | Implemented part | |---|---|---| | [RFC-CM-0001: compute machine, addressable memory and system boundary](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) | Draft / design input | R1 (P1): `Memory` value, `memory.new/size/grow`, byte read/write, Memory-index assignment, budgets, wire/host rejection. Implemented per [ADR-0022](../adr/0022-addressable-memory-and-place-assignment.md); see [STATUS §5](../STATUS.md). | -| [RFC-CM-0002: minimal core, stdlib self-hosting and system capability migration](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | Draft / design input | P0/P1, a **partial P2**, and a **partial P3** first slice (ADR-0025/0026: stdout write, granted project-file read and buffered project-file write via `resource.*`; strict permits only the stdio console channel; ADR-0027/0030/0031 Compat-C print via bundled stdlib, no legacy host formatter). P2: fixtures 48–50 (`.mq.md` bump allocator, u32 codec, linked list/traversal, failed-allocation atomicity), CM-A04 structure primitives for List/Map/Text/Table (ADR-0023, fixtures 51–54), and a `.mq.md` first-fit free-list allocator with block splitting, address-adjacent coalescing and `Result` maps (ADR-0024, fixture 55), plus a `Memory`-backed binary tree with recursive insert/in-order traversal (fixture 56), a `.mq.md` CSV row splitter (fixture 57), an open-addressing hash table with linear probing (fixture 58), a `.mq.md` UTF-8 encoder and validating decoder (`utf8.truncated`/`invalid_continuation`/`invalid_scalar`) (fixtures 59–60), a `Memory`-backed graph adjacency list (fixture 61), a recursive JSON parser/formatter for arrays, objects, strings with escapes, ints/floats/booleans/null and error Results (fixture 62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (fixture 63), big-endian 16/32/64-bit plus LEB128 integer codecs (fixture 64), a `.mq.md` table sort rebuilding the table without mutating the input (fixture 65), a table join built from `table.row/columns/cell` (fixture 67), and a quoted/escaped CSV row parser (fixture 68), table group (fixture 69) and an Int matrix multiply (fixture 70). The stdlib intentionally remains in this repository for now (ADR-0039); no separate external repo, network/generic providers, P4 physical storage migration, P5 providers or P6 performance work. | +| [RFC-CM-0002: minimal core, stdlib self-hosting and system capability migration](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | Draft / design input | P0/P1, a **partial P2**, and a **partial P3** first slice (ADR-0025/0026/0040: stdin read, separate stdout/stderr write, granted project-file read and buffered project-file write via `resource.*`; strict permits only the stdio stdout console channel; ADR-0027/0030/0031 Compat-C print via bundled stdlib, no legacy host formatter). P2: fixtures 48–50 (`.mq.md` bump allocator, u32 codec, linked list/traversal, failed-allocation atomicity), CM-A04 structure primitives for List/Map/Text/Table (ADR-0023, fixtures 51–54), and a `.mq.md` first-fit free-list allocator with block splitting, address-adjacent coalescing and `Result` maps (ADR-0024, fixture 55), plus a `Memory`-backed binary tree with recursive insert/in-order traversal (fixture 56), a `.mq.md` CSV row splitter (fixture 57), an open-addressing hash table with linear probing (fixture 58), a `.mq.md` UTF-8 encoder and validating decoder (`utf8.truncated`/`invalid_continuation`/`invalid_scalar`) (fixtures 59–60), a `Memory`-backed graph adjacency list (fixture 61), a recursive JSON parser/formatter for arrays, objects, strings with escapes, ints/floats/booleans/null and error Results (fixture 62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (fixture 63), big-endian 16/32/64-bit plus LEB128 integer codecs (fixture 64), a `.mq.md` table sort rebuilding the table without mutating the input (fixture 65), a table join built from `table.row/columns/cell` (fixture 67), and a quoted/escaped CSV row parser (fixture 68), table group (fixture 69) and an Int matrix multiply (fixture 70). The stdlib intentionally remains in this repository for now (ADR-0039); no separate external repo, network/generic providers, P4 physical storage migration, P5 providers or P6 performance work. | The RFCs are architecture constraints, not evidence that a feature exists. The implementation status lives in `docs/STATUS.md`; frozen behavior is in @@ -36,7 +36,8 @@ implementation status lives in `docs/STATUS.md`; frozen behavior is in | CM-A09 provider package manifest v1 | [0037](../adr/0037-provider-package-manifest.md) | Accepted (host-side install) | | CM-A09 C ABI provider worker binding v1 | [0038](../adr/0038-c-abi-provider-worker-binding.md) | Accepted (trusted helper) | | Stdlib extraction deferral | [0039](../adr/0039-stdlib-extraction-intent.md) | Accepted (in-repo for now) | -| Table physical storage (T1/T2/T3), C/Worker provider transport, strict file/network resources | — | Not decided; do not implement by implication | +| Stdio stdin/stderr plus stdlib stream/console APIs | [0040](../adr/0040-stdio-stdin-stderr-and-stdlib-streams.md) | Accepted (stdin read, separate stdout/stderr, product stdlib) | +| Table physical storage (T1/T2/T3), strict stdin/stderr/file/network resources | — | Not decided; do not implement by implication | ## What a future phase must not do diff --git a/docs/evidence/2026-11-21-stdio-streams.md b/docs/evidence/2026-11-21-stdio-streams.md new file mode 100644 index 0000000..99c5d69 --- /dev/null +++ b/docs/evidence/2026-11-21-stdio-streams.md @@ -0,0 +1,30 @@ +# Evidence: stdio stdin/stderr and stdlib streams + +Date: 2026-11-21 (Asia/Shanghai) +Repository: `marqdo-cli` +Decision record: [ADR-0040](../adr/0040-stdio-stdin-stderr-and-stdlib-streams.md) + +## Local gate + +```text +cargo fmt --all -- --check +cargo test --workspace --no-fail-fast +``` + +Both ran locally in the writable `.cargo-home`. The workspace suite passed. + +## Covered slices + +* host: pure-mode stdin chunking and EOF, separate stdout/stderr buffers, + strict refusal of stdin/stderr, closed-handle errors, output budget; +* CLI: piped stdin through `std:io/console.读取行`, separate JSON `stdout` and + `stderr`, EOF without a panic; +* stdlib: `std::io/stream.写全部` loops on `{ok,count}`, rejects zero progress, + `写文本` grows `Memory` from the UTF-8 byte length and checks close; + `std::text/utf8.解码` rejects truncated, invalid-continuation, overlong, + surrogate and out-of-range scalar input. + +## Not covered + +Directories, process/device resources, DNS/TLS/UDP, seek/mmap, async I/O, +interactive terminal line editing, Windows/macOS native lanes, and remote CI. diff --git a/spec/evaluation.md b/spec/evaluation.md index 9a3117f..7762c5b 100644 --- a/spec/evaluation.md +++ b/spec/evaluation.md @@ -168,18 +168,19 @@ resource.close(handle) -> {ok:true, value:None} | {ok:false, code, message} ``` -* Providers: `stdio/stdout` with `["write"]` (pure/project, and strict's - existing console channel, console grant); - `file` with `["read"]` (project, `fs_read` grant, bounded by the 8 MiB file - limit); `file` with `["write"]` (project, `fs_write` grant, 8 MiB buffered - write flushed on `close`, ADR-0026). Mixed rights and other kinds are - `resource.unsupported`. +* Providers: `stdio/stdin` with `["read"]` and `stdio/stdout`/`stdio/stderr` + with `["write"]` (pure/project; one shared stdin cursor and separate stdout + and stderr buffers; console grant). Strict permits only the existing + `stdio/stdout` channel. `file` with `["read"]` (project, `fs_read` grant, + bounded by the 8 MiB file limit); `file` with `["write"]` (project, + `fs_write` grant, 8 MiB buffered write flushed on `close`, ADR-0026). + Mixed rights and other kinds are `resource.unsupported`. * `resource.read` checks the destination `Memory` range before calling the provider, caps the request to the remaining buffer, writes the returned bytes and reports the actual count plus `eof`. EOF is success, not an error. * `resource.write` copies bytes out of `Memory` for the call only; a short write - is a count. stdout output is charged to the output budget and delivered to - the program stdout channel. A project-file write stream appends to a bounded + is a count. stdout/stderr output is charged to the output budget and delivered + to separate program channels. A project-file write stream appends to a bounded host buffer and does not touch the filesystem until `close`; a failed close leaves the handle open with its buffer, and an unclosed writer never creates the file. @@ -247,8 +248,9 @@ refuses to run (see `security.md`). For a fixed source, mode, capabilities and budget, evaluation is deterministic. Stdout/stderr from host operations are separated: program -output goes to stdout, diagnostics/traces go to stderr, and `--json` output -never mixes with program output. +stdout and program stderr are captured separately, diagnostics/traces go to the +CLI stderr, and `--json` output never mixes with any program output. JSON +reports `stdout` and `stderr` as separate strings. ## 8. Builtins v1 diff --git a/spec/security.md b/spec/security.md index 52f2af4..35a09ae 100644 --- a/spec/security.md +++ b/spec/security.md @@ -27,8 +27,8 @@ Normative implementation: | Mode | Intended guarantee | Actual status in this build | |---|---|---| | `pure` | No external I/O; memory computation and budgets only. | **Enforced.** `Capabilities::pure()` = `console`, `math` only; every FS/foreign/plugin grant flag makes the CLI exit 2 with "pure mode prohibits external I/O grants". | -| `project` | Operator-approved project resources under `CapabilityHost` + bounded resources. | **Enforced for the implemented operations.** Defaults equal `pure`; `--allow-read`, `--allow-write`, `--allow-foreign`, `--allow-plugin` add grants. `resource.open` supports stdout write, granted project-file read/write (ADR-0025/0026), granted in-process provider kinds (ADR-0033) and granted IP-literal TCP streams (ADR-0034); process/device remain unsupported. Sandboxing is cooperative (in-process checks), not OS isolation; in-process providers are trusted, not sandboxed. | -| `strict` | Untrusted documents under real OS isolation. | **Linux: enforced when the runtime probe passes (ADR-0021).** `resource.open` is **refused** in strict except for the existing `stdio/stdout` console channel (console grant); file/TCP/provider/process resources remain refused until a provider passes the isolation lane (CM-A11, ADR-0030/0034). `PlatformCapabilities::detect()` runs a real Bubblewrap smoke test; success reports `Available { mechanism }`, failure reports `Unavailable { reason }`. Strict L1 supports the interpreter, console, native Math and the read-only bundled stdlib (published at `/run/marqdo-stdlib` for the Compat-C print chain); `--allow-read/--allow-write/--allow-foreign/--allow-plugin/--plugin/--trust-native-plugin/--kernel/--trace-file` are rejected with exit 2. macOS/Windows and Linux hosts without the mechanisms remain `Unavailable`; there is **no fallback** to an unrestricted process. | +| `project` | Operator-approved project resources under `CapabilityHost` + bounded resources. | **Enforced for the implemented operations.** Defaults equal `pure`; `--allow-read`, `--allow-write`, `--allow-foreign`, `--allow-plugin` add grants. `resource.open` supports stdin read, separate stdout/stderr write, granted project-file read/write (ADR-0025/0026/0040), granted in-process provider kinds (ADR-0033) and granted IP-literal TCP streams (ADR-0034); process/device remain unsupported. Sandboxing is cooperative (in-process checks), not OS isolation; in-process providers are trusted, not sandboxed. | +| `strict` | Untrusted documents under real OS isolation. | **Linux: enforced when the runtime probe passes (ADR-0021).** `resource.open` is **refused** in strict except for the existing `stdio/stdout` console channel (console grant); stdin/stderr, file/TCP/provider/process resources remain refused until a provider passes the isolation lane (CM-A11, ADR-0030/0034/0040). `PlatformCapabilities::detect()` runs a real Bubblewrap smoke test; success reports `Available { mechanism }`, failure reports `Unavailable { reason }`. Strict L1 supports the interpreter, console, native Math and the read-only bundled stdlib (published at `/run/marqdo-stdlib` for the Compat-C print chain); `--allow-read/--allow-write/--allow-foreign/--allow-plugin/--plugin/--trust-native-plugin/--kernel/--trace-file` are rejected with exit 2. macOS/Windows and Linux hosts without the mechanisms remain `Unavailable`; there is **no fallback** to an unrestricted process. | `marqdo inspect --platform` prints the detection report, for example: @@ -55,7 +55,7 @@ set for one run: | Field | Default (`pure`/`project`) | Used by | |---|---|---| -| `console: bool` | `true` | `resource.open("stdio","stdout",["write"])` and the stdlib print chain; a missing grant is `permission.denied` and output is also bounded by the output budget | +| `console: bool` | `true` | `resource.open("stdio","stdin",["read"])`, `resource.open("stdio","stdout"/"stderr",["write"])` and the stdlib console chain; a missing grant is `permission.denied` and stdout/stderr are also bounded by the output budget | | `math: bool` | `true` | `math.*` host ops | | `fs_read: Vec` | empty | `fs.read` and project `resource.open("file", path, ["read"])` | | `fs_write: Vec` | empty | `fs.write` and project `resource.open("file", path, ["write"])` | @@ -143,10 +143,10 @@ no host bypass for tests: tests use the same `StdHost`/`Capabilities` types. |---|---|---| | evaluation fuel | `DEFAULT_FUEL = 5_000_000` | 1 unit per statement, 1 per expression node, 1 per branch condition, 1 per loop iteration; exhaustion → `resource.exhausted` | | call stack depth | `DEFAULT_STACK_LIMIT = 256` | checked before every function call → `resource.exhausted` | -| program output bytes | `DEFAULT_OUTPUT_BYTES = 1 MiB` | `print` output and `fs.read` contents are charged; overflow → `resource.exhausted` | +| program output bytes | `DEFAULT_OUTPUT_BYTES = 1 MiB` | `print`, `stdio/stdout` and `stdio/stderr` output plus `fs.read` contents are charged; overflow → `resource.exhausted` | | wall clock | none by default | `--timeout ` sets a deadline; checked between statements/iterations and before loop conditions → `resource.exhausted`. It does **not** preempt a running host call. | | addressable memory | 64 MiB committed total, 16 MiB per region, 64 KiB pages; `--max-memory` / `--max-region-memory` override | Runtime-owned `MemoryStore` (ADR-0022); create/grow checks both budgets before allocation and failed growth is atomic → `resource.exhausted`. Memory is pure computation and needs no capability grant in any mode. | -| resource streams | stdout writes charged to the existing output budget; file reads and writer buffers bounded by the 8 MiB per-file limit | Host-owned resource table (ADR-0025/0026); output budget is checked before a stdout write is buffered; a failed check produces no bytes; writer buffers flush only on `close`. No separate I/O byte budget yet. | +| resource streams | stdin reads over a shared run cursor; stdout/stderr writes charged to the existing output budget; file reads and writer buffers bounded by the 8 MiB per-file limit | Host-owned resource table (ADR-0025/0026/0040); output budget is checked before a stdout/stderr write is buffered; a failed check produces no bytes; writer buffers flush only on `close`. No separate I/O byte budget yet. | CLI overrides (`run`/`call`): `--fuel`, `--stack`, `--max-output`, `--timeout`. Additional fixed limits: @@ -181,8 +181,9 @@ overflow). In-process math and native plugins are not OS-limited. * argument values and file contents are **not** recorded, so ordinary secrets do not leak into traces by design. Paths are also not recorded today. * In text mode trace entries are printed to stderr (`trace: ...`); in JSON - mode they are a `trace` array next to the program stdout string. Program - output is always separate from diagnostics/trace (stdout vs stderr). + mode they are a `trace` array next to separate program `stdout`/`stderr` + strings. Program output is always separate from diagnostics/trace, and + `stdio/stderr` never shares the `stdout` buffer. ## 5. Foreign kernels: process isolation, not a sandbox diff --git a/tests/README.md b/tests/README.md index 14647bf..4ec3042 100644 --- a/tests/README.md +++ b/tests/README.md @@ -261,12 +261,14 @@ RFC-CM CM-A06 resource streams (ADR-0025) are pinned by: `stdio/stdout` resource, closed, then re-used to prove `{ok:false,code:"resource.closed"}`; the expected `stdout` is the decoded text; -* `crates/marqdo-host/tests/resource_streams.rs` (9): pure-mode stdout write, - output-budget atomicity, strict refusal, project-file chunking/EOF, missing - grant, traversal rejection and unsupported rights/kinds; +* `crates/marqdo-host/tests/resource_streams.rs` (17): pure-mode stdin read, + stdout/stderr separation, output-budget atomicity, strict refusal of + stdin/stderr/file, project-file chunking/EOF, missing grant, traversal + rejection and unsupported rights/kinds; * `crates/cli/tests/resource_streams.rs`: real `marqdo run` with - `--allow-read data.txt`, three chunked reads into `Memory` plus EOF, the same - document failing with `permission.denied` without the grant, and a + `--allow-read data.txt`, three chunked reads into `Memory` plus EOF, piped + stdin line reads through `std:io/console`, separate stdout/stderr capture, + the same document failing with `permission.denied` without the grant, and a write-then-read program with `--allow-write/--allow-read out.bin` proving the file is created only at `resource.close`, and a chunked `.mq.md` file copy (`resource.read` → `resource.write`) that reproduces `data.txt` in `out.bin`;