Compare commits

..
12 Commits
Author SHA1 Message Date
cflmyandCursor 237962c3be release: v0.3.9 — GAP-06–10 parser, invoke, exec, SSE
Ship business-integration fixes: star/JSON body recovery, invoke Null,
sys.exec capture, and proxy anti-buffer headers.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-09 14:12:43 +08:00
cflmyandCursor 1637acce6e fix: GAP-06–10 parser, invoke Null, exec capture, SSE headers
Recover ## bodies after bad *…* terminators; bracket-aware call args;
invoke Null is 500 not fake ok; sys.exec capture={code,stdout,stderr};
proxy stream sets anti-buffer headers.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-09 13:58:02 +08:00
cflmyandCursor 68c4f8c822 release: v0.3.8 — Mid2 stdlib M7–M10
Ship calendar/URL/TOML/HTML/fs/stats plus GLOBAL_HOST restore;
bump Cargo, README, CHANGELOG, and release workflow highlights.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-09 10:08:59 +08:00
cflmyandCursor 784d5ec346 feat(stdlib): Mid2 M10 stats and log fields
Add thin mean/median/stdev and optional structured fields on log
lines; close Mid2 M7–M10 (zip skipped).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-09 09:42:13 +08:00
cflmyandCursor 955834e453 feat(stdlib): Mid2 M9 fs make_dirs, walk, and stat
Thicken lib/fs with recursive mkdir/rmdir-tree, metadata maps, and
depth-first walk; add gold tests and public demos.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-09 09:27:06 +08:00
cflmyandCursor 435f36b6e9 feat(stdlib): Mid2 M8 toml, html escape, and base32
Add read-only TOML subset parse, HTML escape/unescape, and RFC 4648
base32 on encoding; gold tests and public demos included.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-09 09:22:00 +08:00
cflmyandCursor 08c48f3acb feat(stdlib): Mid2 M7 datetime and url
Add calendar moments as {unix,zone,iso} maps and URL/query parse
helpers in lib/ with thin host_* forwards; gold and public demos.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-09 09:14:18 +08:00
cflmyandCursor ff1870cedc docs: Mid2 gap research and M7–M10 design/roadmap
Inventory stdlib gaps after Mid M1–M6 and lock Mid2 waves
(datetime/url, toml/html, fs++, optional stats).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-09 09:04:29 +08:00
cflmyandCursor 5e51ee1bb2 fix(host): restore GLOBAL_HOST after nested plugin calls
Nested call_registered was clearing the process-wide host pointer to null,
so web_listen lost context for later HTTP invoke rounds after an inner ABI call.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-08 10:40:27 +08:00
cflmyandCursor 55f1a37b3d chore: sync Cargo.lock package versions to 0.3.7
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-07 22:21:54 +08:00
cflmyandCursor 58dff0a1e9 release: v0.3.7 — Hosting H1–H4 (proxy, invoke, MCP stdio)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-07 22:18:43 +08:00
cflmyandCursor fc543f1ca5 feat(web,agent): hosting H1–H4 — proxy, invoke, MCP stdio, cross-module db
Close GAP-01–05 with ext/web streaming reverse proxy and HTTP→## invoke,
transitive object method lookup, corpus_search plugin ensure, and agent MCP
server stdio so downstream apps can drop Python side relays.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-07 22:14:51 +08:00
101 changed files with 6037 additions and 188 deletions

No files matched your search

+4 -4
View File
@@ -113,10 +113,10 @@ jobs:
## Marqdo ${{ github.ref_name }}
### 亮点 / Highlights
- **中层标准库 Mid M1–M6**:`re`/`encoding`/`path`/`hash`/`secrets`/`csv`/`text++`/`cli`/`log`/`uuid`(中英对称,零插件)
- **Mid stdlib M1–M6**: regex, encoding, path, hash, secrets, csv, thickened text/lists, cli, log, uuid
- **`ext/linalg` 公式文档面 F1–F3**;agent/llm 写回与 SSE 稳健性修复
- Details: [CHANGELOG](https://github.com/cflmy/marqdo/blob/main/CHANGELOG.md) · [stdlib-mid.md](https://github.com/cflmy/marqdo/blob/main/doc/design/stdlib-mid.md)
- **GAP-06–10**:解析恢复 · invoke Null · exec capture · SSE 反缓冲头
- **GAP-06–10**: parser recovery, invoke Null, exec capture, SSE anti-buffer headers
- **修复**:嵌套插件调用后恢复 `GLOBAL_HOST`(listen/invoke)
- Details: [CHANGELOG](https://github.com/cflmy/marqdo/blob/main/CHANGELOG.md) · [stdlib-mid2.md](https://github.com/cflmy/marqdo/blob/main/doc/roadmap/stdlib-mid2.md)
### Downloads
+68
View File
@@ -8,6 +8,74 @@
### Changed
## v0.3.9 — 2026-09-09
### Highlights
**业务联调缺口 GAP-06–10**:解析器不再吞掉嵌套反引号/`text=[…]` 后的函数体;`invoke` 的 `Null` 不再伪装成功;`sys.exec capture=` 可取 stdout;流式 proxy 反缓冲头。
```bash
git checkout v0.3.9
```
### Added
- **`sys.exec` / `系统.执行`**:可选 `capture=` / `捕获=`;为真时返回 `{code, stdout, stderr}`(GAP-09)。
### Fixed
- **GAP-06/07**:`*…*` 分类对反引号/字符串内的 `*` 不再误判;错误收尾的赋值行不再吞掉后续 `##` 体(`>` / 合法 `*…*` 可脱出注释段);命名参数边界尊重 `[…]`/`{…}`。
- **GAP-08**:`app.invoke` 对 `Null`/`None` 返回不再伪装成 `{"ok":true}`,改为 500 + `{"ok":false,"error":"null result"}`。
- **GAP-10**:`app.proxy` `stream=true` 响应补齐 `Cache-Control: no-cache, no-transform` 与 `X-Accel-Buffering: no`。
### Changed
## v0.3.8 — 2026-09-09
### Highlights
**中层标准库 Mid2(M7–M10)收口**:日历/时区、URL、TOML、HTML 转义、base32、实用 fs、薄统计与日志字段——日常脚本与工具配置不再依赖外联或插件。另修复嵌套插件调用后 `GLOBAL_HOST` 被清空导致 `invoke` 失败。
```bash
git checkout v0.3.8
# 或下载 Release 的 exe / zip
marqdo run public/features/14-datetime-url.mq.md
marqdo run public/features/17-stats-log.mq.md
```
### Added
- **中层标准库 Mid2 M10**:`lib/stats` / `统计`(mean/median/stdev)与 `log`/`日志` 可选 `fields=`;金样与 public `17-stats-log`。**Mid2 M7–M10 收口**(zip 跳过)。设计 [stdlib-stats-log.md](doc/design/stdlib-stats-log.md)。
- **中层标准库 Mid2 M9**:`fs.make_dirs` / `remove_tree` / `stat` / `walk`(中英);金样与 public `16-fs-mid2`。设计 [stdlib-fs-mid2.md](doc/design/stdlib-fs-mid2.md)。
- **中层标准库 Mid2 M8**:`lib/toml` / `配置表`(只读子集)、`lib/html` / `超文本`(escape/unescape)、`encoding` base32;金样与 public `15-toml-html-encoding`。设计 [stdlib-toml.md](doc/design/stdlib-toml.md) · [stdlib-html.md](doc/design/stdlib-html.md)。
- **中层标准库 Mid2 M7**:`lib/datetime` / `日期时间`(`{unix,zone,iso}` · parse/format/add/in_zone)与 `lib/url` / `地址`(parse · query_parse/stringify);金样与 public `14-datetime-url`。设计 [stdlib-datetime.md](doc/design/stdlib-datetime.md) · [stdlib-url.md](doc/design/stdlib-url.md)。
- **文档**:中层标准库 Mid 之后缺口盘点与 Mid2(M7–M10)设计/路线——[stdlib-gaps-after-mid.md](doc/research/stdlib-gaps-after-mid.md) · [stdlib-mid2.md](doc/design/stdlib-mid2.md) · [stdlib-mid2.md](doc/roadmap/stdlib-mid2.md)。
### Fixed
- **`host_query` / GLOBAL_HOST**:嵌套 `call_registered` 结束后恢复外层宿主指针,不再清空;`web_listen` 多轮 `invoke`(中间再调插件)不再报 `no active host context`。
### Changed
## v0.3.7 — 2026-09-07
### Highlights
**宿主集成 H1–H4**:浏览器同域 SSE/HTTP 流式中继(`app.proxy`)、HTTP→用户 `##`(`app.invoke`)、跨模块 db 方法分发、`corpus_search` 可发现、MCP Server stdio——下游可去掉 Python 旁路。
```bash
git checkout v0.3.7
# 或下载 Release 的 exe / zip
marqdo ext add web && marqdo ext add agent
# *app*.proxy / *app*.invoke · agent.mcp_server
```
### Added
- **宿主集成 H1–H4**:`ext/web` `app.proxy`(同域 SSE/HTTP 流式中继)与 `app.invoke`(HTTP→`lib.member`);跨模块 `#` 对象方法查找;`agent.corpus_search` 自动加载插件;`agent.mcp_server` stdio。金样 `web-proxy-invoke-smoke` · `web-hosting-live` · `web-db-cross-module-smoke` · `agent-mcp-server-smoke`。设计 [ext-hosting-fill.md](doc/design/ext-hosting-fill.md) · [ext-hosting.md](doc/roadmap/ext-hosting.md)。
- **文档**:宿主集成缺口盘点——[ext-hosting-gaps.md](doc/research/ext-hosting-gaps.md)。
### Fixed
- **方法分发**:`find_object_type` 沿导入树传递查找,跨模块拿到的 `web.db` 等句柄可再调方法。
### Changed
## v0.3.6 — 2026-09-07
### Highlights
Generated
+263 -5
View File
@@ -317,6 +317,12 @@ version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "cfg_aliases"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f079e83a288787bcd14a6aea84cee5c87a67c5a3e660c30f557a3d24761b3527"
[[package]]
name = "chacha20"
version = "0.10.2"
@@ -631,6 +637,12 @@ version = "0.3.33"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2cd50c473c80f6d7c3670a752354b8e569b1a7cbfdc0419ec88e5edad85e0dc7"
[[package]]
name = "futures-io"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "53c0fa8157de1303bfffdaa1cc2a673bfffb60102f76b0ef4441659124373fed"
[[package]]
name = "futures-macro"
version = "0.3.33"
@@ -661,9 +673,11 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a77a90a256fce34da66415271e30f94ee91c57b04b8a2c042d9cf3220179deaa"
dependencies = [
"futures-core",
"futures-io",
"futures-macro",
"futures-sink",
"futures-task",
"memchr",
"pin-project-lite",
"slab",
]
@@ -694,8 +708,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0"
dependencies = [
"cfg-if",
"js-sys",
"libc",
"wasi 0.11.1+wasi-snapshot-preview1",
"wasm-bindgen",
]
[[package]]
@@ -717,9 +733,11 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099"
dependencies = [
"cfg-if",
"js-sys",
"libc",
"r-efi 6.0.0",
"rand_core 0.10.1",
"wasm-bindgen",
]
[[package]]
@@ -848,6 +866,23 @@ dependencies = [
"pin-project-lite",
"smallvec",
"tokio",
"want",
]
[[package]]
name = "hyper-rustls"
version = "0.27.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "33ca68d021ef39cf6463ab54c1d0f5daf03377b70561305bb89a8f83aab66e0f"
dependencies = [
"http",
"hyper",
"hyper-util",
"rustls 0.23.44",
"tokio",
"tokio-rustls",
"tower-service",
"webpki-roots 1.0.9",
]
[[package]]
@@ -856,13 +891,21 @@ version = "0.1.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "96547c2556ec9d12fb1578c4eaf448b04993e7fb79cbaad930a656880a6bdfa0"
dependencies = [
"base64",
"bytes",
"futures-channel",
"futures-util",
"http",
"http-body",
"hyper",
"ipnet",
"libc",
"percent-encoding",
"pin-project-lite",
"socket2 0.6.5",
"tokio",
"tower-service",
"tracing",
]
[[package]]
@@ -939,6 +982,12 @@ dependencies = [
"quote",
]
[[package]]
name = "ipnet"
version = "2.12.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "791930b43c0d5973160d90a8f3894509f2b273430f5c5c73b668636d0287c5c0"
[[package]]
name = "is_terminal_polyfill"
version = "1.70.2"
@@ -1022,9 +1071,15 @@ version = "0.4.33"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "lru-slab"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154"
[[package]]
name = "marqdo"
version = "0.3.5"
version = "0.3.9"
dependencies = [
"anyhow",
"assert_cmd",
@@ -1055,7 +1110,7 @@ dependencies = [
[[package]]
name = "marqdo-wasm"
version = "0.3.5"
version = "0.3.9"
dependencies = [
"marqdo",
"serde_json",
@@ -1098,6 +1153,7 @@ version = "0.1.0"
dependencies = [
"argon2",
"axum",
"bytes",
"futures-util",
"getrandom 0.2.17",
"hex",
@@ -1105,6 +1161,7 @@ dependencies = [
"postgres",
"pulldown-cmark",
"redis",
"reqwest",
"rusqlite",
"serde",
"serde_json",
@@ -1448,6 +1505,62 @@ version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "007d8adb5ddab6f8e3f491ac63566a7d5002cc7ed73901f72057943fa71ae1ae"
[[package]]
name = "quinn"
version = "0.11.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c1a41e437b6bbd489372cd4971de128e85c855f56c57f283d20ff016cf7c0a8"
dependencies = [
"bytes",
"cfg_aliases",
"pin-project-lite",
"quinn-proto",
"quinn-udp",
"rustc-hash",
"rustls 0.23.44",
"socket2 0.6.5",
"thiserror 2.0.20",
"tokio",
"tracing",
"web-time",
]
[[package]]
name = "quinn-proto"
version = "0.11.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "04759210543be93709136e28212294a659ef5001836ff4eab4d663e4529bba83"
dependencies = [
"bytes",
"getrandom 0.4.3",
"lru-slab",
"rand 0.10.2",
"rand_pcg",
"ring",
"rustc-hash",
"rustls 0.23.44",
"rustls-pki-types",
"slab",
"thiserror 2.0.20",
"tinyvec",
"tracing",
"web-time",
]
[[package]]
name = "quinn-udp"
version = "0.5.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "35a133f956daabe89a61a685c2649f13d82d5aa4bd5d12d1277e1072a21c0694"
dependencies = [
"cfg_aliases",
"libc",
"once_cell",
"socket2 0.6.5",
"tracing",
"windows-sys 0.61.2",
]
[[package]]
name = "quote"
version = "1.0.47"
@@ -1545,6 +1658,15 @@ version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69"
[[package]]
name = "rand_pcg"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "caa0f4137e1c0a72f4c651489402276c8e8e1cf081f3b0ba156d2cbeef09e86a"
dependencies = [
"rand_core 0.10.1",
]
[[package]]
name = "redis"
version = "0.27.6"
@@ -1601,6 +1723,47 @@ version = "0.8.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
[[package]]
name = "reqwest"
version = "0.12.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eddd3ca559203180a307f12d114c268abf583f59b03cb906fd0b3ff8646c1147"
dependencies = [
"base64",
"bytes",
"futures-core",
"futures-util",
"http",
"http-body",
"http-body-util",
"hyper",
"hyper-rustls",
"hyper-util",
"js-sys",
"log",
"percent-encoding",
"pin-project-lite",
"quinn",
"rustls 0.23.44",
"rustls-pki-types",
"serde",
"serde_json",
"serde_urlencoded",
"sync_wrapper",
"tokio",
"tokio-rustls",
"tokio-util",
"tower",
"tower-http",
"tower-service",
"url",
"wasm-bindgen",
"wasm-bindgen-futures",
"wasm-streams",
"web-sys",
"webpki-roots 1.0.9",
]
[[package]]
name = "ring"
version = "0.17.14"
@@ -1629,6 +1792,12 @@ dependencies = [
"smallvec",
]
[[package]]
name = "rustc-hash"
version = "2.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6b1e7f9a428571be2dc5bc0505c13fb6bf936822b894ec87abf8a08a4e51742d"
[[package]]
name = "rustls"
version = "0.22.4"
@@ -1638,7 +1807,21 @@ dependencies = [
"log",
"ring",
"rustls-pki-types",
"rustls-webpki",
"rustls-webpki 0.102.8",
"subtle",
"zeroize",
]
[[package]]
name = "rustls"
version = "0.23.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6725596c3f2c3a0aef021139e145d4eafe314a6623e4680ca83852b2c67ab2ba"
dependencies = [
"once_cell",
"ring",
"rustls-pki-types",
"rustls-webpki 0.103.15",
"subtle",
"zeroize",
]
@@ -1649,6 +1832,7 @@ version = "1.15.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2f4925028c7eb5d1fcdaf196971378ed9d2c1c4efc7dc5d011256f76c99c0a96"
dependencies = [
"web-time",
"zeroize",
]
@@ -1663,6 +1847,17 @@ dependencies = [
"untrusted",
]
[[package]]
name = "rustls-webpki"
version = "0.103.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f3c3cf1d8b1e7d4927e2d154c3fcb02979afb9939629c62cd9048d4f07b60ac2"
dependencies = [
"ring",
"rustls-pki-types",
"untrusted",
]
[[package]]
name = "rustversion"
version = "1.0.23"
@@ -1902,6 +2097,9 @@ name = "sync_wrapper"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0bf256ce5efdfa370213c1dabab5935a12e49f2c58d15e9eac2870d3b4f27263"
dependencies = [
"futures-core",
]
[[package]]
name = "termtree"
@@ -2029,6 +2227,16 @@ dependencies = [
"whoami",
]
[[package]]
name = "tokio-rustls"
version = "0.26.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b0c85f2c3ef0b1cd58b36682f4b17aaa995f0e5db534d85692b4903abce21f67"
dependencies = [
"rustls 0.23.44",
"tokio",
]
[[package]]
name = "tokio-tungstenite"
version = "0.24.0"
@@ -2105,9 +2313,11 @@ dependencies = [
"pin-project-lite",
"tokio",
"tokio-util",
"tower",
"tower-layer",
"tower-service",
"tracing",
"url",
]
[[package]]
@@ -2142,6 +2352,12 @@ dependencies = [
"once_cell",
]
[[package]]
name = "try-lock"
version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b"
[[package]]
name = "tungstenite"
version = "0.24.0"
@@ -2243,9 +2459,9 @@ dependencies = [
"flate2",
"log",
"once_cell",
"rustls",
"rustls 0.22.4",
"rustls-pki-types",
"rustls-webpki",
"rustls-webpki 0.102.8",
"url",
"webpki-roots 0.26.11",
]
@@ -2301,6 +2517,15 @@ dependencies = [
"libc",
]
[[package]]
name = "want"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bfa7760aed19e106de2c7c0b581b509f2f25d3dacaf737cb82ac61bc6d760b0e"
dependencies = [
"try-lock",
]
[[package]]
name = "wasi"
version = "0.11.1+wasi-snapshot-preview1"
@@ -2347,6 +2572,16 @@ dependencies = [
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-futures"
version = "0.4.77"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6b7777d5cc23d0e91404e53ce2d5e8ec7acae3026b16233dba62cd3246457950"
dependencies = [
"js-sys",
"wasm-bindgen",
]
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.127"
@@ -2379,6 +2614,19 @@ dependencies = [
"unicode-ident",
]
[[package]]
name = "wasm-streams"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "15053d8d85c7eccdbefef60f06769760a563c7f0a9d6902a13d35c7800b0ad65"
dependencies = [
"futures-util",
"js-sys",
"wasm-bindgen",
"wasm-bindgen-futures",
"web-sys",
]
[[package]]
name = "web-sys"
version = "0.3.104"
@@ -2389,6 +2637,16 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "web-time"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a6580f308b1fad9207618087a65c04e7a10bc77e02c8e84e9b00dd4b12fa0bb"
dependencies = [
"js-sys",
"wasm-bindgen",
]
[[package]]
name = "webpki-roots"
version = "0.26.11"
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "marqdo"
version = "0.3.6"
version = "0.3.9"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo — Markdown markers as an interpreted programming language"
+7 -7
View File
@@ -99,7 +99,7 @@ marqdo catalog public -o .marqdo
---
## 现状(v0.3.6)
## 现状(v0.3.9)
- 映射与解释器:Phase I 树遍历 + 字节码后端可用;金样例在 `tests/`
- **对象**:`#` = 类型/构造,`##`+ = 函数/方法;见 [objects.md](doc/design/objects.md)
@@ -107,12 +107,12 @@ marqdo catalog public -o .marqdo
- **`marqdo debug`**:独立调试页(断点 / 单步 / locals;默认端口 7430;页面 favicon / 品牌使用官方 Logo)
- **`marqdo catalog` / `sync`**:OKF 风格 YAML + 模块概念页
- **`marqdo version --check`**:与 GitHub 最新 release 对比
- 标准库:**内置于二进制**(v0.1.2+);磁盘 `lib/` 或 `MARQDO_LIB` 可覆盖。模块含文本、表、**浏览器效应**、文件、系统、时间、JSON、网络、数学、外联、插件、**自写回**、**子任务**;**中层 Mid M1–M6**(`re`/`encoding`/`path`/`hash`/`secrets`/`csv`/`cli`/`log`/`uuid` 及中英对称库,见 [stdlib-mid.md](doc/design/stdlib-mid.md))
- 标准库:**内置于二进制**(v0.1.2+);磁盘 `lib/` 或 `MARQDO_LIB` 可覆盖。模块含文本、表、**浏览器效应**、文件、系统、时间、JSON、网络、数学、外联、插件、**自写回**、**子任务**;**中层 Mid M1–M6** + **Mid2 M7–M10**(datetime/url/toml/html/fs++/stats/log.fields 等,见 [stdlib-mid.md](doc/design/stdlib-mid.md) · [stdlib-mid2.md](doc/design/stdlib-mid2.md))
- **官方扩展库 `ext/`**(**非** stdlib,本版收口):
- **`web`**:W0–W7 + P3 + **W8**;**定制 C0–C4**(admin 前缀/gates/`shell_css`/`layout`/样式 strict/脚本 defer·version/条件导航,见 [ext-web-customization](doc/design/ext-web-customization/));示例 [marqdo-blog](examples/marqdo-blog/);生产路径见 [web-asgi-servers-and-marqdo.md](doc/design/web-asgi-servers-and-marqdo.md)
- **`web`**:W0–W7 + P3 + **W8**;**定制 C0–C4**;**宿主 H1–H2**(`proxy`/`invoke`,见 [ext-hosting.md](doc/roadmap/ext-hosting.md));示例 [marqdo-blog](examples/marqdo-blog/);生产路径见 [web-asgi-servers-and-marqdo.md](doc/design/web-asgi-servers-and-marqdo.md)
- **`quantum`**:Q0–Q7 + Q8a/b 主题 SVG;示例 [quantum-entanglement](examples/quantum-entanglement/)
- **`linalg`**:L0–L6 + 公式文档面(中缀 / `declare` / 展示化简);示例 [linalg-transpose](examples/linalg-transpose/) · [linalg-svd](examples/linalg-svd/) · [linalg-least-squares](examples/linalg-least-squares/)
- **`agent`**:A1–A4(OKF 复用、过程事件、上下文预算、RAG/MCP fixture);下一波缺口 [agent-framework-gaps-after-a4.md](doc/research/agent-framework-gaps-after-a4.md)
- **`agent`**:A1–A4 + **MCP Server stdio**(`mcp_server`);宿主缺口见 [ext-hosting.md](doc/roadmap/ext-hosting.md) · [agent-framework-gaps-after-a4.md](doc/research/agent-framework-gaps-after-a4.md)
- **`llm`**:OpenAI 兼容对话
- 安装:`marqdo ext list` / `add …` / `remove`([ext-cli.md](doc/design/ext-cli.md);默认 `~/.marqdo/ext`)。原生插件先 `cargo build -p marqdo_plugin_*` 再 `ext add`
- **原生插件 ABI**:[`include/marqdo_abi.h`](include/marqdo_abi.h) · [ext-abi.md](doc/design/ext-abi.md);`plugins/{demo,agent,web,quantum,linalg}`
@@ -121,12 +121,12 @@ marqdo catalog public -o .marqdo
- **浏览器 Marqdo(WASM)**:`marqdo wasm build` + 官方 bridge 自启(作者零业务 JS;桥内可含列表/路由/storage/ws/文件/Canvas/音频/Observer/拖放);`lib/browser` + GFM;`web.client_embed`;示例 [browser-hello](examples/browser-hello/) · [browser-app](examples/browser-app/) · [browser-media](examples/browser-media/) · [web-client-site](examples/web-client-site/)([ADR 0002](doc/adr/0002-browser-marqdo-wasm.md) · [D](doc/roadmap/browser-wasm-d.md) · [E](doc/roadmap/browser-wasm-e.md) · [F](doc/roadmap/browser-wasm-f.md))
- 选型:[ADR 0001 — Rust](doc/adr/0001-implementation-language.md) · [ADR 0002 — 浏览器 WASM](doc/adr/0002-browser-marqdo-wasm.md)(C0–C5 完结,见 [roadmap/browser-wasm.md](doc/roadmap/browser-wasm.md))· [ADR 0003 — 异步效应](doc/adr/0003-browser-async-effects.md)
### 如何使用最新 Marqdo(v0.3.6)
### 如何使用最新 Marqdo(v0.3.9)
```bash
# 1) 源码安装(跟 tag 或 main)
git clone https://github.com/cflmy/marqdo.git && cd marqdo
git checkout v0.3.6 # 或留在 main
git checkout v0.3.9 # 或留在 main
cargo build --release
export PATH="$PWD/target/release:$PATH"
marqdo version
@@ -157,7 +157,7 @@ marqdo wasm build
# → dist/wasm/ … 见 examples/browser-hello/
# 6) Windows:也可从 GitHub Releases 下载 exe / zip / vsix
# https://github.com/cflmy/marqdo/releases/tag/v0.3.6
# https://github.com/cflmy/marqdo/releases/tag/v0.3.9
```
开发期也可用 `cargo run -- …` 代替已安装的 `marqdo`:
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "marqdo-wasm"
version = "0.3.6"
version = "0.3.9"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo interpreter as WebAssembly (browser host bridge)"
+14
View File
@@ -32,6 +32,17 @@
| [design/stdlib.md](design/stdlib.md) | **标准库:内置 + `lib/` + 错误约定** |
| [design/stdlib-i18n.md](design/stdlib-i18n.md) | **标准库中英命名:靠库文件名区分,无 lang 开关** |
| [design/stdlib-modules.md](design/stdlib-modules.md) | **L1:文件/系统/时间/JSON/网络/数学/外联/插件** |
| [design/stdlib-mid.md](design/stdlib-mid.md) | **中层 Mid M1–M6(已落地)** |
| [design/stdlib-mid2.md](design/stdlib-mid2.md) | **中层 Mid2:datetime / url / toml / html / fs++** |
| [design/stdlib-datetime.md](design/stdlib-datetime.md) | **`lib/datetime`(Mid2 M7)** |
| [design/stdlib-url.md](design/stdlib-url.md) | **`lib/url`(Mid2 M7)** |
| [design/stdlib-toml.md](design/stdlib-toml.md) | **`lib/toml`(Mid2 M8)** |
| [design/stdlib-html.md](design/stdlib-html.md) | **`lib/html`(Mid2 M8)** |
| [design/stdlib-fs-mid2.md](design/stdlib-fs-mid2.md) | **`lib/fs` Mid2 M9 加厚** |
| [design/stdlib-stats-log.md](design/stdlib-stats-log.md) | **`lib/stats` + log.fields(Mid2 M10)** |
| [research/stdlib-gaps-after-mid.md](research/stdlib-gaps-after-mid.md) | **Mid 之后缺口盘点**(对照 Python/Go) |
| [roadmap/stdlib-mid.md](roadmap/stdlib-mid.md) | **Mid M1–M6 路线(done)** |
| [roadmap/stdlib-mid2.md](roadmap/stdlib-mid2.md) | **Mid2 M7–M10 路线(规划)** |
| [design/stdlib-math.md](design/stdlib-math.md) | **数学库(数值/随机/作图/轻量求解)** |
| [design/stdlib-foreign.md](design/stdlib-foreign.md) | **外联胶水(\`\`\`lang 具名块)— 已落地** |
| [design/ext-abi.md](design/ext-abi.md) | **原生插件 C ABI v1**(`lib/plugin`) |
@@ -71,6 +82,9 @@
| [research/agent-frameworks-and-marqdo.md](research/agent-frameworks-and-marqdo.md) | 主流智能体框架 vs Marqdo 优势 |
| [research/agent-framework-gaps-after-a4.md](research/agent-framework-gaps-after-a4.md) | **`ext/agent` A0–A4 之后缺口盘点** |
| [research/agent-framework-2026-09.md](research/agent-framework-2026-09.md) | **2026-09 刷新:宪法优先 Wave B** |
| [research/ext-hosting-gaps.md](research/ext-hosting-gaps.md) | **宿主集成缺口**:同域 SSE 中继 · HTTP→`##` · MCP Server · 副端口 · 跨模块 db/corpus |
| [design/ext-hosting-fill.md](design/ext-hosting-fill.md) | **宿主缺口补齐设计**(`ext/web` proxy/invoke · `ext/ai` mcp_server) |
| [roadmap/ext-hosting.md](roadmap/ext-hosting.md) | **宿主集成 H0–H4**(规划) |
| [roadmap/ext-agent-optimize.md](roadmap/ext-agent-optimize.md) | **`ext/agent` A0–A4 done · Wave B Active** |
金样例与集成测试:[`tests/`](../tests/)。面向访客的可执行文档:[`public/`](../public/)。Spike 存档:[`spike/`](../spike/)。
+306
View File
@@ -0,0 +1,306 @@
# 设计:宿主集成缺口补齐(`ext/web` · `ext/ai`)
| | |
|---|---|
| 状态 | **Accepted · H1–H4 stdio 已落地**(H4b HTTP 可选) |
| 日期 | 2026-09-07 |
| 缺口盘点 | [ext-hosting-gaps.md](../research/ext-hosting-gaps.md) |
| 路线图 | [ext-hosting.md](../roadmap/ext-hosting.md) |
| 边界 | [ext-web-net.md](ext-web-net.md) · [ext-abi.md](ext-abi.md) · [ext-agent.md](ext-agent.md) |
| 目标 | 用**扩展库一等 API**关闭 GAP-01–05,取代下游 Python 旁路 |
---
## 0. 一句话
**服务器中继与「请求→`##`」进 `ext/web`;MCP Server 与 corpus 可发现进 `ext/ai`;标准库只保留已有客户端 SSE。** 作者用 GFM 表声明路由/工具,不写 JS、不启第二语言运行时。
---
## 1. 分层与禁止项
| 层 | 职责 | 落点 |
|----|------|------|
| 作者面 | 声明路径、上游、函数名、工具表 | `ext/web/web.mq.md` · `ext/ai/agent.mq.md`(+ 中文对偶) |
| ABI | axum 流式代理、每请求 interp 调用、MCP 传输 | `plugins/web` · `plugins/agent` |
| 标准库 | 不变:缓冲客户端 SSE / 解析 | `lib/net`(**不**加中继) |
| 核心 host | **禁止**新增 agent/web 领域 `host_*` | 见 Skill 硬规则 9 |
**禁止:**
1. 为中继在浏览器手写业务 JS。
2. `json=` 继续塞「任意 `##`」语义(保持 DB-only;新能力用独立装配名)。
3. MCP Server 把向量库升为权威真相(工具结果可带 `authority=workbook`)。
4. 无白名单的「任意文件系统路径 + 任意 `##`」远程触发。
---
## 2. GAP-01 · `app.proxy` 流式反向代理
### 2.1 作者 API
```markdown
*应用 = > `应用`.proxy path="/v1/chat/completions" upstream=$OPENAI_BASE_URL*
*应用 = > `应用`.proxy path="/llm/stream" upstream="https://api.example.com" strip_prefix="/llm" stream=True*
```
或装配表(一次挂多条):
```markdown
*应用 = > `应用`.configure proxy=`代理表`*
| path | upstream | stream | strip_prefix | headers_from_env |
|------|----------|--------|--------------|------------------|
| /proxy/chat | $OPENAI_BASE_URL | True | | OPENAI_API_KEY=Authorization |
```
| 参数 | 默认 | 含义 |
|------|------|------|
| `path` | 必填 | 浏览器同域路径 |
| `upstream` | 必填 | 上游根或完整 URL;支持 `$ENV` |
| `stream` | `True` | `True`:响应按上游 `Content-Type` 管道转发(含 `text/event-stream`) |
| `strip_prefix` | 空 | 转发前去掉本站前缀 |
| `methods` | `POST` | 允许方法 |
| `headers_from_env` | 空 | `ENV=Header-Name` 注入(密钥不进前端) |
| `timeout_ms` | 插件默认 | 上游超时 |
中文:`## 代理` / `流式=` / `上游=` / `去前缀=`。
### 2.2 ABI / 插件行为
| 步骤 | 行为 |
|------|------|
| 注册 | `web_app_proxy` 写入 `AppState` 路由表 |
| 请求 | 读 body + 过滤危险 hop-by-hop 头;合并 env 注入头 |
| 上游 | `reqwest`/`hyper` 流式客户端(插件内,不进核心) |
| 响应 | 复制状态码与 `content-type`;`body` 为 stream;SSE 不整段缓冲 |
| CORS | 仍走既有 `configure cors=`(同域调用时浏览器不依赖上游 CORS) |
### 2.3 与 `lib/net.http_post_sse` 关系
| API | 角色 |
|-----|------|
| `http_post_sse` | **出站消费者**(CLI/agent 侧收齐 events) |
| `app.proxy` | **入站中继**(浏览器 ← Marqdo ← 上游) |
二者互补,不互相替代。
### 2.4 验收
- 金样:mock 上游发两帧 SSE,浏览器侧(或 `curl -N`)收到两帧且中间无全缓冲延迟门槛。
- 密钥仅出现在服务端 env 注入头。
- 金样名建议:`tests/ext/web-proxy-sse-smoke.mq.md`。
---
## 3. GAP-02 · `app.invoke` / `configure invoke=`(HTTP → `##`)
### 3.1 作者 API
```markdown
*应用 = > `应用`.invoke path="/api/沉淀" method=POST fn="run.沉淀"*
```
表形式:
```markdown
*应用 = > `应用`.configure invoke=`调用表`*
| path | method | fn | body | return |
|------|--------|-----|------|--------|
| /api/note | POST | run.写出 | json | json |
| /api/search | GET | db.recent | query | json |
```
| 列 / 参数 | 含义 |
|-----------|------|
| `path` / `method` | HTTP 路由 |
| `fn` | `库.成员`(入口模块已 `import`)或 `模块路径##标题`(白名单前缀内) |
| `body` | `json` \| `form` \| `query` \| `raw` → 映射到 `##` 形参(同名优先;其余进 `payload` 袋) |
| `return` | `json`(默认)\| `text` \| `status` |
中文:`## 调用` / `函数=` / `正文=` / `返回=`。
### 3.2 安全默认(锁定)
1. **仅**解析入口 workbook 已导入的 `bind` 路径,或 `invoke_allow=` 显式前缀列表。
2. 单请求超时 + 可选并发上限(防 listen 线程饿死)。
3. 默认不启;作者显式 `invoke` / `configure invoke=`。
4. 错误:`{ok:false, error}` + HTTP 4xx/5xx;不把内部路径栈默认回传生产模式。
### 3.3 ABI / 插件行为
| 步骤 | 行为 |
|------|------|
| 注册 | `web_app_invoke` 存 `(path, method, FnRef)` |
| 命中 | 解析 body → Value 袋;`call_lib_path` / 等价「同步调用已加载模块的 `##`」 |
| 返回 | Value → JSON(复用现有 host JSON 通路);侧效(写盘)在同一解释器事务语义内尽最大努力 |
**不**把 `json=` DB 路由改语义;`invoke` 是平行装配。
### 3.4 与 compose 期 `call_lib_path` 关系
| 时机 | API |
|------|-----|
| 装配页 | 现有 compose / `call_lib_path` |
| **每个 HTTP 请求** | 本设计 `invoke` |
### 3.5 验收
- `POST` JSON → 用户 `##` 写入临时文件 + 返回 `{ok:true,…}`。
- 未导入路径 → 4xx,不执行。
- 金样:`tests/ext/web-invoke-smoke.mq.md`。
---
## 4. GAP-03 · MCP Server(`ext/ai`)
### 4.1 作者 API
```markdown
*服务 = > agent.mcp_server name="marqdo-qd"*
*服务 = > `服务`.tool name="qd_search" fn="求道.搜索" description="关键词检索"*
*服务 = > `服务`.tool name="qd_capture" fn="求道.捕捉"*
*服务 = > `服务`.serve transport=stdio*
# 或
*服务 = > `服务`.serve transport=http host=127.0.0.1 port=7433*
```
工具表:
| name | fn | description |
|------|-----|-------------|
| qd_search | 求道.搜索 | … |
| qd_list | 求道.列出 | … |
### 4.2 传输
| transport | 用途 |
|-----------|------|
| `stdio` | Cursor / Claude Desktop 拉起 |
| `http` | Streamable HTTP(与官方 MCP 演进对齐;首版可先 SSE+POST 子集) |
### 4.3 与 A4 fixture 客户端关系
| API | 方向 |
|-----|------|
| `mcp_list_tools` / `mcp_call` | **出站**(Agent 调外部 MCP;现 fixture) |
| `mcp_server` + `serve` | **入站**(外部调 Marqdo) |
客户端「真 MCP」仍见 agent 缺口文;本设计先交 Server,fixture 可保留测客户端。
### 4.4 ABI
- `plugins/agent`:`agent_mcp_server_*`(注册工具、stdio loop、可选 HTTP)。
- 工具实现:复用 GAP-02 同款「调用 `##`」内核(可抽共享 `invoke_fn`),避免两套分发。
- 返回包装:可含 `authority=workbook`。
### 4.5 验收
- 金样:stdio 夹具发 `tools/list` + `tools/call`,断言 JSON-RPC 形结果。
- 可选:`tests/ext/agent-mcp-server-smoke.mq.md`。
---
## 5. GAP-04 · 同寿副端口
### 5.1 关闭策略(优先)
GAP-01 + GAP-02 挂在**同一** `app.listen` → **不实现副端口**,文档宣布 GAP-04 关闭。
### 5.2 备选 API(仅当必须隔离)
```markdown
*应用 = > `应用`.listen host=127.0.0.1 port=18081 also_port=7432 also_routes=`中继表`*
```
`also_routes` 仅允许 `proxy` / `invoke` 已注册子集。实现成本高,**默认不做**。
---
## 6. GAP-05 · 跨模块 `db` 与 corpus 可发现
### 6.1 `db` 句柄(扩展约定 + 运行时)
**作者约定(立即文档化):**
1. 跨模块优先传 **`url` 字符串**,调用方 `*store = > web.db url=…*` 再 `select`。
2. 若传对象:必须保留 `_type` 为插件注册名(与 `web.db` 构造一致)。
**运行时修补(H 波次):**
| 选项 | 做法 | 选用 |
|------|------|------|
| A | 跨模块返回插件对象时规范 `_type` + 全局方法表查找 | **首选**(一次修好所有插件对象) |
| B | `db.open` 只返回 `{url}` 朴素 map,禁止「裸 select」跨文件 | 文档契约;体验较差 |
金样:模块 A `open` → 返回 → 模块 B `select` 成功。
`tests/ext/web-db-cross-module-smoke.mq.md`。
### 6.2 `corpus_search` 可发现
```markdown
## corpus_search
+ `query`
+ `root`=".marqdo/agent-kb"
* > plugin.load name=agent *
** > agent_corpus_search query=`query` root=`root` **
```
或在 `# agent` / `## corpus_search` 入口统一 `ensure_plugin`(与现 `agent` 构造对齐),保证 **纯 `marqdo run` + import agent** 即可用。
**不**把 `agent_corpus_search` 搬进 `src/host/`。
### 6.3 验收
- 无显式手工 ABI 名的情况下,`marqdo run` 金样调用 `agent.corpus_search` 成功。
- 跨模块 `db.select` 金样绿。
---
## 7. 扩展库表面清单(落地时改这些文件)
| 文件 | 增补 |
|------|------|
| `ext/web/web.mq.md` · `网页.mq.md` | `## proxy` / `## invoke`;`configure` 增 `proxy`/`invoke` 列 |
| `plugins/web/src/http.rs` · `middleware.rs` · 新 `proxy.rs` / `invoke.rs` | 路由注册与流式/调用 |
| `ext/ai/agent.mq.md` · `智能体.mq.md` | `# mcp_server` · `## tool` · `## serve`;`corpus_search` ensure_plugin |
| `plugins/agent/` | MCP Server 传输 + 共享 invoke |
| `tests/ext/web-proxy-sse-smoke.mq.md` 等 | 见上 |
| `doc/roadmap/ext-web.md` | 链到 H 波次(不重开 W0–W7) |
| `public/` / Skill | 发版时补「同域 LLM 代理 / HTTP 调用 ##」短节 |
**不**新增 `lib/run` 作为核心依赖:用户业务 `##` 仍在其自己的 `lib/*.mq.md`;Marqdo 只提供 **invoke 管道**。
---
## 8. 迁移:下游旁路 → 扩展 API
| 旁路(下游) | 替换为 |
|--------------|--------|
| `llm_proxy.py` `/proxy/stream` | `app.proxy` / `configure proxy=` |
| `llm_proxy.py` `/store/*` + `marqdo run` | `app.invoke` → 用户 `##` |
| `mq_bridge.py` 仅过滤 | 用户 `##` 内 `table`/`re` 过滤,或修复后的 `corpus_search` |
| `qdagent_mcp.py` | `agent.mcp_server` + `serve` |
| `mq.sh` 双进程 | 单 `marqdo run index.mq.md`(proxy+invoke 同端口) |
---
## 9. 开放点(实现前拍板)
| # | 问题 | 默认倾向 |
|---|------|----------|
| 1 | `invoke` 的 `fn` 是否允许 `path.mq.md##标题` 字面量 | 允许但必须 `invoke_allow` 前缀 |
| 2 | MCP HTTP 首版范围 | stdio 先交;HTTP 跟一波 |
| 3 | 代理是否缓冲「非 SSE」大 body | 非流式可限长缓冲;SSE 禁止整段缓冲 |
| 4 | GAP-05 选项 A vs B | **A** |
---
## 10. 完成定义(整包)
1. 无 Python 旁路即可:浏览器同域 SSE 对话 + HTTP 触发沉淀 `##` +(可选)MCP stdio 暴露同一批 `##`。
2. 金样覆盖 proxy / invoke / cross-db / corpus / mcp-server(stdio)。
3. `doc/research/ext-hosting-gaps.md` 各 GAP 标为关闭或降级为「备选副端口不做」。
4. 发版说明指向 `ext/web` · `ext/ai` 新作者面,不指向 `scripts/*.py`。
+40
View File
@@ -0,0 +1,40 @@
# 设计:`lib/datetime` / `lib/日期时间`(Mid2 M7)
| | |
|---|---|
| 状态 | **已落地** |
| 日期 | 2026-09-09 |
| 父设计 | [stdlib-mid2.md](stdlib-mid2.md) §2.1 |
| 宿主 | `src/host/datetime.rs` · `host_datetime_*` |
---
## 1. 时刻形状
每次返回 **map**:
| 键 | 含义 |
|----|------|
| `unix` | 秒级 Unix 戳(int) |
| `zone` | 固定偏移文本,如 `+08:00` / `+00:00` |
| `iso` | RFC3339(秒精度) |
`to_unix` 也接受裸 int 或 ISO 文本。
## 2. API
| EN | ZH | 说明 |
|----|-----|------|
| `now` | `现在` | UTC 此刻(wasm 无时钟桥则失败) |
| `from_unix` | `自戳` | `unix` + 可选 `zone` |
| `to_unix` | `成戳` | |
| `parse` | `解析` | 默认 RFC3339 / `YYYY-MM-DD[ HH:MM:SS]`;`pattern=` |
| `format` | `格式化` | `rfc3339`/`date`/`time` 或 strftime |
| `add` | `加` | `days`/`hours`/`minutes`/`seconds` |
| `in_zone` | `时区` | 同瞬间换区 |
**区名**:`+HH:MM` / `UTC` / `Z`,以及少量固定偏移别名(如 `Asia/Shanghai`)。**不做**完整 tzdb / DST。
## 3. 与 `lib/time`
`lib/time` 保留极简 unix/format/parse/sleep。日历语义用本库。
+25
View File
@@ -0,0 +1,25 @@
# 设计:`lib/fs` Mid2 加厚(M9)
| | |
|---|---|
| 状态 | **已落地** |
| 日期 | 2026-09-09 |
| 父设计 | [stdlib-mid2.md](stdlib-mid2.md) §4 · [stdlib-fs-mid.md](stdlib-fs-mid.md)(M2) |
| 宿主 | `src/host/fs.rs` · `host_make_dirs` / `host_remove_tree` / `host_stat` / `host_walk` |
---
## API
| EN | ZH | 行为 |
|----|-----|------|
| `make_dirs` | `建目录树` | 递归创建(与既有 `make_dir` 同为 `create_dir_all`;显式命名) |
| `remove_tree` | `删目录树` | 仅删目录树;路径为文件则报错 |
| `stat` | `状态` | `{size, mtime_unix, is_file, is_dir}` |
| `walk` | `遍历` | **深度优先**;相对路径列表(正斜杠);含目录项;同级按名排序 |
既有 `remove` 对目录仍递归删除;`remove_tree` 拒绝文件,语义更窄。
## 刻意不做
ACL、watch、硬链接、`copy_tree`、惰性迭代器协议。
+19
View File
@@ -0,0 +1,19 @@
# 设计:`lib/html` / `lib/超文本`(Mid2 M8)
| | |
|---|---|
| 状态 | **已落地** |
| 日期 | 2026-09-09 |
| 父设计 | [stdlib-mid2.md](stdlib-mid2.md) §3.2 |
| 宿主 | `src/host/html_ops.rs` · `host_html_escape` / `host_html_unescape` |
---
## API
| EN | ZH | 行为 |
|----|-----|------|
| `escape` | `转义` | `& < > " '` → 实体(`'` → `&#x27;`) |
| `unescape` | `还原` | 上述实体(含 `&#39;` / `&apos;`) |
**不做** DOM / 模板引擎。
+6 -4
View File
@@ -4,7 +4,7 @@
|---|---|
| 状态 | **M1–M6 已落地** |
| 日期 | 2026-09-07 |
| 相关 | [stdlib.md](stdlib.md) · [stdlib-modules.md](stdlib-modules.md) · [ext-abi.md](ext-abi.md) · [roadmap/stdlib-mid.md](../roadmap/stdlib-mid.md) |
| 相关 | [stdlib.md](stdlib.md) · [stdlib-modules.md](stdlib-modules.md) · [ext-abi.md](ext-abi.md) · [roadmap/stdlib-mid.md](../roadmap/stdlib-mid.md) · **后续** [stdlib-mid2.md](stdlib-mid2.md) · [stdlib-gaps-after-mid.md](../research/stdlib-gaps-after-mid.md) |
| 目标 | 在接下来数个发版周期内,把「通用编程」能力沉到 **L0.5 宿主 + L1 `.mq.md`**,减少「凡是稍有能力就必须 ABI 插件」的路径依赖 |
---
@@ -123,10 +123,12 @@ OS / 算法库(精选 crate,默认 features 最小化)
| **M3** | `hash` / `哈希` · `secrets` / `机密` | sha256 / hmac;token | **done** |
| **M4** | `csv` / `逗号表` · `text` 加厚 | parse / stringify;contains / replace | **done** |
| **M5** | `table` 加厚 · `cli` / `命令行` | sort / zip;flags | **done** |
| **M6** | `log` / `日志` · `uuid` / `标识` | level 日志;v4 | 薄 host 或纯 mq |
| **可选** | `yaml` / `Toml` | 仅当 public/catalog 强需求 | 慎入 |
| **M6** | `log` / `日志` · `uuid` / `标识` | level 日志;v4 | **done** |
| **可选(Mid1)** | `yaml` | 默认 **不做** | 见 Mid2:优先 TOML |
**刻意延后:** 完整 `datetime` 时区库、连接池、异步 runtime、模板引擎、完整 `collections` 类型系统——等中层前几波用起来再议。
**Mid1 刻意延后、现已升格到 Mid2:** 日历/时区、URL 解析 —— 见 [stdlib-mid2.md](stdlib-mid2.md) · [stdlib-gaps-after-mid.md](../research/stdlib-gaps-after-mid.md)。
**仍不准入 Mid:** 连接池、异步 runtime、模板引擎、完整 `collections` 类型系统、HTTP 服务端 / SQLite(`ext/`)。
---
+139
View File
@@ -0,0 +1,139 @@
# 设计:中层标准库第二波(Mid2 · M7–M10)
| | |
|---|---|
| 状态 | **Accepted · M7–M10 已落地(zip 跳过)** |
| 日期 | 2026-09-09 |
| 缺口盘点 | [stdlib-gaps-after-mid.md](../research/stdlib-gaps-after-mid.md) |
| 路线图 | [stdlib-mid2.md](../roadmap/stdlib-mid2.md) |
| 宪法 | [stdlib-mid.md](stdlib-mid.md) §3(准入 / 不准入 **不变**) |
| 目标 | 在 M1–M6 之上补「日历 · URL · 配置 · HTML 安全 · fs 再厚」,仍保持宿主面积极小 |
---
## 0. 一句话
**Mid2 = 第二档通用原语,不是第二套 stdlib。** 仍落在 `lib/` + 薄 `host_*`;禁止借机引入 YAML 全家桶、HTTP 服务端、数据库。
---
## 1. 分层与禁止
| 允许 | 禁止 |
|------|------|
| 新 `lib/*.mq.md` 中英对偶 | 新开 `lib-mid2/` 目录树 |
| 少数 `host_datetime_*` / `host_url_*` / `host_toml_parse` | 为 Mid 新增 ABI 插件 |
| 加厚既有 `fs` / `encoding` / `net` / `text` | 把 `ext/web` 能力下沉进 `lib/net` 服务端 |
| TOML **只读** | 完整 YAML 1.2、任意对象反序列化 |
| HTML **转义** | 模板引擎 / CSS 选择器引擎 |
---
## 2. M7 — `datetime` + `url`
### 2.1 `lib/datetime.mq.md` / `lib/日期时间.mq.md`
| 函数(英) | 中文建议 | 行为 |
|------------|----------|------|
| `now` | `现在` | 返回结构化时刻(见下)或 ISO 文本(二选一,实现时锁定) |
| `from_unix` / `to_unix` | `自戳` / `成戳` | 秒级;可选 `ms` |
| `parse` | `解析` | 支持 RFC3339 / 常见 `YYYY-MM-DD[ HH:MM:SS]` |
| `format` | `格式化` | strftime 子集或命名样式 `rfc3339`/`date`/`time` |
| `add` | `加` | `days`/`hours`/`minutes`/`seconds` 具名 |
| `in_zone` | `时区` | IANA 名或 `+08:00`;失败 → 诊断 |
**结构化时刻(建议 map)**:`{unix, zone, iso}` 或宿主不透明柄 + `_type=datetime`。优先 **map + 稳定字段**,便于 GFM/table 协作。
**刻意不做**:农历、营业日、完整 tzdb 热更新 UI、闰秒政治。
### 2.2 `lib/url.mq.md` / `lib/地址.mq.md`(或并入 `net`)
| 函数 | 行为 |
|------|------|
| `parse` | → `{scheme, userinfo, host, port, path, query, fragment}` |
| `query_parse` | `a=1&b=2` → map(多值用列表) |
| `query_stringify` | map → query |
| `join` | base + relative(可选,可二期) |
既有 `net.url_encode` **保留**;文档写清:编码单段用 `net`/`encoding`,解析整 URL 用 `url`。
**验收**:金样解析带 query 的 HTTPS URL;往返 query。
---
## 3. M8 — `toml` + `html` + `encoding` 加厚
### 3.1 `lib/toml.mq.md` / `lib/Toml.mq.md`(或 `配置表`)
| 函数 | 行为 |
|------|------|
| `parse` | TOML 文本 → Marqdo map/list(只读) |
**不做** `stringify`(首版);需要写出时继续用 JSON。动机对齐 PEP 680:配置引导,而非通用文档格式战争。
### 3.2 HTML 转义
二选一(实现时定一):
- A:`lib/html.mq.md` · `escape` / `unescape`
- B:`text.html_escape` / `text.html_unescape`
推荐 **A**(安全语义独立可见)。
### 3.3 `encoding` 加厚
| 函数 | 行为 |
|------|------|
| `base32_encode` / `base32_decode` | RFC 4648 |
| (可选)`percent_decode` | 与 `url` 协作 |
---
## 4. M9 — `fs` 加厚
| 函数 | 行为 |
|------|------|
| `make_dirs` | 递归创建 |
| `remove_tree` | 递归删除(目录) |
| `stat` | `{size, mtime_unix, is_file, is_dir}` |
| `walk` | 返回路径列表(深度优先或广度;文档锁定) |
**不做**:权限 ACL、watch/inotify、硬链接全家桶。
---
## 5. M10 — 收口糖(可选)
| 项 | 说明 |
|----|------|
| `lib/stats` 或 `math.mean`/`median`/`stdev` | 极薄;非 pandas |
| `log` 结构化字段 | 可选 `fields=` map;非完整 slog 生态 |
| `zip` 读写 | **默认跳过**;有明确 Release 需求再开 |
---
## 6. 宿主与 WASM
| Host | WASM |
|------|------|
| `datetime` / `url` / `toml` / `html` / `fs.stat` | 尽量可进;IANA tz 数据体积需评估(可 host-only + 文档标注) |
| `fs.walk` / `remove_tree` | host-only 可接受 |
新 crate 写进 [dependencies.md](dependencies.md);禁止顺手拉大型框架。
---
## 7. 作者面风格
- 中英分文件;形参表声明全部可选并转发 `host_*`。
- 数据优先 GFM / `table`;`json` 仍只 parse/stringify/quote。
- 每波:设计短文 → host → lib → `tests/lib/*` → `public/features` 短页。
---
## 8. 完成定义(Mid2 整包)
1. 作者可用 `datetime` + `url` + `toml.parse` + `html.escape` 写配置驱动脚本,无需 foreign。
2. `fs.stat` / `make_dirs` 金样绿。
3. 研究文高优先级缺口标 closed;YAML/模板/sqlite 仍标明 won't。
4. [stdlib-modules.md](stdlib-modules.md) 总表更新;`stdlib.md` 状态改为 Mid2 进行中/完成。
+13 -6
View File
@@ -2,10 +2,10 @@
| | |
|---|---|
| 状态 | **核心已落地**;**中层 Mid:M1–M6 已落地** · [stdlib-mid.md](stdlib-mid.md) / [roadmap/stdlib-mid.md](../roadmap/stdlib-mid.md) |
| 日期 | 2026-09-07 |
| 状态 | **核心已落地**;**Mid M1–M6 已落地**;**Mid2 规划中** · [stdlib-mid2.md](stdlib-mid2.md) / [roadmap/stdlib-mid2.md](../roadmap/stdlib-mid2.md) |
| 日期 | 2026-09-08 |
| 原则 | 全部经 frontmatter **导入**;除 JSON 外中英分文件;内核保持少而精;**通用能力优先 Mid,插件留给领域核** |
| 相关 | [stdlib.md](stdlib.md) · [stdlib-i18n.md](stdlib-i18n.md) · [stdlib-mid.md](stdlib-mid.md) · [ext-abi.md](ext-abi.md) · [ext-llm.md](ext-llm.md) · [ext-agent.md](ext-agent.md) · [**module-namespace.md**](module-namespace.md)(导入命名空间;**M1/M2 已落地**:`库.成员` / `库.对象.成员` / `use`) |
| 相关 | [stdlib.md](stdlib.md) · [stdlib-i18n.md](stdlib-i18n.md) · [stdlib-mid.md](stdlib-mid.md) · [stdlib-mid2.md](stdlib-mid2.md) · [stdlib-gaps-after-mid.md](../research/stdlib-gaps-after-mid.md) · [ext-abi.md](ext-abi.md) · [ext-llm.md](ext-llm.md) · [ext-agent.md](ext-agent.md) · [**module-namespace.md**](module-namespace.md)(导入命名空间;**M1/M2 已落地**:`库.成员` / `库.对象.成员` / `use`) |
---
@@ -17,17 +17,22 @@
|--------|--------|--------|------|
| 文本 | `lib/text.mq.md` | `lib/文本.mq.md` | 已有 + **Mid M4** 加厚 |
| **正则** | `lib/re.mq.md` | `lib/正则.mq.md` | **Mid M1** |
| **编解码** | `lib/encoding.mq.md` | `lib/编码.mq.md` | **Mid M1** |
| **编解码** | `lib/encoding.mq.md` | `lib/编码.mq.md` | **Mid M1** + Mid2 M8 base32 |
| **路径** | `lib/path.mq.md` | `lib/路径.mq.md` | **Mid M2** |
| **哈希** | `lib/hash.mq.md` | `lib/哈希.mq.md` | **Mid M3** |
| **机密** | `lib/secrets.mq.md` | `lib/机密.mq.md` | **Mid M3** |
| **逗号表** | `lib/csv.mq.md` | `lib/逗号表.mq.md` | **Mid M4** |
| 表 | `lib/table.mq.md` | `lib/表.mq.md` | 已有 + Mid M4/M5:`sort`/`unique`/`zip`/… |
| **命令行** | `lib/cli.mq.md` | `lib/命令行.mq.md` | **Mid M5** |
| **日志** | `lib/log.mq.md` | `lib/日志.mq.md` | **Mid M6** |
| **日志** | `lib/log.mq.md` | `lib/日志.mq.md` | **Mid M6** + Mid2 M10 `fields=` |
| **标识** | `lib/uuid.mq.md` | `lib/标识.mq.md` | **Mid M6** |
| **日期时间** | `lib/datetime.mq.md` | `lib/日期时间.mq.md` | **Mid2 M7** |
| **地址** | `lib/url.mq.md` | `lib/地址.mq.md` | **Mid2 M7** |
| **配置表** | `lib/toml.mq.md` | `lib/配置表.mq.md` | **Mid2 M8** |
| **超文本** | `lib/html.mq.md` | `lib/超文本.mq.md` | **Mid2 M8** |
| **统计** | `lib/stats.mq.md` | `lib/统计.mq.md` | **Mid2 M10** |
| 浏览器效应 | `lib/browser.mq.md` | `lib/浏览器.mq.md` | WASM 客户端效应助手(GFM + helpers;勿用 `json.set` 链) |
| 文件 | `lib/fs.mq.md` | `lib/文件.mq.md` | 已有 + Mid M2:`copy_file` / `move` / `make_temp` |
| 文件 | `lib/fs.mq.md` | `lib/文件.mq.md` | 已有 + Mid M2 + **Mid2 M9**:`make_dirs` / `remove_tree` / `stat` / `walk` |
| 系统 | `lib/sys.mq.md` | `lib/系统.mq.md` | 已有(含 `load_dotenv`) |
| 时间 | `lib/time.mq.md` | `lib/时间.mq.md` | 已有 |
| JSON | `lib/json.mq.md` | **同文件** | **仅** parse / stringify / quote;集合请用 table / GFM |
@@ -42,6 +47,8 @@
**中层已落地:** M1–M6(含 log / uuid)— 见 [stdlib-mid.md](stdlib-mid.md) · [stdlib-log-uuid.md](stdlib-log-uuid.md)。
**Mid2:** **M7–M10 done**(datetime / url / toml / html / base32 / fs++ / stats / log.fields;zip 跳过)— [stdlib-mid2.md](stdlib-mid2.md) · [stdlib-stats-log.md](stdlib-stats-log.md)。
设计细节:[stdlib-writeback.md](stdlib-writeback.md) · [stdlib-subtask.md](stdlib-subtask.md)。
---
+30
View File
@@ -0,0 +1,30 @@
# 设计:`lib/stats` / `lib/统计` 与 `log.fields`(Mid2 M10)
| | |
|---|---|
| 状态 | **已落地** |
| 日期 | 2026-09-09 |
| 父设计 | [stdlib-mid2.md](stdlib-mid2.md) §5 |
| 宿主 | `src/host/stats.rs` · `host_stats_*`;`log::line` 可选 `fields` |
---
## 1. `lib/stats`
| EN | ZH | 行为 |
|----|-----|------|
| `mean` | `均值` | 算术平均 → Num |
| `median` | `中位数` | 排序后中位;偶数取两中点平均 |
| `stdev` | `标准差` | **样本**标准差(n≥2) |
**不做** dataframe、分位数全家桶、加权统计。
## 2. `log` `fields=`
各级别函数可选 `fields`(map)。宿主拼到行尾:`INFO msg key=val …`(map 插入序)。
**不做** 远程 sink、结构化 JSON 日志协议。
## 3. zip
按路线图 **默认跳过**。
+24
View File
@@ -0,0 +1,24 @@
# 设计:`lib/toml` / `lib/配置表`(Mid2 M8)
| | |
|---|---|
| 状态 | **已落地** |
| 日期 | 2026-09-09 |
| 父设计 | [stdlib-mid2.md](stdlib-mid2.md) §3.1 |
| 宿主 | `src/host/toml_ops.rs` · `host_toml_parse`(手写子集,无 `toml` crate) |
---
## 1. API
| EN | ZH | 行为 |
|----|-----|------|
| `parse` | `解析` | TOML 文本 → map/list |
**不做** `stringify`;写出继续用 JSON。
## 2. 子集
支持:`#` 注释、基本/字面字符串、int/float/bool、标量数组、`[table]` 与 dotted key。
**不支持:** `[[array of tables]]`、内联表、TOML datetime、多行字符串。
+24
View File
@@ -0,0 +1,24 @@
# 设计:`lib/url` / `lib/地址`(Mid2 M7)
| | |
|---|---|
| 状态 | **已落地** |
| 日期 | 2026-09-09 |
| 父设计 | [stdlib-mid2.md](stdlib-mid2.md) §2.2 |
| 宿主 | `src/host/url_ops.rs` · `host_url_*`(手写子集,无 `url` crate) |
---
## 1. 与 `net.url_encode`
| API | 用途 |
|-----|------|
| `net.url_encode` | **单段** form 编码(空格→`+`) |
| `url.parse` | 整 URL → `{scheme,userinfo,host,port,path,query,fragment}` |
| `url.query_parse` / `query_stringify` | 查询串 ↔ map(重复键→列表;空格→`%20`) |
## 2. 解析子集
`scheme://[userinfo@]host[:port]/path[?query][#fragment]`,含 IPv6 `[addr]`。缺 path 时规范为 `/`。
**不做**:相对 URL join、完整 WHATWG 边角、IDNA 规范化(host 原样保留)。
+7 -5
View File
@@ -2,13 +2,14 @@
| | |
|---|---|
| 状态 | **S0–S3 + 核心 L1 已落地 · 中层 L1(Mid)规划中** |
| 日期 | 2026-09-07 |
| 相关 | [keywords.md](keywords.md) · [objects.md](objects.md) · [call-arguments.md](call-arguments.md) · [stdlib-i18n.md](stdlib-i18n.md) · [stdlib-modules.md](stdlib-modules.md) · [**stdlib-mid.md**](stdlib-mid.md) · [module-namespace.md](module-namespace.md) · [ext-abi.md](ext-abi.md) |
| 状态 | **S0–S3 + 核心 L1 + Mid M1–M6 + Mid2 M7–M10 已落地** |
| 日期 | 2026-09-09 |
| 相关 | [keywords.md](keywords.md) · [objects.md](objects.md) · [call-arguments.md](call-arguments.md) · [stdlib-i18n.md](stdlib-i18n.md) · [stdlib-modules.md](stdlib-modules.md) · [**stdlib-mid.md**](stdlib-mid.md) · [**stdlib-mid2.md**](stdlib-mid2.md) · [module-namespace.md](module-namespace.md) · [ext-abi.md](ext-abi.md) |
> **多语言库命名**:见 [stdlib-i18n.md](stdlib-i18n.md)(不设 `lang:`;中英分文件)。
> **L1 模块清单**(含 math/foreign/plugin):见 [stdlib-modules.md](stdlib-modules.md)。
> **中层标准库(数个版本)**:通用原语沉 `lib/`、减少「事事插件」——设计 [stdlib-mid.md](stdlib-mid.md) · 路线 [roadmap/stdlib-mid.md](../roadmap/stdlib-mid.md)。
> **中层标准库 Mid(M1–M6)**:已落地——[stdlib-mid.md](stdlib-mid.md) · [roadmap/stdlib-mid.md](../roadmap/stdlib-mid.md)。
> **中层第二波 Mid2(M7–M10)**:已落地——[stdlib-mid2.md](stdlib-mid2.md) · [roadmap/stdlib-mid2.md](../roadmap/stdlib-mid2.md) · 缺口 [stdlib-gaps-after-mid.md](../research/stdlib-gaps-after-mid.md)。
> **`ext/` 不是 stdlib**:官方可选扩展见 [ext-llm.md](ext-llm.md) / [ext-agent.md](ext-agent.md)。
---
@@ -19,7 +20,8 @@
|----|------|
| **L0 内置** | 解释器内:`print` `input` `len` `str` `int` `type` `trim` `split` `join` `at`(`type` 可含 `map`) |
| **L1 核心** | 已有 [`lib/`](../../lib/):fs/sys/time/json/net/math/foreign/plugin/table/text/… |
| **L1 中层(Mid)** | 规划中:re / encoding / path / hash / csv / … — 仍在 `lib/`,见 [stdlib-mid.md](stdlib-mid.md) |
| **L1 中层(Mid)** | **已落地** re / encoding / path / hash / secrets / csv / text++ / table++ / cli / log / uuid — [stdlib-mid.md](stdlib-mid.md) |
| **L1 Mid2** | **已落地** datetime / url / toml / html / fs++ / stats / log.fields — [stdlib-mid2.md](stdlib-mid2.md) |
| **官方 ext** | [`ext/`](../../ext/)(`.mq.md` + 可选原生插件);`MARQDO_EXT`;**不属于** L1;插件仅领域热路径 |
| **L2** | catalog / 用户站收录(随发版);应主要靠核心+中层、默认无插件 |
+2
View File
@@ -13,6 +13,8 @@
**`ext/web` + `plugins/web` 已完成 W0–W7 + P3 路线图**:中间件/JSON API、数据层(连接池/事务/分页/FTS/迁移/外键/审计时间戳)、安全硬化(argon2/CSRF/会话持久化/限速/RBAC)、内容站点标配(SEO/RSS/Markdown/分页 UI)、上传下载/相册/ETag、sitemap/robots/错误页/重定向、WebSocket 广播与访问日志。足以在 Marqdo 上实现博客/CMS/中小型 API 站点;**未内置**的仅剩标签页模板(D6,可用路由+`db.count` 自建)与应用层反垃圾。边界不变:**纯解析进 `lib/net`,HTTP 服务器与领域能力进 `plugins/web` + `ext/web` 作者面**。
**宿主集成下一刀**(同域 LLM SSE 中继、HTTP→用户 `##`、与 MCP Server 协作)不在本 W 表内,见 [ext-hosting-gaps.md](../research/ext-hosting-gaps.md) · [ext-hosting-fill.md](ext-hosting-fill.md) · [ext-hosting.md](../roadmap/ext-hosting.md)。
---
@@ -70,6 +70,7 @@
| **高** | 官方端到端示例 + 文档新鲜度 | 无独立 `examples/agent-*`;Skill/参考里仍有「layout / roadmap」陈旧表述;外人难「照抄即跑通 plan→固化→二次命中」 | `examples/agent-pong`(或同类)+ README;Skill 专节与 A0–A4 对齐 |
| **高** | Agent 评测 / 回归 harness | 有离线金样,缺成本·恢复·命中率·补丁质量的系统评测;业界强调 evaluation 先于上线 | `tests/` 或脚本:命中率、llm_free 占比、失败恢复、可选 live 预算 |
| **高** | 真 MCP(HTTP/SSE)与凭证 | A4 仅 JSON fixture;缺 streamable HTTP、鉴权、resources/prompts | 金样打 mock MCP server;返回仍带 `authority=workbook` |
| **高** | **MCP Server 宿主**(stdio / HTTP) | Agent 只能当客户端 fixture;IDE 无法把 `.mq.md` `##` 暴露为 MCP 工具 | 见 [ext-hosting-gaps.md](ext-hosting-gaps.md) GAP-03 · [ext-hosting-fill.md](../design/ext-hosting-fill.md) |
| **中** | Plan 断点续跑 | 工作簿在盘上,缺显式 checkpoint / resume(对标 LangGraph checkpointer) | `plan resume=` 从第 N 轮 DECISION 继续;进程崩溃可恢复 |
| **中** | 人机协同(HITL)加深 | 仅有 `confirm=True` 建簿即停;缺审批卡、拒绝重开、view 一键继续 | view 审批控件 + 金样 |
| **中** | 飞轮可观测指标 | `hits` / `improve_every` / `llm_free` 有字段,缺汇总与 CLI/view 报表 | `agent_kb_stats` 或 view 面板:命中率、探索浪费 |
+156
View File
@@ -0,0 +1,156 @@
# 调研:宿主集成缺口(浏览器中继 · HTTP→`##` · MCP Server · 同寿副端口 · 跨模块 DB/检索)
| | |
|---|---|
| 状态 | **调研笔记 · 缺口盘点(H1–H4 stdio 已落地;H4b HTTP 可选)** |
| 日期 | 2026-09-07 |
| 触发 | 下游应用(对话站 / 求道类 Agent)在真实集成中被迫旁路 Python 脚本 |
| 基线版本 | **v0.3.6+**(宿主波次 H1–H4 见 [ext-hosting.md](../roadmap/ext-hosting.md)) |
| 相关设计 | [ext-web.md](../design/ext-web.md) · [ext-web-net.md](../design/ext-web-net.md) · [web-net-capabilities.md](../design/web-net-capabilities.md) · [ext-agent.md](../design/ext-agent.md) · [ext-abi.md](../design/ext-abi.md) |
| 邻近缺口 | [agent-framework-gaps-after-a4.md](agent-framework-gaps-after-a4.md)(真 MCP **客户端**) |
| 补齐设计 | [ext-hosting-fill.md](../design/ext-hosting-fill.md) |
| 实现路线 | [ext-hosting.md](../roadmap/ext-hosting.md) |
---
## 0. 一句话结论
`ext/web` 已能做**内容站 / CRUD / JSON-DB API / WS**;`ext/ai` 已能做**工作簿 Agent + 客户端 SSE 消费**。下游要把「浏览器同域流式 LLM」「请求触发任意 `##`」「MCP Server 被 IDE 拉起」合进**单一 Marqdo 进程**时,现有扩展**没有一等装配**——只能外挂缓冲代理 / CLI 子进程。缺口集中在 **扩展库作者面 + `plugins/web|agent` ABI**,不是语法层,也不是再开一套标准库。
**原则:** 旁路脚本只证明需求;关闭缺口必须把能力沉进 `ext/*` + `plugins/*`(可选极薄 `lib/` 编排),禁止把领域逻辑塞进 `src/host/`。
---
## 1. 已有基线(不是缺口)
| 能力 | 现状 | 文档 / 金样 |
|------|------|-------------|
| CORS / 安全头 / gzip / body limit | `app.configure` | `tests/ext/web-middleware-smoke.mq.md` |
| JSON **DB select** 路由 | `configure json=` | `plugins/web/src/middleware.rs` |
| `/_form/*` SQL CRUD | `mount_form` | `plugins/web/src/form.rs` |
| 客户端缓冲 SSE | `lib/net.http_post_sse` → `ext/llm.complete stream=` | `tests/lib/net-openai-sse.mq.md` |
| 执行过程 SSE(非 LLM 中继) | `marqdo view` `/api/run` | [view.md](../design/view.md) §5 |
| Agent 证据工具(fixture MCP) | `corpus_search` / `mcp_list_tools` / `mcp_call` | A4;[agent-framework-gaps-after-a4.md](agent-framework-gaps-after-a4.md) |
| 同模块 `# db` CRUD | `ext/web` `# db` | `examples/web-site/db/` |
---
## 2. 缺口总表
| ID | 缺口 | 需要 | Marqdo 现状 | 典型旁路 | 解除条件 |
|----|------|------|-------------|----------|----------|
| **GAP-01** | 浏览器 → 上游 LLM **同域流式中继** | 绕过对方 CORS;**SSE 原样转发** | **`app.proxy` / `configure proxy=`**(H1) | ~~`llm_proxy.py`~~ | ✅ 已关闭 |
| **GAP-02** | HTTP 处理函数可调用用户 `##` | 请求内执行用户 `##`,返回 JSON | **`app.invoke` / `configure invoke=`**(H2) | ~~外挂 `marqdo run`~~ | ✅ 已关闭 |
| **GAP-03** | **MCP Server** 宿主 | IDE 拉起暴露工具 | **`agent.mcp_server` + `serve` stdio**(H4);HTTP=H4b | ~~`qdagent_mcp.py`~~ | ✅ stdio;HTTP 可选 |
| **GAP-04** | 同寿「第二监听端口」 | 与 `listen` 同寿中继 | 并入主端口(H1+H2) | ~~`mq.sh` 双进程~~ | ✅ won't |
| **GAP-05** | 跨模块 `db`;纯 run 用 corpus | 跨文件 `select`;CLI 检索 | 传递归查找 + `corpus_search` ensure_plugin(H3) | ~~db/index 导出~~ | ✅ 已关闭 |
旁路脚本名来自下游集成仓库;**本仓库 `scripts/` 不含这些文件**——它们是需求证据,不是上游实现。
---
## 3. 分项说明
### 3.1 GAP-01 · 同域 SSE 中继
**问题**
浏览器页面跑在 `ext/web` 源上时,直连 OpenAI 兼容上游会撞 CORS;`lib/net.http_post_sse` 只在 **CLI/宿主侧缓冲整段流**,不能把字节边收边推回浏览器。
**现状边界**
| 已有 | 缺什么 |
|------|--------|
| `app.configure cors=` | 解决的是**本站对外** CORS,不是替上游开 CORS |
| `http_post_sse` → `{status, events}` | **客户端消费**,非服务器中继 |
| `app.download` 对象流 | 存储下载,非上游 SSE |
| view `/api/run` SSE | Marqdo **执行事件**,非 LLM token 流 |
**解除形态(产品)**
作者声明:`path` + `upstream`(或 env)+ 可选 header 白名单;监听端对浏览器返回 `Content-Type: text/event-stream`,chunk 原样或按 OpenAI SSE 帧转发。
### 3.2 GAP-02 · 路由 → `##`
**问题**
对话沉淀、表单后写 `data/runs/*.mq.md`、触发 `lib/run.沉淀` 等,需要**同进程**执行用户函数。今天只有:
1. 页面装配期 `call_lib_path`(compose,非每请求);
2. `/_form` / `json=` 的 **DB 专用** handler。
**解除形态(产品)**
声明式:`path` · `method` · `fn=lib.member` 或 `file=… ## name` · body→形参映射 · 返回值序列化为 JSON(错误码约定)。安全默认:仅允许入口模块已导入路径;禁止任意路径注入。
### 3.3 GAP-03 · MCP Server
**问题**
A4 解决的是 Agent **调用** MCP 工具(且仍是 fixture)。下游要把 Marqdo 工作簿 **暴露为** MCP Server(stdio / Streamable HTTP),IDE 拉起后调 `qd_*`。
与 [agent-framework-gaps-after-a4.md](agent-framework-gaps-after-a4.md)「真 MCP 客户端」正交:本缺口是 **Server 宿主**。
**解除形态(产品)**
`ext/ai`(或 `ext/mcp`)提供:`server tools=` 表(name → `.mq.md`/`##`);`serve stdio=` / `serve http port=`;工具结果仍可带 `authority=workbook`。
### 3.4 GAP-04 · 同寿副端口
**问题**
在 GAP-01/02 未并入主端口前,下游用第二端口跑代理。`app.listen` 同步阻塞,无「同进程再绑一个 TcpListener」的作者 API;只能 shell 并起另一运行时。
**解除形态(产品)**
优先:GAP-01/02 挂在同一 `listen` → **本缺口关闭**。备选:`app.attach` / `listen also=` 同 tokio 运行时附加服务(仅当必须隔离路径树)。
### 3.5 GAP-05 · 跨模块 DB 句柄与 corpus 可发现性
**问题**
1. **类型分发:** 插件对象依赖 `_type` / 方法表;跨模块传回的 store 若丢类型或未注册方法,报 `unknown object type db`。同文件 `db/index.mq.md` 内 `## recent` 再导出 JSON 是现行绕过。
2. **`agent_corpus_search`:** 注册在 `plugins/agent`;普通 `marqdo run` 未 `plugin.load` / 未构造 `# agent` 时不可用。导入 `ext/ai/agent.mq.md` 文本本身不保证原生插件已加载。
**解除形态(产品)**
句柄:跨模块返回值保留 `_type=db` 且方法分发查全局/插件表(修运行时或约定「只返回 url,调用方再 `web.db`」二选一,见补齐设计)。corpus:`plugin.load` 自动路径,或 `ext/ai` 导出独立 `## corpus_search` 并在调用前 `ensure_plugin`。
---
## 4. 归属判定(标准库 vs 扩展)
沿用 [ext-web-net.md](../design/ext-web-net.md) / [web-net-capabilities.md](../design/web-net-capabilities.md) 边界:
| 能力 | 归属 | 理由 |
|------|------|------|
| SSE **中继**路由、HTTP→`##`、副端口 | **`plugins/web` + `ext/web`** | 服务器结构 / 网站领域;axum 已在依赖链 |
| MCP **Server**、corpus 可发现 | **`plugins/agent` + `ext/ai`** | Agent/工具宿主;禁止塞 `src/host/` |
| 缓冲 SSE 客户端、解析 | 已有 **`lib/net`** | 不新增中继到 stdlib |
| 任意路径 shell 出去跑 `marqdo` | **刻意不做进核心** | 旁路脚本可留在下游;上游给一等 API 后废弃 |
---
## 5. 优先级建议
| 优先级 | ID | 理由 |
|--------|-----|------|
| **高** | GAP-01 + GAP-02 | 去掉 Python 中继的主路径;浏览器站 + 沉淀闭环 |
| **高** | GAP-03 | IDE 生态入口;与「真 MCP 客户端」可并行设计、分波交付 |
| **中** | GAP-05 | 减少 `db/index` 与 bridge 胶水;影响面偏语言/插件加载 |
| **低(可随 01/02 关闭)** | GAP-04 | 并入主端口后自然消失 |
---
## 6. 明确「不是缺口」
| 项 | 说明 |
|----|------|
| 内容站 W0–W7+P3 | 已完结;本清单不重开上传/RSS/RBAC |
| Mid M1–M6 | 通用原语已够;本清单不是再开 `lib/re` 一类 |
| 隐藏 JSON TOOL 主循环 | 与 Agent 宪法冲突 |
| Dense embedding 默认化 | 仍见 [okf-near-match.md](../roadmap/okf-near-match.md) |
| 浏览器手写业务 JS 代理 | 与 WASM 路线「作者零 JS」冲突;中继应在服务端 `ext/web` |
---
## 7. 与其它文档
| 文档 | 关系 |
|------|------|
| [ext-hosting-fill.md](../design/ext-hosting-fill.md) | **补齐设计(作者 API + ABI)** |
| [ext-hosting.md](../roadmap/ext-hosting.md) | 实现波次 H0–H4 |
| [ext-web.md](../roadmap/ext-web.md) | 站点主线完结;本缺口为后续宿主波次 |
| [agent-framework-gaps-after-a4.md](agent-framework-gaps-after-a4.md) | 客户端真 MCP;与 GAP-03 Server 互补 |
| [web-asgi-servers-and-marqdo.md](../design/web-asgi-servers-and-marqdo.md) | 反代模型;中继仍在 Marqdo `listen` 内 |
+144
View File
@@ -0,0 +1,144 @@
# 调研:Mid M1–M6 之后标准库还缺什么
| | |
|---|---|
| 状态 | **调研笔记 · 缺口盘点(M1–M6 闭环之后)** |
| 日期 | 2026-09-08 |
| 基线 | **v0.3.7+** · Mid M1–M6 **done** |
| 已落地设计 | [stdlib-mid.md](../design/stdlib-mid.md) · [stdlib-modules.md](../design/stdlib-modules.md) · [roadmap/stdlib-mid.md](../roadmap/stdlib-mid.md) |
| 补齐设计 | [stdlib-mid2.md](../design/stdlib-mid2.md) |
| 实现路线 | [stdlib-mid2.md](../roadmap/stdlib-mid2.md) |
| 外部对照 | Python stdlib([use-case 索引](https://www.socratopia.app/library/python-programming-ai-era-en/chapter-15) · [PEP 680 tomllib](https://peps.python.org/pep-0680/))· Go `strings`/`regexp`/`encoding/*`/`crypto`/`path`/`time`/`net/url` · Node 内置 vs userland |
---
## 0. 一句话结论
**M1–M6 已把「写日常脚本 / 多数扩展库不再事事插件」的硬缺口补齐。** 相对 Python / Go 的「通用原语层」,Marqdo **不再缺** 正则、编解码、路径、哈希、CSV、集合糖、CLI、日志、UUID。
仍缺的是 **第二档通用能力**:日历与时区、URL/查询串、配置格式(优先 TOML)、HTML 安全转义、文件系统再厚一点(`mkdir -p` / 元数据 / 递归删除)、轻量统计。这些**值得开 Mid2(M7+)**,但**不应**再开一轮「事事插件」;也**不要**把 YAML 全家桶、HTTP 服务端、SQLite、图像/ML 塞进 `lib/`。
---
## 1. 已有基线(不是缺口)
| 族 | Marqdo | 对标 |
|----|--------|------|
| 文本 / 正则 | `text`++ · `re` | Python `str`/`re` · Go `strings`/`regexp` |
| 编解码 | `encoding`(base64/hex)· `net.url_encode` | Go `encoding/base64`/`hex` · Python `base64`/`binascii` |
| 路径 / 文件 | `path` · `fs`(含 copy/move/temp) | Go `path/filepath` · Python `pathlib`(子集) |
| 完整性 / 机密 | `hash` · `secrets` | Go `crypto` 子集 · Python `hashlib`/`secrets` |
| 表数据 | `csv` · `table`++ · `json` | Go `encoding/csv`/`json` · Python `csv`/`json` |
| CLI / 可观测 | `cli` · `log` · `uuid` | Python `argparse`/`logging`/`uuid`(窄) |
| 时间(极简) | `time`:unix/ms · format · parse · sleep | **远薄于** Python `datetime`/`zoneinfo` · Go `time` |
| 网络客户端 | `net`:HTTP/SSE/cookie/multipart/markdown | Go `net/http` **客户端**侧;服务端在 `ext/web` |
| 数学 | `math`(含公式/作图) | Python `math` + 教学向扩展 |
准入宪法仍见 [stdlib-mid.md](../design/stdlib-mid.md) §3:跨领域、难写对、解锁无插件写库、宿主面积极小。
---
## 2. 对照总表(按「脚本作者是否还要 foreign / 自造轮子」)
| 能力 | Python / Go 常态 | Marqdo 现状 | 缺口级别 | Mid2 建议 |
|------|------------------|-------------|----------|-----------|
| 日历日期 / 时区 / timedelta | `datetime`+`zoneinfo` · `time` | 仅 unix 戳 + 简单 format/parse | **高** | **M7** `datetime` |
| URL 解析 / 查询串 | `urllib.parse` · `net/url` | 仅 `url_encode` | **高** | **M7** `url`(或并入 `net`) |
| 配置 TOML 只读 | `tomllib`(PEP 680) | 无 | **中高** | **M8** `toml`(只读优先) |
| 配置 YAML | 第三方;**故意不进** CPython | 无 | **低 / 慎入** | 默认 **不做**;与 Python 同理(复杂度/安全) |
| HTML / XML 转义 | `html` · `html/template` 转义 | 无(`markdown_parse` ≠ 转义) | **中** | **M8** `html` 或 `text.html_escape` |
| `mkdir -p` / 递归删 / stat | `pathlib`/`shutil` | `make_dir`/`remove` 偏浅 | **中** | **M9** `fs` 加厚 |
| 文件元数据(size/mtime) | `os.stat` · `Path.stat` | 无一等 API | **中** | **M9** |
| base32 / percent 完整 | `base64.b32*` · `url` | 无 base32 | **低–中** | **M8** `encoding` 加厚 |
| 轻量统计 | `statistics` | `math` 无 mean/median | **低–中** | **M10** `stats` 或 `math` 薄加 |
| 计数 / 默认字典 / deque | `collections` | `table` 可凑 | **低** | 优先用 `table`;必要时 M10 `collections` 极薄 |
| 组合迭代 | `itertools` | 无 | **低** | 多数可用 `table`;**延后** |
| 模板引擎 | Jinja / Go `text/template` | 无 | **不准入 Mid** | 应用层 / 未来 ext |
| SQLite 客户端 | `sqlite3` · `database/sql` | **`ext/web` `# db`** | **不准入** | 保持扩展 |
| HTTP 服务端 | — | **`ext/web`** | **不准入** | 已关闭(宿主波次另文) |
| 密码哈希 Argon2 | 第三方 / 框架 | `ext/web` 已有 | **不准入 Mid** | 安全产品面 |
| 完整 TLS/证书 API | `ssl` · `crypto/tls` | 客户端 HTTPS 由 ureq | **不准入** | 宿主实现细节即可 |
| 压缩 zip/tar | `zipfile` · `archive/zip` | 无 | **低** | 可选 M10;体积敏感 |
| 环境 `.env` | 第三方;Go 常自写 | **`sys.load_dotenv` 已有** | 无 | — |
---
## 3. 分项说明(高优先级)
### 3.1 日历 / 时区(M7 候选之首)
**问题**
脚本与站点大量需要「今天」「本地日界」「RFC3339」「时区换算」。现有 `time.parse`/`format` 够演示,不够「难写对」的日历语义(闰秒除外,但 DST / offset 已够劝退字符串拼接)。
**对照**
Python 强调 **aware datetime + zoneinfo**;Go `time.Location` 是日常依赖。Mid 原文已把「完整 datetime 时区库」标为刻意延后——M1–M6 完成后应升格。
**建议窄 API(设计文锁定)**
`now` / `parse` / `format` / `add`(天/时)/ `to_unix` / `from_unix` / 可选 `in_zone`(IANA 或固定 offset)。**不做**完整历法 UI、农历、业务假期库。
### 3.2 URL 与查询串
**问题**
`url_encode` 只解决「编码一段」;解析 `https://a/b?x=1&y=2#frag`、合并 query、取 path 仍靠 `re` 自造。浏览器代理 / `invoke` / webhook 场景高频。
**建议**
`url.parse` → `{scheme,host,path,query,fragment}`;`url.query_parse` / `query_stringify`;与 `url_encode` 边界写进 `net` vs 新 `lib/url`。
### 3.3 TOML vs YAML
**外部事实**
Python 因 **打包引导**(`pyproject.toml`)接受只读 `tomllib`([PEP 680](https://peps.python.org/pep-0680/));YAML 多次被拒:规范复杂、实现难维护、安全面大。
**对 Marqdo**
catalog / 工具配置若走向 TOML,只读 `toml.parse` 性价比高。YAML:**默认不准入 Mid**;若 public 强需求,单独 ADR,且优先「JSON 超集子集」而非完整 1.2。
### 3.4 HTML 转义
**问题**
`ext/web` 与浏览器 WASM 拼 HTML 时,缺 `escape` 易 XSS。这是「难写对 / 安全相关」的经典 stdlib 项(Python `html.escape`)。
**建议**
极薄:`html.escape` / `unescape`(或挂 `text`)。**不做** DOM / 模板语言。
### 3.5 文件系统再厚
**问题**
`make_dir` 是否递归、`remove` 是否递归、无 `stat`(size/mtime)、无 `walk`——脚本仍会 `foreign` 或脆弱字符串。
**建议**
`fs.make_dirs` · `fs.remove_tree` · `fs.stat` · 可选 `fs.walk`(返回路径列表,非惰性迭代器协议)。
---
## 4. 明确「不是缺口」
| 项 | 说明 |
|----|------|
| Mid M1–M6 清单本身 | 已收口;不要用本表重开 base64/re |
| HTTP 服务 / DB / Agent | `ext/*`;见 [ext-hosting](ext-hosting-gaps.md) |
| Dense embedding / 向量库默认化 | OKF 路线不变 |
| 完整 `collections` / `itertools` 类型系统 | 语言没有泛型迭代协议;用 `table` |
| 「对齐 Python 模块个数」 | 反模式;对齐**用例与难写对** |
---
## 5. 优先级建议(转入 Mid2)
| 优先级 | 波次草案 | 内容 |
|--------|----------|------|
| **高** | **M7** | `datetime` + `url` — **done** |
| **高** | **M8** | `toml` 只读 + `html.escape` + `encoding` base32 — **done** |
| **中** | **M9** | `fs` 加厚(dirs/stat/walk) — **done** |
| **低** | **M10** | `stats` 薄 · log 字段 · (zip 默认跳过) — **done**(zip 仍跳过) |
| **won't** | — | 完整 YAML · 模板引擎 · sqlite 进 `lib/` |
---
## 6. 与其它文档
| 文档 | 关系 |
|------|------|
| [stdlib-mid2.md](../design/stdlib-mid2.md) | **补齐设计(作者 API + 边界)** |
| [stdlib-mid2.md](../roadmap/stdlib-mid2.md) | 实现波次 M7–M10 |
| [stdlib-mid.md](../design/stdlib-mid.md) | Mid1 宪法;本调研不改准入原则 |
| [web-net-capabilities.md](../design/web-net-capabilities.md) | 网络服务端能力在 ext,不进 Mid2 |
+102
View File
@@ -0,0 +1,102 @@
# 路线图:宿主集成补齐(H0–H4)
| | |
|---|---|
| 状态 | **Active · 规划** |
| 日期 | 2026-09-07 |
| 缺口 | [ext-hosting-gaps.md](../research/ext-hosting-gaps.md) |
| 设计 | [ext-hosting-fill.md](../design/ext-hosting-fill.md) |
| 相关 | [ext-web.md](ext-web.md)(W/C 完结)· [ext-agent-optimize.md](ext-agent-optimize.md) · [agent-streaming.md](agent-streaming.md) |
| 基线 | v0.3.6+ |
本文只跟踪**实现波次**。作者 API、ABI 边界、禁止项以设计文为准。
---
## 0. 目标回顾
让 `ext/web` + `ext/ai` 具备:同域 SSE 中继、HTTP→用户 `##`、MCP Server(至少 stdio)、跨模块 db/corpus 可发现——从而下游无需 Python 旁路进程。
---
## 1. 波次总表
| 波次 | 内容 | 关闭缺口 | 状态 |
|------|------|----------|------|
| **H0** | 文档契约:跨模块 `db` 传 `url`;`corpus_search` 调用前 `ensure_plugin` 说明;缺口索引进 `doc/README` | GAP-05 文档侧 | **done** |
| **H1** | `app.proxy` / `configure proxy=`:上游 SSE/HTTP **流式**转发 | GAP-01 | **done** |
| **H2** | `app.invoke` / `configure invoke=`:请求 → 白名单 `##` → JSON | GAP-02 | **done** |
| **H3** | 跨模块插件对象 `_type` 分发修补 + `corpus_search` 自动 `ensure_plugin` | GAP-05 运行时 | **done** |
| **H4** | `agent.mcp_server` + `serve transport=stdio`(HTTP 可选 H4b) | GAP-03 | **done**(stdio) |
| **H4b** | MCP Streamable HTTP | GAP-03 余量 | **todo**(可选) |
| **—** | 副端口 API | GAP-04 | **won't**:H1+H2 同端口关闭 |
建议顺序:**H1 → H2 → H3 → H4**(H2 的 invoke 内核可被 H4 工具复用,故 H2 不宜晚于 H4)。
---
## 2. 分波工作项
### H1 — 流式代理
1. `plugins/web`:`proxy.rs` + `web_app_proxy`;listen 挂流式 handler。
2. `ext/web`:`## proxy` + `configure proxy=` 表。
3. 金样 `web-proxy-sse-smoke`(mock 上游)。
4. 教程/public 短节:「浏览器同域调 LLM」。
**完成定义:** `curl -N` 同域 path 收到多帧 SSE;密钥仅 env 注入。
### H2 — HTTP invoke
1. 共享或插件内 `invoke_fn`(解析 `库.成员`、映射 body)。
2. `web_app_invoke` + 白名单。
3. 金样:临时 `##` 写文件 + JSON 响应;拒绝未允许路径。
4. 迁移说明:替换 `/store/*` + 外挂 `marqdo run`。
**完成定义:** 单进程 `listen` 下表单/fetch 可触发用户 `##`。
### H3 — db / corpus
1. 复现并修跨模块 `unknown object type db`(设计选项 A)。
2. `ext/ai` `corpus_search` 路径 `ensure_plugin`。
3. 金样 `web-db-cross-module-smoke` + corpus CLI run 金样。
### H4 — MCP Server
1. `plugins/agent` stdio JSON-RPC 最小子集:`initialize` / `tools/list` / `tools/call`。
2. `ext/ai` `# mcp_server` · `## tool` · `## serve`。
3. 工具调用复用 H2 `invoke_fn`。
4. 金样夹具;文档对接 Cursor `mcp.json` 示例。
**H4b:** Streamable HTTP 与鉴权;不阻塞 H4 stdio 发版。
---
## 3. 刻意不做
| 项 | 原因 |
|----|------|
| 默认开启任意 `##` 远程执行 | 安全 |
| 中继进 `lib/net` / `src/host` | 边界 |
| 重开 W0–W7 内容站清单 | 已完结 |
| 以副端口为推荐架构 | H1+H2 同端口即可 |
| Dense embedding 默认 | OKF 路线不变 |
---
## 4. 与发版
- 每波可随小版本发布;H1+H2 建议同一版本对外宣传「可去掉 llm_proxy」。
- H4 可标 `ext/ai` 能力点,与「真 MCP 客户端」分列 Release notes。
---
## 5. 进度勾选(落地时改)
- [x] H0 缺口 + 补齐文档
- [x] H1 proxy SSE
- [x] H2 invoke
- [x] H3 db/corpus
- [x] H4 mcp_server stdio
- [ ] H4b mcp http(可选)
- [x] 研究文 GAP 状态改为 closed(H4b 除外)
+1
View File
@@ -69,3 +69,4 @@
| [design/ext-web.md](../design/ext-web.md) | **真理** |
| [user-site.md](../design/user-site.md) | 静态站互补 |
| [ext-quantum.md](ext-quantum.md) | 同属官方 ext + ABI + `ext add` |
| [ext-hosting.md](ext-hosting.md) · [ext-hosting-gaps.md](../research/ext-hosting-gaps.md) | **W/C 之后**:同域 SSE 中继、HTTP→`##`(不重开本表 W0–W7) |
+1 -1
View File
@@ -18,7 +18,7 @@
| **诊断** | 语句级 `path:line:col: message`;字节码指令带 `spans`;errors 金样例双后端断言位置 | **P0 完成** |
| **view** | 结构 + 执行 + 源码 + 函数大纲/搜索;无断点 | 文档浏览器 |
| **debug** | 独立 `marqdo debug`:断点 / 单步 / locals(树遍历) | **P3-D3 v1** |
| **标准库** | 核心 L1 已完成([stdlib-modules.md](../design/stdlib-modules.md));下一主线 = **中层 Mid**([stdlib-mid.md](../design/stdlib-mid.md) · [stdlib-mid 路线](stdlib-mid.md)) | **P2 完成;Mid M1–M6 规划中** |
| **标准库** | 核心 L1 + Mid M1–M6 + Mid2 M7–M10 已完成([stdlib-modules.md](../design/stdlib-modules.md) · [stdlib-mid2.md](../design/stdlib-mid2.md)) | **P2 + Mid1 + Mid2 完成** |
| **模块** | [module-namespace.md](../design/module-namespace.md):**M1+M2 已落地**(库名绑定、`库.成员` / `use`;无扁平合并) | — |
用户点名的三件事都对,但**不能并行硬开**:断点调试依赖「可信位置 + 可暂停执行」;标准库依赖稳定运行时与 I/O;`input` 在 CLI 与 view 是两条路径。
+2 -1
View File
@@ -2,9 +2,10 @@
| | |
|---|---|
| 状态 | **M1–M6 已落地** |
| 状态 | **M1–M6 已落地**(收口) |
| 日期 | 2026-09-07 |
| 设计 | [stdlib-mid.md](../design/stdlib-mid.md) |
| 后续 | [stdlib-mid2.md](stdlib-mid2.md) · [stdlib-gaps-after-mid.md](../research/stdlib-gaps-after-mid.md) |
| 相关 | [stdlib-modules.md](../design/stdlib-modules.md) · [next-phase.md](next-phase.md) · [ext-cli.md](../design/ext-cli.md) |
> **不预标 SemVer。** 每一波随当时产品发版合入;下文用 **M1…M6** 波次名。每波须可单独演示、可回滚。
+91
View File
@@ -0,0 +1,91 @@
# 路线图:中层标准库第二波(M7–M10)
| | |
|---|---|
| 状态 | **Completed · M7–M10 已落地(zip 跳过)** |
| 日期 | 2026-09-09 |
| 设计 | [stdlib-mid2.md](../design/stdlib-mid2.md) |
| 缺口 | [stdlib-gaps-after-mid.md](../research/stdlib-gaps-after-mid.md) |
| 前序 | [stdlib-mid.md](stdlib-mid.md)(**M1–M6 done**) |
| 基线 | v0.3.7+ |
> **不预标 SemVer。** 波次名 **M7…M10**;每波可单独演示、可回滚。
---
## 0. 总序
```text
M7 时间与地址 datetime + url
M8 配置与安全字面 toml(只读) + html.escape + encoding/base32
M9 文件再厚 fs make_dirs / remove_tree / stat / walk
M10 收口糖(可选) stats 薄 · log 字段 · (zip 默认跳过)
```
原则:
1. 继承 Mid1 宪法:通用、难写对、小宿主、可冻 API。
2. **TOML 优先于 YAML**(对齐 PEP 680 教训)。
3. 不阻塞宿主 / ext 产品线;但新「通用」ABI 插件默认拒绝。
---
## 1. 波次明细
### M7 — datetime + url — **done**
| 交付 | 说明 |
|------|------|
| `lib/datetime` · `日期时间` | now/parse/format/add/unix/zone 子集 |
| `lib/url` · `地址` | parse · query_parse/stringify(手写;无 `url` crate) |
| Host | `host_datetime_*` · `host_url_*` |
| 金样 | `tests/lib/datetime-smoke` · `url-smoke`(中英) |
| public | `14-datetime-url` · `14-日期时间地址` |
| 设计 | [stdlib-datetime.md](../design/stdlib-datetime.md) · [stdlib-url.md](../design/stdlib-url.md) |
### M8 — toml + html + encoding — **done**
| 交付 | 说明 |
|------|------|
| `lib/toml` · `配置表` | 只读子集 `parse` |
| `lib/html` · `超文本` | `escape` / `unescape` |
| `encoding` base32 | RFC 4648 |
| 金样 | `toml-html-encoding-smoke`(中英) |
| public | `15-toml-html-encoding` · `15-配置表超文本编码` |
| 设计 | [stdlib-toml.md](../design/stdlib-toml.md) · [stdlib-html.md](../design/stdlib-html.md) |
### M9 — fs 加厚 — **done**
| 交付 | 说明 |
|------|------|
| `make_dirs` / `remove_tree` / `stat` / `walk` | 见 [stdlib-fs-mid2.md](../design/stdlib-fs-mid2.md) |
| 金样 | `fs-mid2-smoke`(中英) |
| public | `16-fs-mid2` · `16-文件mid2` |
### M10 — 可选收口 — **done**(zip 跳过)
| 交付 | 说明 |
|------|------|
| `lib/stats` · `统计` | mean / median / sample stdev |
| `log` `fields=` | 行尾 `key=val` |
| zip | **跳过** |
| 金样 | `stats-log-smoke`(中英) |
| public | `17-stats-log` · `17-统计日志` |
| 设计 | [stdlib-stats-log.md](../design/stdlib-stats-log.md) |
---
## 2. 进度勾选
- [x] M7 datetime + url
- [x] M8 toml + html + base32
- [x] M9 fs 加厚
- [x] M10 可选(zip 跳过)
- [x] 研究文高优先级缺口 → closed(M7–M9;M10 为收口糖)
- [x] `stdlib-modules.md` / `stdlib.md` 状态同步
---
## 3. 成功标准
对外可宣称:在 Mid1(文本/路径/哈希/CSV/CLI/日志)之上,**日历、URL、TOML 配置、HTML 转义、实用 fs** 无需外联即可完成常见脚本与 catalog 工具——领域能力仍在 `ext/`。
+52
View File
@@ -388,6 +388,15 @@ Callable dotted names (whitelist for `CALL:lib…`) plus static module file list
Local keyword search over `.md` / `.mq.md` / `.txt` under `root`. Returns ranked excerpts. **Evidence only** — runnable workbook / OKF skill stays authority (`authority=workbook`).
Ensures the native agent plugin is loaded so plain `marqdo run` (without constructing `# agent`) can call this.
*p = > plugin.native_path name="agent"*
1. `p`
> plugin.load path=`p`
2. *
> print text=ext/ai/agent: native agent plugin not found (build marqdo_plugin_agent or marqdo ext add agent)
> sys.exit code=1
**> agent_corpus_search query=`query` root=`root` limit=`limit`**
---
@@ -2099,3 +2108,46 @@ Deterministic success stop (loop engineering): when the child already returned a
*out = > plan_finish_stream out=`out` events=`events` stream=`stream` trace=`trace` result=`plan_result`*
**out**
---
# mcp_server
+ `name`=marqdo
Minimal MCP **Server** host (stdio JSON-RPC). Tools point at entry-module `lib.member` functions (same bridge as `ext/web` invoke). Evidence/results may carry `authority=workbook`.
*p = > plugin.native_path name="agent"*
1. `p`
> plugin.load path=`p`
2. *
> print text=ext/ai/agent: native agent plugin not found (build marqdo_plugin_agent or marqdo ext add agent)
> sys.exit code=1
*h = > json.parse text={"_type":"mcp_server","tools":[]}*
*h = > json.set map=`h` key="name" value=`name`*
**h**
## tool
+ `name`
+ `fn`
+ `description`=""
Register one tool. `fn` must be `lib.member` imported on the entry workbook.
*tools = > json.get value=`self` key="tools"*
*row = > json.parse text={"name":"","fn":"","description":""}*
*row = > json.set map=`row` key="name" value=`name`*
*row = > json.set map=`row` key="fn" value=`fn`*
*row = > json.set map=`row` key="description" value=`description`*
*tools = > json.append list=`tools` item=`row`*
*h = > json.set map=`self` key="tools" value=`tools`*
**h**
## serve
+ `transport`=stdio
Block on stdio MCP (`initialize` / `tools/list` / `tools/call`). HTTP transport is H4b.
*name = > json.get value=`self` key="name"*
*tools = > json.get value=`self` key="tools"*
**> agent_mcp_serve name=`name` tools=`tools` transport=`transport`**
+29 -3
View File
@@ -653,16 +653,42 @@ Register site icons from a GFM table (`|path|rel|type|sizes|url|` or ZH
+ `json`=None
+ `access_log`=None
+ `cache_control`=None
+ `proxy`=None
+ `invoke`=None
Each capability is declared as a data table and assembled at listen time. The `cors` parameter takes a `|允许来源|方法|头|暴露头|凭证|` table (one row per origin; an empty `允许来源` column means any origin). The `security` parameter takes a `|头|值|` response-header table (e.g. `X-Frame-Options`, `Content-Security-Policy`, `X-Content-Type-Options`, `Referrer-Policy`, `Strict-Transport-Security`). Set `compress` to `True` to gzip response bodies. `body_limit` is the max request body bytes (e.g. `1048576`). The `json` parameter takes a `|路径|方法|表|条件|排序|上限|` table of JSON API endpoints backed by DB queries (each returns `application/json`). Set `access_log` to `True` to log `METHOD path status duration_ms` on stderr. `cache_control` sets a global `Cache-Control` header (e.g. `public, max-age=3600`).
Each capability is declared as a data table and assembled at listen time. The `cors` parameter takes a `|允许来源|方法|头|暴露头|凭证|` table (one row per origin; an empty `允许来源` column means any origin). The `security` parameter takes a `|头|值|` response-header table (e.g. `X-Frame-Options`, `Content-Security-Policy`, `X-Content-Type-Options`, `Referrer-Policy`, `Strict-Transport-Security`). Set `compress` to `True` to gzip response bodies. `body_limit` is the max request body bytes (e.g. `1048576`). The `json` parameter takes a `|路径|方法|表|条件|排序|上限|` table of JSON API endpoints backed by DB queries (each returns `application/json`). Set `access_log` to `True` to log `METHOD path status duration_ms` on stderr. `cache_control` sets a global `Cache-Control` header (e.g. `public, max-age=3600`). `proxy` takes a `|路径|上游|流式|去前缀|方法|环境头|超时|` table of streaming reverse-proxy routes. `invoke` takes a `|路径|方法|函数|正文|返回|` table that calls entry-module `lib.member` handlers per request.
Tables stay as data; `configure` assembles them. 配置即数据、装配即函数.
**> web_app_middleware app=`self` cors=`cors` security=`security` compress=`compress` body_limit=`body_limit` json_routes=`json` access_log=`access_log` cache_control=`cache_control`**
**> web_app_middleware app=`self` cors=`cors` security=`security` compress=`compress` body_limit=`body_limit` json_routes=`json` access_log=`access_log` cache_control=`cache_control` proxy=`proxy` invoke=`invoke`**
## proxy
+ `path`
+ `upstream`
+ `stream`=True
+ `strip_prefix`=None
+ `methods`=None
+ `headers_from_env`=None
+ `timeout_ms`=120000
Same-origin reverse proxy to an upstream HTTP(S) URL. When `stream` is True (default), the response body — including `text/event-stream` SSE — is piped without buffering the full payload, and responses include `Cache-Control: no-cache, no-transform` plus `X-Accel-Buffering: no` for Nginx/CDN. `upstream` may contain `$ENV` / `${ENV}`. `headers_from_env` is `ENV=Header-Name` (comma-separated); secrets stay on the server. Prefer this over a side Python LLM proxy.
**> web_app_proxy app=`self` path=`path` upstream=`upstream` stream=`stream` strip_prefix=`strip_prefix` methods=`methods` headers_from_env=`headers_from_env` timeout_ms=`timeout_ms`**
## invoke
+ `path`
+ `fn`
+ `method`=POST
+ `body`=json
+ `return`=json
Call a user `##` on each HTTP request. `fn` must be `lib.member` imported on the entry module. `body` is `json` / `form` / `query` / `raw`; object keys map to named params (plus `payload`). Returns JSON by default.
**> web_app_invoke app=`self` path=`path` method=`method` fn=`fn` body=`body` return=`return`**
## listen
Serve `/`, routed pages, `/_part/{id}` (home) and `{path}/_part/{id}` (routes), `/_form/{id}` (from mounts + page embeds), optional `/static` (or custom mount), optional `/admin`, upload/download/WebSocket/RSS/sitemap/robots routes, redirects, and a custom 404 fallback. Production HTTPS should terminate at a reverse proxy; set `cookie_secure=True` when serving over TLS.
Serve `/`, routed pages, `/_part/{id}` (home) and `{path}/_part/{id}` (routes), `/_form/{id}` (from mounts + page embeds), optional `/static` (or custom mount), optional `/admin`, upload/download/WebSocket/RSS/sitemap/robots routes, redirects, proxy/invoke routes, and a custom 404 fallback. Production HTTPS should terminate at a reverse proxy; set `cookie_secure=True` when serving over TLS.
**> web_listen app=`self`**
+29 -3
View File
@@ -623,16 +623,42 @@ SEO / OpenGraph 元数据表(`|键|值|`)。如 `title`、`description`、`c
+ `接口`=None
+ `访问日志`=None
+ `缓存控制`=None
+ `代理`=None
+ `调用`=None
每项能力用数据表声明,`监听` 时统一装配。`跨域` 参数接受 `|允许来源|方法|头|暴露头|凭证|` 表(每来源一行;`允许来源` 留空表示允许任意来源)。`安全` 参数接受 `|头|值|` 响应头表(如 `X-Frame-Options`、`Content-Security-Policy`、`X-Content-Type-Options`、`Referrer-Policy`、`Strict-Transport-Security`)。`压缩` 为 `真` 时对响应体做 gzip 压缩。`请求体上限` 是请求体最大字节数(如 `1048576`)。`接口` 参数接受 `|路径|方法|表|条件|排序|上限|` JSON API 端点表(每个返回 `application/json`)。`访问日志` 为 `真` 时在 stderr 打印 `METHOD path status duration_ms`。`缓存控制` 设置全局 `Cache-Control`(如 `public, max-age=3600`)。
每项能力用数据表声明,`监听` 时统一装配。`跨域` 参数接受 `|允许来源|方法|头|暴露头|凭证|` 表(每来源一行;`允许来源` 留空表示允许任意来源)。`安全` 参数接受 `|头|值|` 响应头表(如 `X-Frame-Options`、`Content-Security-Policy`、`X-Content-Type-Options`、`Referrer-Policy`、`Strict-Transport-Security`)。`压缩` 为 `真` 时对响应体做 gzip 压缩。`请求体上限` 是请求体最大字节数(如 `1048576`)。`接口` 参数接受 `|路径|方法|表|条件|排序|上限|` JSON API 端点表(每个返回 `application/json`)。`访问日志` 为 `真` 时在 stderr 打印 `METHOD path status duration_ms`。`缓存控制` 设置全局 `Cache-Control`(如 `public, max-age=3600`)。`代理` 接受 `|路径|上游|流式|去前缀|方法|环境头|超时|` 流式反向代理表。`调用` 接受 `|路径|方法|函数|正文|返回|` 表,按请求调用入口模块已导入的 `库.成员`。
表仍是数据,`装配` 把它们变成能力。配置即数据、装配即函数。
**> web_app_middleware app=`self` cors=`跨域` security=`安全` compress=`压缩` body_limit=`请求体上限` json_routes=`接口` access_log=`访问日志` cache_control=`缓存控制`**
**> web_app_middleware app=`self` cors=`跨域` security=`安全` compress=`压缩` body_limit=`请求体上限` json_routes=`接口` access_log=`访问日志` cache_control=`缓存控制` proxy=`代理` invoke=`调用`**
## 代理
+ `路径`
+ `上游`
+ `流式`=真
+ `去前缀`=None
+ `方法`=None
+ `环境头`=None
+ `超时`=120000
同域反向代理到上游 HTTP(S)。`流式` 为真(默认)时管道转发响应体(含 `text/event-stream`),不整段缓冲。`上游` 可含 `$ENV` / `${ENV}`。`环境头` 为 `ENV=Header-Name`(逗号分隔),密钥留在服务端。
**> web_app_proxy app=`self` path=`路径` upstream=`上游` stream=`流式` strip_prefix=`去前缀` methods=`方法` headers_from_env=`环境头` timeout_ms=`超时`**
## 调用
+ `路径`
+ `函数`
+ `方法`=POST
+ `正文`=json
+ `返回`=json
每个 HTTP 请求调用用户 `##`。`函数` 必须是入口模块已 `导入` 的 `库.成员`。`正文` 为 `json` / `form` / `query` / `raw`;对象键映射为具名形参(另有 `payload`)。默认返回 JSON。
**> web_app_invoke app=`self` path=`路径` method=`方法` fn=`函数` body=`正文` return=`返回`**
## 监听
提供 `/`、路由页、首页 `/_part/{id}`、路由 `{路径}/_part/{id}`、`/_form/{id}`、可选静态目录、可选 `/admin`,上传/下载/WebSocket/RSS/sitemap/robots、重定向,以及自定义 404。生产 HTTPS 请在反向代理终止;启用 TLS 时设 `cookie_secure`。
提供 `/`、路由页、首页 `/_part/{id}`、路由 `{路径}/_part/{id}`、`/_form/{id}`、可选静态目录、可选 `/admin`,上传/下载/WebSocket/RSS/sitemap/robots、重定向、代理/调用路由,以及自定义 404。生产 HTTPS 请在反向代理终止;启用 TLS 时设 `cookie_secure`。
**> web_listen app=`self`**
+54
View File
@@ -0,0 +1,54 @@
---
title: lib/datetime
description: Calendar moments as maps {unix, zone, iso} (Mid2 M7). Fixed offsets + a few IANA aliases.
---
## now
Current UTC moment map.
**> host_datetime_now**
## from_unix
+ `unix`
+ `zone`=None
**> host_datetime_from_unix unix=`unix` zone=`zone`**
## to_unix
+ `dt`
**> host_datetime_to_unix dt=`dt`**
## parse
+ `text`
+ `pattern`=None
RFC3339 / `YYYY-MM-DD[ HH:MM:SS]` by default; `pattern=` is strftime or `rfc3339`.
**> host_datetime_parse text=`text` pattern=`pattern`**
## format
+ `dt`
+ `style`=None
Styles: `rfc3339` (default), `date`, `time`, or a strftime pattern.
**> host_datetime_format dt=`dt` style=`style`**
## add
+ `dt`
+ `days`=None
+ `hours`=None
+ `minutes`=None
+ `seconds`=None
**> host_datetime_add dt=`dt` days=`days` hours=`hours` minutes=`minutes` seconds=`seconds`**
## in_zone
+ `dt`
+ `zone`
Re-express the same instant in `+08:00` / `UTC` / fixed aliases like `Asia/Shanghai`.
**> host_datetime_in_zone dt=`dt` zone=`zone`**
+13 -1
View File
@@ -1,6 +1,6 @@
---
title: lib/encoding
description: Base64 and hex encode/decode (Mid M1). UTF-8 text round-trip.
description: Base64, hex, and base32 encode/decode (Mid M1 + Mid2 M8). UTF-8 text round-trip.
---
## base64_encode
@@ -22,3 +22,15 @@ description: Base64 and hex encode/decode (Mid M1). UTF-8 text round-trip.
+ `text`
**> host_encoding_hex_decode text=`text`**
## base32_encode
+ `text`
RFC 4648 alphabet; padding with `=`.
**> host_encoding_base32_encode text=`text`**
## base32_decode
+ `text`
**> host_encoding_base32_decode text=`text`**
+28
View File
@@ -35,11 +35,39 @@ description: English filesystem wrappers
**> host_make_dir path=`path`**
## make_dirs
+ `path`
Create directories recursively (`mkdir -p`).
**> host_make_dirs path=`path`**
## remove
+ `path`
**> host_remove path=`path`**
## remove_tree
+ `path`
Remove a directory recursively. Errors if `path` is a file.
**> host_remove_tree path=`path`**
## stat
+ `path`
Returns `{size, mtime_unix, is_file, is_dir}`.
**> host_stat path=`path`**
## walk
+ `path`
Depth-first listing of relative paths under `path` (forward slashes).
**> host_walk path=`path`**
## copy_file
+ `src`
+ `dest`
+14
View File
@@ -0,0 +1,14 @@
---
title: lib/html
description: HTML escape / unescape (Mid2 M8). XSS floor for string assembly.
---
## escape
+ `text`
**> host_html_escape text=`text`**
## unescape
+ `text`
**> host_html_unescape text=`text`**
+9 -5
View File
@@ -1,6 +1,6 @@
---
title: lib/log
description: Level-filtered one-line logs (Mid M6). Default level info.
description: Level-filtered one-line logs (Mid M6 + Mid2 M10 fields). Default level info.
---
## set_level
@@ -12,8 +12,9 @@ level is debug / info / warn / error.
## debug
+ `text`
+ `fields`=None
*line = > host_log_line level="debug" text=`text`*
*line = > host_log_line level="debug" text=`text` fields=`fields`*
1. `line`
> print text=`line`
2. *
@@ -21,8 +22,9 @@ level is debug / info / warn / error.
## info
+ `text`
+ `fields`=None
*line = > host_log_line level="info" text=`text`*
*line = > host_log_line level="info" text=`text` fields=`fields`*
1. `line`
> print text=`line`
2. *
@@ -30,8 +32,9 @@ level is debug / info / warn / error.
## warn
+ `text`
+ `fields`=None
*line = > host_log_line level="warn" text=`text`*
*line = > host_log_line level="warn" text=`text` fields=`fields`*
1. `line`
> print text=`line`
2. *
@@ -39,8 +42,9 @@ level is debug / info / warn / error.
## error
+ `text`
+ `fields`=None
*line = > host_log_line level="error" text=`text`*
*line = > host_log_line level="error" text=`text` fields=`fields`*
1. `line`
> print text=`line`
2. *
+21
View File
@@ -0,0 +1,21 @@
---
title: lib/stats
description: Thin mean / median / sample stdev (Mid2 M10). Not a dataframe library.
---
## mean
+ `list`
**> host_stats_mean list=`list`**
## median
+ `list`
**> host_stats_median list=`list`**
## stdev
+ `list`
Sample standard deviation; needs at least two values.
**> host_stats_stdev list=`list`**
+8 -1
View File
@@ -37,8 +37,15 @@ Load `.env` (optional named arg `path=`). Existing process env is not overridden
## exec
+ `cmd`
+ `args`=None
+ `capture`=False
**> host_exec cmd=`cmd` args=`args`**
Run a process. Default return is the exit code (int).
With `capture=True`, return a map `{code, stdout, stderr}` (stdout/stderr may be truncated for huge output).
Inside `*…*` prefer bare `args=args` (variable); `` args=`args` `` is also accepted.
**> host_exec cmd=`cmd` args=`args` capture=`capture`**
## stream_publish
+ `event`
+9
View File
@@ -0,0 +1,9 @@
---
title: lib/toml
description: Read-only TOML subset parse (Mid2 M8). Tables, scalars, arrays; no array-of-tables.
---
## parse
+ `text`
**> host_toml_parse text=`text`**
+23
View File
@@ -0,0 +1,23 @@
---
title: lib/url
description: Parse URLs and query strings (Mid2 M7). Segment encoding stays in net.url_encode.
---
## parse
+ `text`
Returns `{scheme, userinfo, host, port, path, query, fragment}`.
**> host_url_parse text=`text`**
## query_parse
+ `text`
Parse `a=1&b=2` into a map; repeated keys become a list.
**> host_url_query_parse text=`text`**
## query_stringify
+ `map`
**> host_url_query_stringify map=`map`**
+19
View File
@@ -0,0 +1,19 @@
---
title: lib/地址
description: 解析 URL 与查询串(中层 Mid2 M7)。单段编码仍用 `网络.网址编码`。
---
## 解析
+ `内容`
**> host_url_parse text=`内容`**
## 查询解析
+ `内容`
**> host_url_query_parse text=`内容`**
## 查询串化
+ `表`
**> host_url_query_stringify map=`表`**
+28
View File
@@ -35,11 +35,39 @@ description: 中文文件系统包装
**> host_make_dir path=`路径`**
## 建目录树
+ `路径`
递归创建目录(等同 `mkdir -p`)。
**> host_make_dirs path=`路径`**
## 删除
+ `路径`
**> host_remove path=`路径`**
## 删目录树
+ `路径`
递归删除目录;若路径是文件则报错。
**> host_remove_tree path=`路径`**
## 状态
+ `路径`
返回 `{size, mtime_unix, is_file, is_dir}`。
**> host_stat path=`路径`**
## 遍历
+ `路径`
深度优先列出相对路径(正斜杠)。
**> host_walk path=`路径`**
## 复制文件
+ `源`
+ `目标`
+9 -5
View File
@@ -1,6 +1,6 @@
---
title: lib/日志
description: 按级别过滤的一行日志(中层 M6)。默认级别 info。
description: 按级别过滤的一行日志(中层 M6 + Mid2 M10 字段)。默认级别 info。
---
## 设级别
@@ -12,8 +12,9 @@ description: 按级别过滤的一行日志(中层 M6)。默认级别 info
## 调试
+ `内容`
+ `字段`=None
*line = > host_log_line level="debug" text=`内容`*
*line = > host_log_line level="debug" text=`内容` fields=`字段`*
1. `line`
> 打印 内容=`line`
2. *
@@ -21,8 +22,9 @@ description: 按级别过滤的一行日志(中层 M6)。默认级别 info
## 信息
+ `内容`
+ `字段`=None
*line = > host_log_line level="info" text=`内容`*
*line = > host_log_line level="info" text=`内容` fields=`字段`*
1. `line`
> 打印 内容=`line`
2. *
@@ -30,8 +32,9 @@ description: 按级别过滤的一行日志(中层 M6)。默认级别 info
## 警告
+ `内容`
+ `字段`=None
*line = > host_log_line level="warn" text=`内容`*
*line = > host_log_line level="warn" text=`内容` fields=`字段`*
1. `line`
> 打印 内容=`line`
2. *
@@ -39,8 +42,9 @@ description: 按级别过滤的一行日志(中层 M6)。默认级别 info
## 错误
+ `内容`
+ `字段`=None
*line = > host_log_line level="error" text=`内容`*
*line = > host_log_line level="error" text=`内容` fields=`字段`*
1. `line`
> 打印 内容=`line`
2. *
+46
View File
@@ -0,0 +1,46 @@
---
title: lib/日期时间
description: 日历时刻 map `{unix, zone, iso}`(中层 Mid2 M7)。固定偏移与少量 IANA 别名。
---
## 现在
**> host_datetime_now**
## 自戳
+ `秒`
+ `区`=None
**> host_datetime_from_unix unix=`秒` zone=`区`**
## 成戳
+ `时刻`
**> host_datetime_to_unix dt=`时刻`**
## 解析
+ `内容`
+ `格式`=None
**> host_datetime_parse text=`内容` pattern=`格式`**
## 格式化
+ `时刻`
+ `样式`=None
**> host_datetime_format dt=`时刻` style=`样式`**
## 加
+ `时刻`
+ `天`=None
+ `小时`=None
+ `分钟`=None
+ `秒数`=None
**> host_datetime_add dt=`时刻` days=`天` hours=`小时` minutes=`分钟` seconds=`秒数`**
## 时区
+ `时刻`
+ `区`
**> host_datetime_in_zone dt=`时刻` zone=`区`**
+8 -1
View File
@@ -37,8 +37,15 @@ description: 中文系统包装(加载环境 可选 路径=)
## 执行
+ `命令`
+ `参数`=None
+ `捕获`=False
**> host_exec cmd=`命令` args=`参数`**
运行子进程。默认返回退出码(整数)。
`捕获=True` 时返回映射 `{code, stdout, stderr}`(过大输出可能截断)。
在 `*…*` 内优先写裸变量 `参数=参数`;`` 参数=`参数` `` 亦可。
**> host_exec cmd=`命令` args=`参数` capture=`捕获`**
## 发布流事件
+ `事件`
+21
View File
@@ -0,0 +1,21 @@
---
title: lib/统计
description: 薄均值 / 中位数 / 样本标准差(中层 Mid2 M10)。非数据框库。
---
## 均值
+ `列表`
**> host_stats_mean list=`列表`**
## 中位数
+ `列表`
**> host_stats_median list=`列表`**
## 标准差
+ `列表`
样本标准差;至少两个值。
**> host_stats_stdev list=`列表`**
+11 -1
View File
@@ -1,6 +1,6 @@
---
title: lib/编码
description: Base64 与十六进制编解码(中层 M1)。UTF-8 文本往返。
description: Base64、十六进制与基32编解码(中层 M1 + Mid2 M8)。UTF-8 文本往返。
---
## 基64编码
@@ -22,3 +22,13 @@ description: Base64 与十六进制编解码(中层 M1)。UTF-8 文本往返
+ `内容`
**> host_encoding_hex_decode text=`内容`**
## 基32编码
+ `内容`
**> host_encoding_base32_encode text=`内容`**
## 基32解码
+ `内容`
**> host_encoding_base32_decode text=`内容`**
+14
View File
@@ -0,0 +1,14 @@
---
title: lib/超文本
description: HTML 转义 / 还原(中层 Mid2 M8)。拼装字符串时的 XSS 底线。
---
## 转义
+ `内容`
**> host_html_escape text=`内容`**
## 还原
+ `内容`
**> host_html_unescape text=`内容`**
+9
View File
@@ -0,0 +1,9 @@
---
title: lib/配置表
description: 只读 TOML 子集解析(中层 Mid2 M8)。表 / 标量 / 数组;无数组表。
---
## 解析
+ `内容`
**> host_toml_parse text=`内容`**
+15 -1
View File
@@ -2,6 +2,7 @@
mod corpus;
mod kb;
mod mcp_server;
use std::collections::HashMap;
use std::ffi::{CStr, CString};
@@ -548,10 +549,18 @@ fn register(host: &MarqdoHostApi, name: &str, params: &str, fn_ptr: PluginFn) ->
}
pub(crate) fn host_query_json(name: &str) -> Result<serde_json::Value, String> {
host_query_json_args(name, &serde_json::json!({}))
}
pub(crate) fn host_query_json_args(
name: &str,
args: &serde_json::Value,
) -> Result<serde_json::Value, String> {
let query = unsafe { HOST_QUERY }.ok_or_else(|| "host_query not available".to_string())?;
let userdata = unsafe { HOST_USERDATA };
let c_name = CString::new(name).map_err(|e| e.to_string())?;
let c_args = CString::new("{}").unwrap();
let args_s = serde_json::to_string(args).map_err(|e| e.to_string())?;
let c_args = CString::new(args_s).map_err(|e| e.to_string())?;
let mut out_ptr: *mut c_char = ptr::null_mut();
let mut err_ptr: *mut c_char = ptr::null_mut();
let rc = unsafe {
@@ -987,6 +996,11 @@ pub unsafe extern "C" fn marqdo_plugin_init(host: *const MarqdoHostApi) -> c_int
if register(host, "agent_mcp_fixture", "action,fixture", agent_mcp_fixture) != 0 {
return 1;
}
if register(host, "agent_mcp_serve", "name,tools,transport", mcp_server::agent_mcp_serve)
!= 0
{
return 1;
}
if register(host, "agent_kb_add_alias", "kb_dir,slug,alias", agent_kb_add_alias) != 0 {
return 1;
}
+212
View File
@@ -0,0 +1,212 @@
//! Minimal MCP Server (stdio JSON-RPC) — tools map to `lib.member` via host `call_lib_path`.
use serde_json::{json, Map, Value};
use std::ffi::CStr;
use std::io::{BufRead, BufReader, Write};
use std::os::raw::{c_char, c_int};
use crate::{host_query_json_args, parse_args, set_err, set_out};
fn read_line_message(reader: &mut impl BufRead) -> Result<Option<Value>, String> {
let mut line = String::new();
let n = reader
.read_line(&mut line)
.map_err(|e| format!("stdin: {e}"))?;
if n == 0 {
return Ok(None);
}
let line = line.trim();
if line.is_empty() {
return read_line_message(reader);
}
let v: Value = serde_json::from_str(line).map_err(|e| format!("json-rpc parse: {e}"))?;
Ok(Some(v))
}
fn write_message(msg: &Value) -> Result<(), String> {
let mut out = std::io::stdout().lock();
let s = serde_json::to_string(msg).map_err(|e| e.to_string())?;
writeln!(out, "{s}").map_err(|e| format!("stdout: {e}"))?;
out.flush().map_err(|e| format!("flush: {e}"))?;
Ok(())
}
fn rpc_result(id: &Value, result: Value) -> Value {
json!({
"jsonrpc": "2.0",
"id": id,
"result": result,
})
}
fn rpc_error(id: &Value, code: i64, message: &str) -> Value {
json!({
"jsonrpc": "2.0",
"id": id,
"error": { "code": code, "message": message },
})
}
fn tools_list(tools: &[Value]) -> Value {
let mut arr = Vec::new();
for t in tools {
let name = t.get("name").and_then(|v| v.as_str()).unwrap_or("");
let desc = t
.get("description")
.and_then(|v| v.as_str())
.unwrap_or("");
if name.is_empty() {
continue;
}
arr.push(json!({
"name": name,
"description": desc,
"inputSchema": {
"type": "object",
"properties": {},
"additionalProperties": true
}
}));
}
json!({ "tools": arr })
}
fn call_tool(tools: &[Value], name: &str, arguments: &Value) -> Result<Value, String> {
let tool = tools
.iter()
.find(|t| t.get("name").and_then(|v| v.as_str()) == Some(name))
.ok_or_else(|| format!("unknown tool `{name}`"))?;
let fn_path = tool
.get("fn")
.and_then(|v| v.as_str())
.ok_or_else(|| format!("tool `{name}` missing fn"))?
.to_string();
let result = host_query_json_args(
"call_lib_path",
&json!({
"path": fn_path,
"args": arguments,
}),
)?;
Ok(json!({
"content": [{
"type": "text",
"text": serde_json::to_string(&result).unwrap_or_else(|_| "{}".into())
}],
"isError": false,
"authority": "workbook"
}))
}
/// Blocking stdio MCP loop. `tools` is a JSON array of `{name, fn, description}`.
pub fn serve_stdio(name: &str, tools: Vec<Value>) -> Result<Value, String> {
let stdin = std::io::stdin();
let mut reader = BufReader::new(stdin.lock());
loop {
let Some(msg) = read_line_message(&mut reader)? else {
break;
};
let id = msg.get("id").cloned().unwrap_or(Value::Null);
let method = msg.get("method").and_then(|v| v.as_str()).unwrap_or("");
let params = msg.get("params").cloned().unwrap_or(json!({}));
if matches!(id, Value::Null) && method.starts_with("notifications/") {
continue;
}
let reply_msg = match method {
"initialize" => rpc_result(
&id,
json!({
"protocolVersion": "2024-11-05",
"capabilities": { "tools": {} },
"serverInfo": { "name": name, "version": "0.1.0" }
}),
),
"tools/list" | "list_tools" => rpc_result(&id, tools_list(&tools)),
"tools/call" | "call_tool" => {
let tname = params.get("name").and_then(|v| v.as_str()).unwrap_or("");
let args = params.get("arguments").cloned().unwrap_or(json!({}));
match call_tool(&tools, tname, &args) {
Ok(r) => rpc_result(&id, r),
Err(e) => rpc_error(&id, -32000, &e),
}
}
"ping" => rpc_result(&id, json!({})),
"" => rpc_error(&id, -32600, "missing method"),
other => rpc_error(&id, -32601, &format!("method not found: {other}")),
};
write_message(&reply_msg)?;
}
Ok(json!({ "ok": true, "transport": "stdio" }))
}
pub fn tools_from_value(v: &Value) -> Vec<Value> {
if let Some(arr) = v.as_array() {
return arr.clone();
}
if let Value::Object(map) = v {
let mut out = Vec::new();
for (name, spec) in map {
let mut m = Map::new();
m.insert("name".into(), json!(name));
if let Some(obj) = spec.as_object() {
for (k, val) in obj {
m.insert(k.clone(), val.clone());
}
} else if let Some(s) = spec.as_str() {
m.insert("fn".into(), json!(s));
}
out.push(Value::Object(m));
}
return out;
}
Vec::new()
}
pub unsafe extern "C" fn agent_mcp_serve(
args_json: *const c_char,
out_json: *mut *mut c_char,
err_msg: *mut *mut c_char,
) -> c_int {
let args = match parse_args(args_json) {
Ok(v) => v,
Err(e) => {
set_err(err_msg, &e);
return 1;
}
};
let transport = args
.get("transport")
.and_then(|v| v.as_str())
.unwrap_or("stdio");
if transport != "stdio" {
set_err(
err_msg,
&format!("mcp serve: transport `{transport}` not implemented yet (use stdio)"),
);
return 1;
}
let name = args
.get("name")
.and_then(|v| v.as_str())
.unwrap_or("marqdo");
let tools = args
.get("tools")
.map(tools_from_value)
.unwrap_or_default();
match serve_stdio(name, tools) {
Ok(v) => {
set_out(out_json, &v.to_string());
0
}
Err(e) => {
set_err(err_msg, &e);
1
}
}
}
// Silence unused import if CStr unused in this module.
#[allow(dead_code)]
fn _unused(_: &CStr) {}
+2
View File
@@ -25,6 +25,8 @@ pulldown-cmark = "0.12"
postgres = { version = "0.19", features = ["with-serde_json-1"] }
redis = { version = "0.27", default-features = true }
ureq = { version = "=2.9.7", default-features = false, features = ["tls"] }
reqwest = { version = "0.12", default-features = false, features = ["rustls-tls", "stream", "json"] }
bytes = "1"
hmac = "0.12"
sha2 = "0.10"
hex = "0.4"
+6
View File
@@ -281,6 +281,8 @@ pub fn listen(
middleware: &Middleware,
mut site_head: Vec<crate::assets::HeadLink>,
mut icon_routes: Vec<crate::assets::IconRoute>,
proxy_routes: Vec<(String, crate::proxy::ProxyRoute)>,
invoke_routes: Vec<(String, crate::invoke::InvokeRoute)>,
) -> Result<Value, String> {
let mut form_owners = HashMap::new();
collect_page_forms(page, &mut forms, &mut form_owners);
@@ -601,6 +603,10 @@ pub fn listen(
);
}
// Streaming proxy + HTTP→## invoke (before fallback so paths are explicit).
app = crate::proxy::mount_all(app, &proxy_routes);
app = crate::invoke::mount_all(app, &invoke_routes);
app = app.fallback(get(fallback_404));
let app = crate::middleware::apply(app, middleware);
+373
View File
@@ -0,0 +1,373 @@
//! HTTP → user `##` invoke routes for `app.invoke` / `configure invoke=`.
use std::sync::Arc;
use axum::extract::{Request, State};
use axum::http::{header, HeaderValue, StatusCode};
use axum::response::{IntoResponse, Response};
use axum::routing::{get, post};
use axum::Router;
use serde_json::{json, Map, Value};
use crate::middleware::{cell_text, JsonBody};
use crate::table;
use crate::call_lib_with_args;
/// One declared invoke mount.
#[derive(Clone, Debug)]
pub struct InvokeRoute {
pub method: String,
pub fn_path: String,
pub body_mode: String,
pub return_mode: String,
}
impl InvokeRoute {
pub fn from_json(v: &Value) -> Option<Self> {
let obj = v.as_object()?;
let fn_path = obj
.get("fn")
.or_else(|| obj.get("function"))
.and_then(|x| x.as_str())?
.trim()
.to_string();
if fn_path.is_empty() {
return None;
}
let method = obj
.get("method")
.and_then(|x| x.as_str())
.unwrap_or("POST")
.to_uppercase();
let body_mode = obj
.get("body")
.and_then(|x| x.as_str())
.unwrap_or("json")
.to_ascii_lowercase();
let return_mode = obj
.get("return")
.and_then(|x| x.as_str())
.unwrap_or("json")
.to_ascii_lowercase();
Some(Self {
method,
fn_path,
body_mode,
return_mode,
})
}
}
/// Normalize invoke-routes GFM table into `app.invoke_routes` map.
pub fn invoke_routes_from_table(table_v: &Value) -> Value {
let rows = table::as_rows(table_v);
let mut routes = Map::new();
if let Some(arr) = rows.as_array() {
for row in arr {
let col = |keys: &[&str]| -> String {
keys.iter()
.find_map(|k| row.get(*k).map(cell_text))
.unwrap_or_default()
};
let path = col(&["路径", "path", "Path"])
.trim()
.trim_start_matches('/')
.to_string();
if path.is_empty() {
continue;
}
let fn_path = col(&["函数", "fn", "function", "Function"]);
if fn_path.is_empty() {
continue;
}
let method = {
let m = col(&["方法", "method", "Method"]).to_uppercase();
if m.is_empty() {
"POST".into()
} else {
m
}
};
let body = {
let b = col(&["正文", "body", "Body"]).to_ascii_lowercase();
if b.is_empty() {
"json".into()
} else {
b
}
};
let ret = {
let r = col(&["返回", "return", "Return"]).to_ascii_lowercase();
if r.is_empty() {
"json".into()
} else {
r
}
};
routes.insert(
path,
json!({
"method": method,
"fn": fn_path,
"body": body,
"return": ret,
}),
);
}
}
Value::Object(routes)
}
pub fn mount_all(
mut app: Router<Arc<crate::http::AppState>>,
routes: &[(String, InvokeRoute)],
) -> Router<Arc<crate::http::AppState>> {
for (path, route) in routes {
app = mount_one(app, path, route.clone());
}
app
}
fn mount_one(
app: Router<Arc<crate::http::AppState>>,
path: &str,
route: InvokeRoute,
) -> Router<Arc<crate::http::AppState>> {
let r = route.clone();
match route.method.as_str() {
"GET" => app.route(
path,
get(move |State(st): State<Arc<crate::http::AppState>>, req: Request| {
let r = r.clone();
async move { invoke_handler(st, r, req).await }
}),
),
_ => app.route(
path,
post(move |State(st): State<Arc<crate::http::AppState>>, req: Request| {
let r = r.clone();
async move { invoke_handler(st, r, req).await }
}),
),
}
}
async fn invoke_handler(
_st: Arc<crate::http::AppState>,
route: InvokeRoute,
req: Request,
) -> Response {
let (parts, body) = req.into_parts();
let query_map = query_to_map(parts.uri.query());
let payload = match route.body_mode.as_str() {
"query" => Value::Object(query_map.clone()),
"raw" => {
let bytes = match axum::body::to_bytes(body, 8 * 1024 * 1024).await {
Ok(b) => b,
Err(e) => {
return err_json(StatusCode::BAD_REQUEST, &format!("body: {e}"));
}
};
json!({ "payload": String::from_utf8_lossy(&bytes).to_string() })
}
"form" => {
let bytes = match axum::body::to_bytes(body, 8 * 1024 * 1024).await {
Ok(b) => b,
Err(e) => {
return err_json(StatusCode::BAD_REQUEST, &format!("body: {e}"));
}
};
form_to_map(&bytes)
}
_ => {
// json (default)
let bytes = match axum::body::to_bytes(body, 8 * 1024 * 1024).await {
Ok(b) => b,
Err(e) => {
return err_json(StatusCode::BAD_REQUEST, &format!("body: {e}"));
}
};
if bytes.is_empty() {
Value::Object(query_map.clone())
} else {
match serde_json::from_slice::<Value>(&bytes) {
Ok(Value::Object(mut m)) => {
for (k, v) in query_map {
m.entry(k).or_insert(v);
}
Value::Object(m)
}
Ok(other) => json!({ "payload": other }),
Err(e) => {
return err_json(StatusCode::BAD_REQUEST, &format!("json: {e}"));
}
}
}
}
};
// Bind JSON object keys as named args; always offer `payload` too.
let mut args = Map::new();
if let Value::Object(ref m) = payload {
for (k, v) in m {
args.insert(k.clone(), v.clone());
}
}
if !args.contains_key("payload") {
args.insert("payload".to_string(), payload.clone());
}
let fn_path = route.fn_path.clone();
let result = tokio::task::spawn_blocking(move || call_lib_with_args(&fn_path, &Value::Object(args)))
.await;
let value = match result {
Ok(Ok(v)) => v,
Ok(Err(e)) => {
let status = if e.contains("unknown library")
|| e.contains("call_lib_path: need")
|| e.contains("not allowed")
|| e.contains("no site")
{
StatusCode::BAD_REQUEST
} else {
StatusCode::INTERNAL_SERVER_ERROR
};
return err_json(status, &e);
}
Err(e) => {
return err_json(StatusCode::INTERNAL_SERVER_ERROR, &format!("join: {e}"));
}
};
match route.return_mode.as_str() {
"text" => {
if matches!(value, Value::Null) {
return err_json(StatusCode::INTERNAL_SERVER_ERROR, "null result");
}
let text = match &value {
Value::String(s) => s.clone(),
other => other.to_string(),
};
(
StatusCode::OK,
[(
header::CONTENT_TYPE,
HeaderValue::from_static("text/plain; charset=utf-8"),
)],
text,
)
.into_response()
}
"status" => {
if matches!(value, Value::Null) {
return err_json(StatusCode::INTERNAL_SERVER_ERROR, "null result");
}
StatusCode::NO_CONTENT.into_response()
}
_ => {
let body = match json_wrap_invoke_value(value) {
Ok((status, body)) => {
let mut resp = JsonBody(body).into_response();
*resp.status_mut() = status;
resp
}
Err(resp) => resp,
};
body
}
}
}
/// Map host return value to invoke JSON. `Null` must not look like success (GAP-08).
pub(crate) fn json_wrap_invoke_value(
value: Value,
) -> Result<(StatusCode, Value), Response> {
match value {
Value::Null => Err(err_json(
StatusCode::INTERNAL_SERVER_ERROR,
"null result",
)),
Value::Object(_) => Ok((StatusCode::OK, value)),
other => Ok((StatusCode::OK, json!({ "ok": true, "result": other }))),
}
}
fn query_to_map(q: Option<&str>) -> Map<String, Value> {
let mut m = Map::new();
let Some(q) = q else {
return m;
};
for pair in q.split('&') {
if pair.is_empty() {
continue;
}
let (k, v) = pair.split_once('=').unwrap_or((pair, ""));
let k = urlencoding_decode(k);
let v = urlencoding_decode(v);
if !k.is_empty() {
m.insert(k, json!(v));
}
}
m
}
fn form_to_map(bytes: &[u8]) -> Value {
let s = String::from_utf8_lossy(bytes);
Value::Object(query_to_map(Some(&s)))
}
fn urlencoding_decode(s: &str) -> String {
let mut out = String::with_capacity(s.len());
let b = s.as_bytes();
let mut i = 0;
while i < b.len() {
match b[i] {
b'+' => {
out.push(' ');
i += 1;
}
b'%' if i + 2 < b.len() => {
let h = u8::from_str_radix(std::str::from_utf8(&b[i + 1..i + 3]).unwrap_or(""), 16);
if let Ok(c) = h {
out.push(c as char);
i += 3;
} else {
out.push('%');
i += 1;
}
}
c => {
out.push(c as char);
i += 1;
}
}
}
out
}
fn err_json(status: StatusCode, msg: &str) -> Response {
(
status,
JsonBody(json!({ "ok": false, "error": msg })),
)
.into_response()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn null_result_is_not_fake_ok() {
let err = json_wrap_invoke_value(Value::Null).expect_err("null must fail");
assert_eq!(err.status(), StatusCode::INTERNAL_SERVER_ERROR);
let ok = json_wrap_invoke_value(json!({ "hits": [] })).expect("object ok");
assert_eq!(ok.0, StatusCode::OK);
let wrap = json_wrap_invoke_value(json!(1)).expect("scalar wrap");
assert_eq!(wrap.0, StatusCode::OK);
assert_eq!(wrap.1["ok"], true);
assert_eq!(wrap.1["result"], 1);
}
}
+292 -62
View File
@@ -8,9 +8,11 @@ mod driver;
mod form;
mod http;
mod cache;
mod invoke;
mod markdown;
mod middleware;
mod password;
mod proxy;
mod rate_limit;
mod render;
mod rss;
@@ -156,6 +158,17 @@ fn call_lib(path: &str) -> Result<Value, String> {
host_query_json("call_lib_path", &json!({ "path": path }))
}
/// Call `lib.member` with a JSON object of named arguments (for HTTP invoke).
pub(crate) fn call_lib_with_args(path: &str, args: &Value) -> Result<Value, String> {
host_query_json(
"call_lib_path",
&json!({
"path": path,
"args": args,
}),
)
}
/// Absolute directory of the entry `.mq.md` (falls back to process cwd).
fn entry_dir() -> PathBuf {
host_query_json("entry_dir", &json!({}))
@@ -1507,74 +1520,245 @@ web_ffi!(web_app_static, |args: &Value| {
// backed by db queries (method GET/POST, `表`=table name).
web_ffi!(web_app_middleware, |args: &Value| {
let mut app = args.get("app").cloned().unwrap_or(json!({}));
{
let obj = app
.as_object_mut()
.ok_or_else(|| "app must be a map".to_string())?;
let mw = obj
.entry("middleware".to_string())
.or_insert_with(|| json!({}));
let m = mw
.as_object_mut()
.ok_or_else(|| "app.middleware must be a map".to_string())?;
if let Some(cors) = args.get("cors") {
m.insert("cors".into(), middleware::cors_from_table(cors));
}
if let Some(security) = args.get("security") {
m.insert(
"security".into(),
middleware::security_from_table(security),
);
}
if let Some(compress) = args.get("compress") {
let on = match compress {
Value::Bool(b) => *b,
Value::String(s) => matches!(s.as_str(), "true" | "True" | "1" | "yes" | "on"),
Value::Number(n) => n.as_i64().unwrap_or(0) != 0,
_ => false,
};
m.insert("compress".into(), json!(on));
}
if let Some(al) = args.get("access_log").or_else(|| args.get("访问日志")) {
let on = match al {
Value::Bool(b) => *b,
Value::String(s) => {
matches!(s.as_str(), "true" | "True" | "1" | "yes" | "on" | "真")
}
Value::Number(n) => n.as_i64().unwrap_or(0) != 0,
_ => false,
};
m.insert("access_log".into(), json!(on));
}
if let Some(cc) = args
.get("cache_control")
.or_else(|| args.get("缓存控制"))
.and_then(|v| v.as_str())
{
let s = cc.trim();
if !s.is_empty() {
m.insert("cache_control".into(), json!(s));
}
}
if let Some(bl) = args.get("body_limit") {
let n = bl
.as_u64()
.or_else(|| bl.as_i64().map(|i| i as u64))
.unwrap_or(0);
if n > 0 {
m.insert("body_limit".into(), json!(n));
}
}
if let Some(routes) = args.get("json_routes") {
m.insert(
"json_routes".into(),
middleware::json_routes_from_table(routes),
);
}
}
if let Some(proxy) = args
.get("proxy")
.or_else(|| args.get("proxy_routes"))
.or_else(|| args.get("代理"))
.filter(|v| !v.is_null())
{
let incoming = proxy::proxy_routes_from_table(proxy);
let obj_app = app
.as_object_mut()
.ok_or_else(|| "app must be a map".to_string())?;
let mut existing = obj_app
.get("proxy_routes")
.and_then(|v| v.as_object())
.cloned()
.unwrap_or_default();
if let Some(map) = incoming.as_object() {
for (k, v) in map {
existing.insert(k.clone(), v.clone());
}
}
obj_app.insert("proxy_routes".into(), Value::Object(existing));
}
if let Some(inv) = args
.get("invoke")
.or_else(|| args.get("invoke_routes"))
.or_else(|| args.get("调用"))
.filter(|v| !v.is_null())
{
let incoming = invoke::invoke_routes_from_table(inv);
let obj_app = app
.as_object_mut()
.ok_or_else(|| "app must be a map".to_string())?;
let mut existing = obj_app
.get("invoke_routes")
.and_then(|v| v.as_object())
.cloned()
.unwrap_or_default();
if let Some(map) = incoming.as_object() {
for (k, v) in map {
existing.insert(k.clone(), v.clone());
}
}
obj_app.insert("invoke_routes".into(), Value::Object(existing));
}
Ok(app)
});
web_ffi!(web_app_proxy, |args: &Value| {
let mut app = args.get("app").cloned().unwrap_or(json!({}));
let path = normalize_route_path_in_app(
arg_str(args, "path").or_else(|_| arg_str(args, "路径"))?,
&app,
)?;
let upstream = arg_str(args, "upstream")
.or_else(|_| arg_str(args, "上游"))?
.to_string();
if upstream.trim().is_empty() {
return Err("proxy requires `upstream`".into());
}
let stream = match args
.get("stream")
.or_else(|| args.get("流式"))
.cloned()
.unwrap_or(json!(true))
{
Value::Bool(b) => b,
Value::String(s) => matches!(
s.as_str(),
"true" | "True" | "1" | "yes" | "on" | "真"
),
Value::Number(n) => n.as_i64().unwrap_or(0) != 0,
_ => true,
};
let strip_prefix = arg_str_opt(args, "strip_prefix")
.or_else(|| arg_str_opt(args, "去前缀"))
.unwrap_or("")
.to_string();
let methods = match args.get("methods").or_else(|| args.get("方法")) {
Some(Value::Array(arr)) => arr
.iter()
.filter_map(|v| v.as_str().map(|s| s.to_uppercase()))
.collect::<Vec<_>>(),
Some(Value::String(s)) => s
.split(',')
.map(|m| m.trim().to_uppercase())
.filter(|m| !m.is_empty())
.collect(),
_ => vec!["POST".into()],
};
let headers_from_env = arg_str_opt(args, "headers_from_env")
.or_else(|| arg_str_opt(args, "环境头"))
.unwrap_or("")
.to_string();
let timeout_ms = args
.get("timeout_ms")
.or_else(|| args.get("超时"))
.and_then(|v| v.as_u64().or_else(|| v.as_i64().map(|i| i as u64)))
.unwrap_or(120_000);
let obj = app
.as_object_mut()
.ok_or_else(|| "app must be a map".to_string())?;
let mw = obj
.entry("middleware".to_string())
.or_insert_with(|| json!({}));
let m = mw
.as_object_mut()
.ok_or_else(|| "app.middleware must be a map".to_string())?;
let mut routes = obj
.get("proxy_routes")
.and_then(|v| v.as_object())
.cloned()
.unwrap_or_default();
let key = path.trim_start_matches('/').to_string();
routes.insert(
key,
json!({
"upstream": upstream,
"stream": stream,
"strip_prefix": strip_prefix,
"methods": methods,
"headers_from_env": headers_from_env,
"timeout_ms": timeout_ms,
}),
);
obj.insert("proxy_routes".into(), Value::Object(routes));
Ok(app)
});
if let Some(cors) = args.get("cors") {
m.insert(
"cors".into(),
middleware::cors_from_table(cors),
);
web_ffi!(web_app_invoke, |args: &Value| {
let mut app = args.get("app").cloned().unwrap_or(json!({}));
let path = normalize_route_path_in_app(
arg_str(args, "path").or_else(|_| arg_str(args, "路径"))?,
&app,
)?;
let fn_path = arg_str(args, "fn")
.or_else(|_| arg_str(args, "function"))
.or_else(|_| arg_str(args, "函数"))?
.to_string();
if fn_path.trim().is_empty() {
return Err("invoke requires `fn`".into());
}
if let Some(security) = args.get("security") {
m.insert(
"security".into(),
middleware::security_from_table(security),
);
}
if let Some(compress) = args.get("compress") {
let on = match compress {
Value::Bool(b) => *b,
Value::String(s) => matches!(s.as_str(), "true" | "True" | "1" | "yes" | "on"),
Value::Number(n) => n.as_i64().unwrap_or(0) != 0,
_ => false,
};
m.insert("compress".into(), json!(on));
}
if let Some(al) = args
.get("access_log")
.or_else(|| args.get("访问日志"))
{
let on = match al {
Value::Bool(b) => *b,
Value::String(s) => matches!(s.as_str(), "true" | "True" | "1" | "yes" | "on" | "真"),
Value::Number(n) => n.as_i64().unwrap_or(0) != 0,
_ => false,
};
m.insert("access_log".into(), json!(on));
}
if let Some(cc) = args
.get("cache_control")
.or_else(|| args.get("缓存控制"))
.and_then(|v| v.as_str())
{
let s = cc.trim();
if !s.is_empty() {
m.insert("cache_control".into(), json!(s));
}
}
if let Some(bl) = args.get("body_limit") {
let n = bl
.as_u64()
.or_else(|| bl.as_i64().map(|i| i as u64))
.unwrap_or(0);
if n > 0 {
m.insert("body_limit".into(), json!(n));
}
}
if let Some(routes) = args.get("json_routes") {
m.insert(
"json_routes".into(),
middleware::json_routes_from_table(routes),
// Optional allowlist: only `lib.member` form for now (no arbitrary file paths).
if !fn_path.contains('.') || fn_path.contains('/') || fn_path.contains('#') {
return Err(
"invoke `fn` must be `lib.member` (imported on the entry module)".into(),
);
}
let method = arg_str_opt(args, "method")
.or_else(|| arg_str_opt(args, "方法"))
.unwrap_or("POST")
.to_uppercase();
let body = arg_str_opt(args, "body")
.or_else(|| arg_str_opt(args, "正文"))
.unwrap_or("json")
.to_ascii_lowercase();
let ret = arg_str_opt(args, "return")
.or_else(|| arg_str_opt(args, "返回"))
.unwrap_or("json")
.to_ascii_lowercase();
let obj = app
.as_object_mut()
.ok_or_else(|| "app must be a map".to_string())?;
let mut routes = obj
.get("invoke_routes")
.and_then(|v| v.as_object())
.cloned()
.unwrap_or_default();
let key = path.trim_start_matches('/').to_string();
routes.insert(
key,
json!({
"method": method,
"fn": fn_path,
"body": body,
"return": ret,
}),
);
obj.insert("invoke_routes".into(), Value::Object(routes));
Ok(app)
});
@@ -1737,6 +1921,8 @@ web_ffi!(web_listen, |args: &Value| {
page_500,
gates,
gallery_routes,
proxy_routes,
invoke_routes,
middleware,
cookie_secure,
site_head,
@@ -1793,6 +1979,8 @@ web_ffi!(web_listen, |args: &Value| {
None,
Vec::<http::Gate>::new(),
HashMap::new(),
Vec::<(String, crate::proxy::ProxyRoute)>::new(),
Vec::<(String, crate::invoke::InvokeRoute)>::new(),
middleware::Middleware::default(),
cookie_secure,
Vec::<crate::assets::HeadLink>::new(),
@@ -1943,6 +2131,34 @@ web_ffi!(web_listen, |args: &Value| {
gallery_routes.insert(k.clone(), v.clone());
}
}
let mut proxy_routes: Vec<(String, proxy::ProxyRoute)> = Vec::new();
if let Some(obj) = app.get("proxy_routes").and_then(|v| v.as_object()) {
for (k, v) in obj {
if let Some(r) = proxy::ProxyRoute::from_json(v) {
let path = if k.starts_with('/') {
k.clone()
} else {
format!("/{k}")
};
proxy_routes.push((path, r));
}
}
proxy_routes.sort_by(|a, b| a.0.cmp(&b.0));
}
let mut invoke_routes: Vec<(String, invoke::InvokeRoute)> = Vec::new();
if let Some(obj) = app.get("invoke_routes").and_then(|v| v.as_object()) {
for (k, v) in obj {
if let Some(r) = invoke::InvokeRoute::from_json(v) {
let path = if k.starts_with('/') {
k.clone()
} else {
format!("/{k}")
};
invoke_routes.push((path, r));
}
}
invoke_routes.sort_by(|a, b| a.0.cmp(&b.0));
}
if app.get("tls_cert").is_some() || app.get("tls_key").is_some() {
eprintln!(
"marqdo web: in-process TLS is not enabled; terminate HTTPS at a reverse proxy (nginx/caddy) and set cookie_secure=True"
@@ -2026,6 +2242,8 @@ web_ffi!(web_listen, |args: &Value| {
page_500,
gates,
gallery_routes,
proxy_routes,
invoke_routes,
middleware,
cookie_secure,
site_head,
@@ -2141,6 +2359,8 @@ web_ffi!(web_listen, |args: &Value| {
&middleware,
site_head,
icon_routes,
proxy_routes,
invoke_routes,
)
});
@@ -2369,9 +2589,19 @@ pub unsafe extern "C" fn marqdo_plugin_init(host: *const MarqdoHostApi) -> c_int
),
(
"web_app_middleware",
"app,cors,security,compress,body_limit,json_routes,access_log,cache_control",
"app,cors,security,compress,body_limit,json_routes,access_log,cache_control,proxy,invoke",
web_app_middleware as PluginFn,
),
(
"web_app_proxy",
"app,path,upstream,stream,strip_prefix,methods,headers_from_env,timeout_ms",
web_app_proxy as PluginFn,
),
(
"web_app_invoke",
"app,path,method,fn,body,return",
web_app_invoke as PluginFn,
),
("web_form_new", "table,action,id", web_form_new as PluginFn),
("web_form_fields", "form,fields", web_form_fields as PluginFn),
("web_form_rules", "form,rules", web_form_rules as PluginFn),
+1 -1
View File
@@ -24,7 +24,7 @@ use tower_http::set_header::SetResponseHeaderLayer;
use crate::db;
/// Render a cell value as text (strings stay, numbers/bools stringify).
fn cell_text(v: &Value) -> String {
pub fn cell_text(v: &Value) -> String {
match v {
Value::Null => String::new(),
Value::String(s) => s.clone(),
+471
View File
@@ -0,0 +1,471 @@
//! Streaming reverse proxy routes for `app.proxy` / `configure proxy=`.
//!
//! Browser → Marqdo (same origin) → upstream, piping response bodies
//! (including `text/event-stream`) without buffering the full stream.
use std::sync::Arc;
use std::time::Duration;
use axum::body::Body;
use axum::extract::{Request, State};
use axum::http::{header, HeaderMap, HeaderName, HeaderValue, StatusCode};
use axum::response::{IntoResponse, Response};
use axum::routing::{MethodFilter, MethodRouter};
use axum::Router;
use futures_util::TryStreamExt;
use serde_json::{json, Map, Value};
use crate::middleware::cell_text;
use crate::table;
/// One declared proxy mount.
#[derive(Clone, Debug)]
pub struct ProxyRoute {
pub upstream: String,
pub stream: bool,
pub strip_prefix: String,
pub methods: Vec<String>,
/// `ENV_NAME=Header-Name` pairs injected on the upstream request.
pub headers_from_env: Vec<(String, String)>,
pub timeout_ms: u64,
}
impl ProxyRoute {
pub fn from_json(v: &Value) -> Option<Self> {
let obj = v.as_object()?;
let upstream = obj.get("upstream")?.as_str()?.trim().to_string();
if upstream.is_empty() {
return None;
}
let stream = obj
.get("stream")
.and_then(|x| x.as_bool())
.unwrap_or(true);
let strip_prefix = obj
.get("strip_prefix")
.and_then(|x| x.as_str())
.unwrap_or("")
.to_string();
let methods = obj
.get("methods")
.and_then(|x| x.as_array())
.map(|arr| {
arr.iter()
.filter_map(|m| m.as_str().map(|s| s.to_uppercase()))
.filter(|s| !s.is_empty())
.collect::<Vec<_>>()
})
.filter(|v| !v.is_empty())
.unwrap_or_else(|| vec!["POST".into()]);
let headers_from_env = parse_headers_from_env_value(obj.get("headers_from_env"));
let timeout_ms = obj
.get("timeout_ms")
.and_then(|x| x.as_u64().or_else(|| x.as_i64().map(|i| i as u64)))
.unwrap_or(120_000)
.max(1);
Some(Self {
upstream,
stream,
strip_prefix,
methods,
headers_from_env,
timeout_ms,
})
}
}
fn parse_headers_from_env_value(v: Option<&Value>) -> Vec<(String, String)> {
let Some(v) = v else {
return Vec::new();
};
match v {
Value::String(s) => parse_headers_from_env_str(s),
Value::Array(arr) => arr
.iter()
.filter_map(|x| x.as_str())
.flat_map(parse_headers_from_env_str)
.collect(),
_ => Vec::new(),
}
}
fn parse_headers_from_env_str(s: &str) -> Vec<(String, String)> {
s.split(|c| c == ',' || c == ';')
.filter_map(|part| {
let part = part.trim();
if part.is_empty() {
return None;
}
let (env, hdr) = part.split_once('=')?;
let env = env.trim();
let hdr = hdr.trim();
if env.is_empty() || hdr.is_empty() {
return None;
}
Some((env.to_string(), hdr.to_string()))
})
.collect()
}
/// Expand `$VAR` / `${VAR}` in upstream URL strings.
pub fn expand_env(s: &str) -> String {
let mut out = String::with_capacity(s.len());
let bytes = s.as_bytes();
let mut i = 0;
while i < bytes.len() {
if bytes[i] == b'$' {
if i + 1 < bytes.len() && bytes[i + 1] == b'{' {
if let Some(end) = s[i + 2..].find('}') {
let name = &s[i + 2..i + 2 + end];
out.push_str(&std::env::var(name).unwrap_or_default());
i += 3 + end;
continue;
}
} else {
let start = i + 1;
let mut end = start;
while end < bytes.len()
&& (bytes[end].is_ascii_alphanumeric() || bytes[end] == b'_')
{
end += 1;
}
if end > start {
let name = &s[start..end];
out.push_str(&std::env::var(name).unwrap_or_default());
i = end;
continue;
}
}
}
out.push(bytes[i] as char);
i += 1;
}
out
}
/// Normalize a proxy-routes GFM table into `app.proxy_routes` map shape.
pub fn proxy_routes_from_table(table_v: &Value) -> Value {
let rows = table::as_rows(table_v);
let mut routes = Map::new();
if let Some(arr) = rows.as_array() {
for row in arr {
let col = |keys: &[&str]| -> String {
keys.iter()
.find_map(|k| row.get(*k).map(cell_text))
.unwrap_or_default()
};
let path = normalize_path(&col(&["路径", "path", "Path"]));
if path.is_empty() || path == "/" {
continue;
}
let upstream = col(&["上游", "upstream", "Upstream"]);
if upstream.is_empty() {
continue;
}
let stream = {
let s = col(&["流式", "stream", "Stream"]);
if s.is_empty() {
true
} else {
matches!(
s.to_ascii_lowercase().as_str(),
"true" | "1" | "yes" | "on" | "真"
)
}
};
let strip_prefix = col(&["去前缀", "strip_prefix", "Strip-Prefix"]);
let methods_raw = col(&["方法", "methods", "Methods"]);
let methods: Vec<String> = if methods_raw.is_empty() {
vec!["POST".into()]
} else {
methods_raw
.split(',')
.map(|m| m.trim().to_uppercase())
.filter(|m| !m.is_empty())
.collect()
};
let headers_from_env = col(&[
"环境头",
"headers_from_env",
"Headers-From-Env",
]);
let timeout_ms = col(&["超时", "timeout_ms", "Timeout"])
.parse::<u64>()
.unwrap_or(120_000);
routes.insert(
path.trim_start_matches('/').to_string(),
json!({
"upstream": upstream,
"stream": stream,
"strip_prefix": strip_prefix,
"methods": methods,
"headers_from_env": headers_from_env,
"timeout_ms": timeout_ms,
}),
);
}
}
Value::Object(routes)
}
fn normalize_path(raw: &str) -> String {
let s = raw.trim();
if s.is_empty() {
return String::new();
}
if s.starts_with('/') {
s.to_string()
} else {
format!("/{s}")
}
}
/// Mount all proxy routes onto the router.
pub fn mount_all(
mut app: Router<Arc<crate::http::AppState>>,
routes: &[(String, ProxyRoute)],
) -> Router<Arc<crate::http::AppState>> {
for (path, route) in routes {
app = mount_one(app, path, route.clone());
}
app
}
fn mount_one(
app: Router<Arc<crate::http::AppState>>,
path: &str,
route: ProxyRoute,
) -> Router<Arc<crate::http::AppState>> {
let mut filter: Option<MethodFilter> = None;
for m in &route.methods {
let f = match m.as_str() {
"GET" => MethodFilter::GET,
"POST" => MethodFilter::POST,
"PUT" => MethodFilter::PUT,
"PATCH" => MethodFilter::PATCH,
"DELETE" => MethodFilter::DELETE,
"HEAD" => MethodFilter::HEAD,
"OPTIONS" => MethodFilter::OPTIONS,
_ => continue,
};
filter = Some(match filter {
None => f,
Some(acc) => acc.or(f),
});
}
let filter = filter.unwrap_or(MethodFilter::POST);
let r = route.clone();
let handler = MethodRouter::new().on(filter, move |State(_st): State<Arc<crate::http::AppState>>, req: Request| {
let r = r.clone();
async move { proxy_handler(r, req).await }
});
app.route(path, handler)
}
async fn proxy_handler(route: ProxyRoute, req: Request) -> Response {
let method = req.method().clone();
let uri = req.uri().clone();
let in_headers = req.headers().clone();
let body_bytes = match axum::body::to_bytes(req.into_body(), 64 * 1024 * 1024).await {
Ok(b) => b,
Err(e) => {
return (
StatusCode::BAD_REQUEST,
format!("read body: {e}"),
)
.into_response();
}
};
let target = match build_upstream_url(&route, uri.path(), uri.query()) {
Ok(u) => u,
Err(e) => return (StatusCode::BAD_GATEWAY, e).into_response(),
};
let client = match reqwest::Client::builder()
.timeout(Duration::from_millis(route.timeout_ms))
.build()
{
Ok(c) => c,
Err(e) => {
return (
StatusCode::INTERNAL_SERVER_ERROR,
format!("client: {e}"),
)
.into_response();
}
};
let mut builder = client.request(
reqwest::Method::from_bytes(method.as_str().as_bytes()).unwrap_or(reqwest::Method::POST),
&target,
);
// Forward selected inbound headers (skip hop-by-hop).
for (name, value) in in_headers.iter() {
let lname = name.as_str().to_ascii_lowercase();
if is_hop_by_hop(&lname) || lname == "host" {
continue;
}
if let Ok(v) = value.to_str() {
builder = builder.header(name.as_str(), v);
}
}
// Inject secrets from env (never from the browser).
for (env_name, header_name) in &route.headers_from_env {
if let Ok(val) = std::env::var(env_name) {
if !val.is_empty() {
// Common pattern: Authorization Bearer
let hdr_val = if header_name.eq_ignore_ascii_case("authorization")
&& !val.to_ascii_lowercase().starts_with("bearer ")
{
format!("Bearer {val}")
} else {
val
};
builder = builder.header(header_name.as_str(), hdr_val);
}
}
}
if !body_bytes.is_empty() {
builder = builder.body(body_bytes);
}
let upstream = match builder.send().await {
Ok(r) => r,
Err(e) => {
return (StatusCode::BAD_GATEWAY, format!("upstream: {e}")).into_response();
}
};
let status = StatusCode::from_u16(upstream.status().as_u16()).unwrap_or(StatusCode::BAD_GATEWAY);
let mut out_headers = HeaderMap::new();
for (name, value) in upstream.headers().iter() {
let lname = name.as_str().to_ascii_lowercase();
if is_hop_by_hop(&lname) {
continue;
}
if let (Ok(n), Ok(v)) = (
HeaderName::from_bytes(name.as_str().as_bytes()),
HeaderValue::from_bytes(value.as_bytes()),
) {
out_headers.append(n, v);
}
}
// Ensure SSE content-type is preserved if present.
if !out_headers.contains_key(header::CONTENT_TYPE) {
if let Some(ct) = upstream.headers().get(reqwest::header::CONTENT_TYPE) {
if let Ok(v) = HeaderValue::from_bytes(ct.as_bytes()) {
out_headers.insert(header::CONTENT_TYPE, v);
}
}
}
if route.stream {
// Industry SSE / reverse-proxy practice: disable buffering at CDN/Nginx.
out_headers.insert(
header::CACHE_CONTROL,
HeaderValue::from_static("no-cache, no-transform"),
);
out_headers.insert(
HeaderName::from_static("x-accel-buffering"),
HeaderValue::from_static("no"),
);
let stream = upstream.bytes_stream().map_err(|e| {
std::io::Error::new(std::io::ErrorKind::Other, e)
});
let body = Body::from_stream(stream);
let mut resp = Response::new(body);
*resp.status_mut() = status;
*resp.headers_mut() = out_headers;
resp
} else {
let bytes = match upstream.bytes().await {
Ok(b) => b,
Err(e) => {
return (StatusCode::BAD_GATEWAY, format!("upstream body: {e}")).into_response();
}
};
let mut resp = Response::new(Body::from(bytes));
*resp.status_mut() = status;
*resp.headers_mut() = out_headers;
resp
}
}
fn build_upstream_url(
route: &ProxyRoute,
req_path: &str,
query: Option<&str>,
) -> Result<String, String> {
let base = expand_env(&route.upstream);
if base.is_empty() {
return Err("proxy upstream empty after env expand".into());
}
// If upstream already looks like a full URL with path (contains scheme and
// more than origin), and strip_prefix empties the remainder, use base as-is.
let mut path = req_path.to_string();
let strip = route.strip_prefix.trim();
if !strip.is_empty() {
let strip_n = normalize_path(strip);
if path.starts_with(&strip_n) {
path = path[strip_n.len()..].to_string();
if path.is_empty() {
path = "/".into();
} else if !path.starts_with('/') {
path = format!("/{path}");
}
}
}
let url = if base.contains("://") {
let trimmed = base.trim_end_matches('/');
// If base has no path beyond origin, append path; if base includes a
// path suffix, append remaining path.
if let Some(idx) = trimmed.find("://") {
let after = &trimmed[idx + 3..];
if after.contains('/') {
// base includes path — append only if remaining path is not "/"
if path == "/" {
trimmed.to_string()
} else {
format!("{trimmed}{path}")
}
} else {
format!("{trimmed}{path}")
}
} else {
format!("{trimmed}{path}")
}
} else {
return Err(format!("proxy upstream must be absolute URL, got `{base}`"));
};
Ok(match query {
Some(q) if !q.is_empty() => {
if url.contains('?') {
format!("{url}&{q}")
} else {
format!("{url}?{q}")
}
}
_ => url,
})
}
fn is_hop_by_hop(name: &str) -> bool {
matches!(
name,
"connection"
| "keep-alive"
| "proxy-authenticate"
| "proxy-authorization"
| "te"
| "trailers"
| "transfer-encoding"
| "upgrade"
| "content-length"
)
}
+19
View File
@@ -0,0 +1,19 @@
---
title: Mid stdlib — datetime and url
description: Calendar moments + URL/query parse (Mid2 M7)
import dt:lib/datetime.mq.md
import url:lib/url.mq.md
---
# Time and addresses without plugins
Moments are maps `{unix, zone, iso}`. Parse HTTPS URLs and round-trip query strings in `lib/url` (segment encoding stays in `net.url_encode`).
# main
*epoch = > dt.from_unix unix=0 zone="Asia/Shanghai"*
> print text=`epoch`[^iso]
*u = > url.parse text="https://example.com/path?q=1#top"*
> print text=`u`[^host]
> print text=`u`[^query]
@@ -0,0 +1,19 @@
---
title: 中层标准库 — 日期时间与地址
description: 日历时刻 + URL/查询串(Mid2 M7)
导入 日时:lib/日期时间.mq.md
导入 址:lib/地址.mq.md
---
# 无需插件的时间与地址
时刻是 map `{unix, zone, iso}`。用 `地址` 拆 HTTPS URL 与查询串。
# main
*epoch = > 日时.自戳 秒=0 区="Asia/Shanghai"*
> print text=`epoch`[^iso]
*u = > 址.解析 内容="https://example.com/path?q=1"*
> print text=`u`[^host]
> print text=`u`[^query]
@@ -0,0 +1,24 @@
---
title: Mid stdlib — toml, html, base32
description: Config TOML + HTML escape + base32 (Mid2 M8)
import toml:lib/toml.mq.md
import html:lib/html.mq.md
import enc:lib/encoding.mq.md
---
# Config and safe text without plugins
Parse a TOML subset for tools, escape HTML before assembly, encode with base32.
# main
*cfg = > toml.parse text="app = \"demo\"\n[server]\nport = 8080\n"*
> print text=`cfg`[^app]
*srv = `cfg`[^server]*
> print text=`srv`[^port]
*safe = > html.escape text="<tag>"*
> print text=`safe`
*b = > enc.base32_encode text="ok"*
> print text=`b`
@@ -0,0 +1,20 @@
---
title: 中层标准库 — 配置表、超文本、基32
description: TOML 配置 + HTML 转义 + base32(Mid2 M8)
导入 配置:lib/配置表.mq.md
导入 超:lib/超文本.mq.md
导入 编:lib/编码.mq.md
---
# 无需插件的配置与安全字面
# main
*cfg = > 配置.解析 内容="app = \"演示\"\n"*
> print text=`cfg`[^app]
*safe = > 超.转义 内容="<tag>"*
> print text=`safe`
*b = > 编.基32编码 内容="ok"*
> print text=`b`
+24
View File
@@ -0,0 +1,24 @@
---
title: Mid stdlib — fs make_dirs, walk, stat
description: Recursive dirs, walk, and file metadata (Mid2 M9)
import fs:lib/fs.mq.md
---
# Practical filesystem without plugins
`make_dirs` / `remove_tree` / `stat` / `walk` thicken `lib/fs` for scripts and tools.
# main
> fs.make_dirs path="m9-demo/sub"
> fs.write_text path="m9-demo/readme.txt" text="ok"
*st = > fs.stat path="m9-demo/readme.txt"*
> print text=`st`[^size]
*w = > fs.walk path="m9-demo"*
*n = > len `w`*
> print text=`n`
> fs.remove_tree path="m9-demo"
+22
View File
@@ -0,0 +1,22 @@
---
title: 中层标准库 — 文件目录树与状态
description: 递归目录、遍历与元数据(Mid2 M9)
导入 文件:lib/文件.mq.md
---
# 无需插件的实用文件系统
# main
> 文件.建目录树 路径="m9-演示/子"
> 文件.写文本 路径="m9-演示/说明.txt" 内容="好"
*st = > 文件.状态 路径="m9-演示/说明.txt"*
> print text=`st`[^size]
*w = > 文件.遍历 路径="m9-演示"*
*n = > 长度 `w`*
> print text=`n`
> 文件.删目录树 路径="m9-演示"
+31
View File
@@ -0,0 +1,31 @@
---
title: Mid stdlib — stats and log fields
description: Thin mean/median/stdev + structured log fields (Mid2 M10)
import stats:lib/stats.mq.md
import log:lib/log.mq.md
---
# Closing Mid2 sugar
Light statistics and optional `fields=` on log lines — no remote sinks, no dataframes.
# main
`xs` =
| |
|---|
| 10 |
| 20 |
| 30 |
*m = > stats.mean list=`xs`*
> print text=`m`
`fld` =
| step |
|------|
| done |
> log.info text="mid2-m10" fields=`fld`
+29
View File
@@ -0,0 +1,29 @@
---
title: 中层标准库 — 统计与日志字段
description: 薄均值/中位数/标准差 + 日志 fields(Mid2 M10)
导入 统计:lib/统计.mq.md
导入 日志:lib/日志.mq.md
---
# Mid2 收口糖
# main
`xs` =
| |
|---|
| 10 |
| 20 |
| 30 |
*m = > 统计.均值 列表=`xs`*
> print text=`m`
`fld` =
| 步骤 | 状态 |
|------|------|
| 完成 | ok |
> 日志.信息 内容="mid2-m10" 字段=`fld`
+1
View File
@@ -87,6 +87,7 @@ pub fn canonical_param(builtin: &str, param: &str) -> String {
("host_dotenv_load" | "load_dotenv", "path" | "路径") => "path".into(),
("host_exec" | "exec", "cmd" | "命令") => "cmd".into(),
("host_exec" | "exec", "args" | "参数") => "args".into(),
("host_exec" | "exec", "capture" | "捕获") => "capture".into(),
(
"host_stream_publish" | "stream_publish",
"event" | "事件",
+302
View File
@@ -0,0 +1,302 @@
//! Calendar / timezone subset for Mid2 M7 (`lib/datetime`).
//!
//! Moments are plain maps: `{unix, zone, iso}`. Zone is a fixed offset (`+08:00`)
//! or alias (`UTC`); a few fixed-offset IANA names are accepted. Full tzdb is out of scope.
use chrono::{DateTime, Duration, FixedOffset, NaiveDate, NaiveDateTime, TimeZone, Utc};
use crate::value::Value;
fn as_i64(v: &Value, label: &str) -> Result<i64, String> {
match v {
Value::Int(n) => Ok(*n),
_ => Err(format!("{label} must be int")),
}
}
fn as_text<'a>(v: &'a Value, label: &str) -> Result<&'a str, String> {
match v {
Value::Text(s) => Ok(s.as_str()),
_ => Err(format!("{label} must be text")),
}
}
fn optional_i64(v: Option<&Value>, label: &str) -> Result<i64, String> {
match v {
None | Some(Value::None) => Ok(0),
Some(x) => as_i64(x, label),
}
}
fn format_offset(off: FixedOffset) -> String {
let secs = off.local_minus_utc();
let sign = if secs >= 0 { '+' } else { '-' };
let abs = secs.abs();
let h = abs / 3600;
let m = (abs % 3600) / 60;
format!("{sign}{h:02}:{m:02}")
}
fn parse_zone(s: &str) -> Result<FixedOffset, String> {
let s = s.trim();
if s.is_empty() || s.eq_ignore_ascii_case("utc") || s.eq_ignore_ascii_case("gmt") || s == "Z" {
return FixedOffset::east_opt(0).ok_or_else(|| "invalid UTC offset".into());
}
// Common fixed-offset IANA aliases (no DST).
let aliased = match s {
"Asia/Shanghai" | "Asia/Hong_Kong" | "Asia/Taipei" | "Asia/Singapore" => Some(8 * 3600),
"Asia/Tokyo" | "Asia/Seoul" => Some(9 * 3600),
"Asia/Kolkata" | "Asia/Calcutta" => Some(5 * 3600 + 30 * 60),
"Europe/Moscow" => Some(3 * 3600),
"America/Phoenix" => Some(-7 * 3600),
"Pacific/Honolulu" => Some(-10 * 3600),
_ => None,
};
if let Some(secs) = aliased {
return FixedOffset::east_opt(secs).ok_or_else(|| format!("invalid zone: {s}"));
}
let (sign, rest) = if let Some(r) = s.strip_prefix('+') {
(1i32, r)
} else if let Some(r) = s.strip_prefix('-') {
(-1i32, r)
} else {
return Err(format!(
"unknown zone {s:?}; use +HH:MM / -HH:MM / UTC or a fixed-offset alias (e.g. Asia/Shanghai)"
));
};
let (hh, mm) = if let Some((h, m)) = rest.split_once(':') {
(
h.parse::<i32>()
.map_err(|_| format!("bad zone hour in {s}"))?,
m.parse::<i32>()
.map_err(|_| format!("bad zone minute in {s}"))?,
)
} else if rest.len() == 4 && rest.chars().all(|c| c.is_ascii_digit()) {
(
rest[..2]
.parse::<i32>()
.map_err(|_| format!("bad zone hour in {s}"))?,
rest[2..]
.parse::<i32>()
.map_err(|_| format!("bad zone minute in {s}"))?,
)
} else if rest.len() == 2 && rest.chars().all(|c| c.is_ascii_digit()) {
(
rest.parse::<i32>()
.map_err(|_| format!("bad zone hour in {s}"))?,
0,
)
} else {
return Err(format!("bad zone offset {s}"));
};
if !(0..=14).contains(&hh) || !(0..60).contains(&mm) {
return Err(format!("zone offset out of range: {s}"));
}
let secs = sign * (hh * 3600 + mm * 60);
FixedOffset::east_opt(secs).ok_or_else(|| format!("invalid zone offset: {s}"))
}
fn moment(unix: i64, zone: FixedOffset) -> Result<Value, String> {
let dt = zone
.timestamp_opt(unix, 0)
.single()
.ok_or_else(|| format!("invalid unix timestamp: {unix}"))?;
Ok(Value::Map(vec![
("unix".into(), Value::Int(unix)),
("zone".into(), Value::Text(format_offset(zone))),
(
"iso".into(),
Value::Text(dt.to_rfc3339_opts(chrono::SecondsFormat::Secs, true)),
),
]))
}
fn zone_of_moment(m: &[(String, Value)]) -> Result<FixedOffset, String> {
for (k, v) in m {
if k == "zone" {
return parse_zone(as_text(v, "zone")?);
}
}
Ok(FixedOffset::east_opt(0).unwrap())
}
fn unix_of_value(v: &Value) -> Result<(i64, FixedOffset), String> {
match v {
Value::Int(n) => Ok((*n, FixedOffset::east_opt(0).unwrap())),
Value::Map(entries) => {
let mut unix = None;
for (k, val) in entries {
if k == "unix" {
unix = Some(as_i64(val, "unix")?);
}
}
let unix = unix.ok_or_else(|| "datetime map needs unix".to_string())?;
Ok((unix, zone_of_moment(entries)?))
}
Value::Text(s) => {
// Treat bare ISO/RFC3339 text as a moment.
let parsed = parse_text(s, None)?;
unix_of_value(&parsed)
}
_ => Err("datetime needs int unix, moment map, or ISO text".into()),
}
}
pub fn now() -> Result<Value, String> {
#[cfg(target_arch = "wasm32")]
{
return Err("datetime.now unavailable in browser wasm (no clock bridge yet)".into());
}
#[cfg(not(target_arch = "wasm32"))]
{
let secs = Utc::now().timestamp();
moment(secs, FixedOffset::east_opt(0).unwrap())
}
}
pub fn from_unix(unix: &Value, zone: Option<&Value>) -> Result<Value, String> {
let secs = as_i64(unix, "unix")?;
let off = match zone {
None | Some(Value::None) => FixedOffset::east_opt(0).unwrap(),
Some(z) => parse_zone(as_text(z, "zone")?)?,
};
moment(secs, off)
}
pub fn to_unix(dt: &Value) -> Result<Value, String> {
let (secs, _) = unix_of_value(dt)?;
Ok(Value::Int(secs))
}
fn parse_text(text: &str, pattern: Option<&str>) -> Result<Value, String> {
if let Some(pat) = pattern {
if pat.eq_ignore_ascii_case("rfc3339") || pat.eq_ignore_ascii_case("iso") {
return parse_rfc3339(text);
}
if let Ok(dt) = DateTime::parse_from_str(text, pat) {
return moment(dt.timestamp(), *dt.offset());
}
let naive = NaiveDateTime::parse_from_str(text, pat)
.or_else(|_| {
NaiveDate::parse_from_str(text, pat).map(|d| d.and_hms_opt(0, 0, 0).unwrap())
})
.map_err(|e| format!("datetime.parse: {e}"))?;
return moment(naive.and_utc().timestamp(), FixedOffset::east_opt(0).unwrap());
}
if let Ok(v) = parse_rfc3339(text) {
return Ok(v);
}
if let Ok(dt) = NaiveDateTime::parse_from_str(text, "%Y-%m-%d %H:%M:%S") {
return moment(dt.and_utc().timestamp(), FixedOffset::east_opt(0).unwrap());
}
if let Ok(dt) = NaiveDateTime::parse_from_str(text, "%Y-%m-%dT%H:%M:%S") {
return moment(dt.and_utc().timestamp(), FixedOffset::east_opt(0).unwrap());
}
if let Ok(d) = NaiveDate::parse_from_str(text, "%Y-%m-%d") {
let dt = d.and_hms_opt(0, 0, 0).unwrap();
return moment(dt.and_utc().timestamp(), FixedOffset::east_opt(0).unwrap());
}
Err(format!(
"datetime.parse: unrecognized {text:?}; pass pattern= or use RFC3339 / YYYY-MM-DD"
))
}
fn parse_rfc3339(text: &str) -> Result<Value, String> {
let dt = DateTime::parse_from_rfc3339(text).map_err(|e| format!("datetime.parse: {e}"))?;
moment(dt.timestamp(), *dt.offset())
}
pub fn parse(text: &Value, pattern: Option<&Value>) -> Result<Value, String> {
let t = as_text(text, "text")?;
let pat = match pattern {
None | Some(Value::None) => None,
Some(p) => Some(as_text(p, "pattern")?),
};
parse_text(t, pat)
}
pub fn format(dt: &Value, style: Option<&Value>) -> Result<Value, String> {
let (secs, zone) = unix_of_value(dt)?;
let local = zone
.timestamp_opt(secs, 0)
.single()
.ok_or_else(|| format!("invalid unix timestamp: {secs}"))?;
let style = match style {
None | Some(Value::None) => "rfc3339",
Some(s) => as_text(s, "style")?,
};
let out = match style {
"rfc3339" | "iso" => local.to_rfc3339_opts(chrono::SecondsFormat::Secs, true),
"date" => local.format("%Y-%m-%d").to_string(),
"time" => local.format("%H:%M:%S").to_string(),
other => local.format(other).to_string(),
};
Ok(Value::Text(out))
}
pub fn add(
dt: &Value,
days: Option<&Value>,
hours: Option<&Value>,
minutes: Option<&Value>,
seconds: Option<&Value>,
) -> Result<Value, String> {
let (secs, zone) = unix_of_value(dt)?;
let days = optional_i64(days, "days")?;
let hours = optional_i64(hours, "hours")?;
let minutes = optional_i64(minutes, "minutes")?;
let seconds = optional_i64(seconds, "seconds")?;
let delta = Duration::days(days)
+ Duration::hours(hours)
+ Duration::minutes(minutes)
+ Duration::seconds(seconds);
let new_secs = secs
.checked_add(delta.num_seconds())
.ok_or_else(|| "datetime.add overflow".to_string())?;
moment(new_secs, zone)
}
pub fn in_zone(dt: &Value, zone: &Value) -> Result<Value, String> {
let (secs, _) = unix_of_value(dt)?;
let off = parse_zone(as_text(zone, "zone")?)?;
moment(secs, off)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn from_unix_utc_iso() {
let m = from_unix(&Value::Int(0), None).unwrap();
let Value::Map(entries) = m else { panic!() };
let iso = entries.iter().find(|(k, _)| k == "iso").unwrap().1.clone();
assert_eq!(iso, Value::Text("1970-01-01T00:00:00Z".into()));
}
#[test]
fn shanghai_alias() {
let m = from_unix(&Value::Int(0), Some(&Value::Text("Asia/Shanghai".into()))).unwrap();
let Value::Map(entries) = m else { panic!() };
let iso = entries.iter().find(|(k, _)| k == "iso").unwrap().1.clone();
assert_eq!(iso, Value::Text("1970-01-01T08:00:00+08:00".into()));
}
#[test]
fn parse_rfc3339_roundtrip() {
let m = parse(
&Value::Text("2020-01-02T03:04:05+08:00".into()),
None,
)
.unwrap();
let u = to_unix(&m).unwrap();
assert_eq!(u, Value::Int(1_577_905_445));
}
#[test]
fn add_day() {
let m = from_unix(&Value::Int(0), None).unwrap();
let n = add(&m, Some(&Value::Int(1)), None, None, None).unwrap();
assert_eq!(to_unix(&n).unwrap(), Value::Int(86_400));
}
}
+155 -6
View File
@@ -3,8 +3,8 @@
use std::collections::HashMap;
use crate::host::{
cli, csv, encoding, fs, hash, json, log, math, net, path, re, secrets, sys, text_ops, time,
uuid_ops, writeback, HostContext,
cli, csv, datetime, encoding, fs, hash, html_ops, json, log, math, net, path, re, secrets, stats,
sys, text_ops, time, toml_ops, url_ops, uuid_ops, writeback, HostContext,
};
#[cfg(feature = "exec-host")]
use crate::host::{foreign, subtask};
@@ -176,6 +176,28 @@ pub enum HostFn {
LogSetLevel = 167,
LogLine = 168,
UuidV4 = 169,
DatetimeNow = 170,
DatetimeFromUnix = 171,
DatetimeToUnix = 172,
DatetimeParse = 173,
DatetimeFormat = 174,
DatetimeAdd = 175,
DatetimeInZone = 176,
UrlParse = 177,
UrlQueryParse = 178,
UrlQueryStringify = 179,
TomlParse = 180,
HtmlEscape = 181,
HtmlUnescape = 182,
EncodingBase32Encode = 183,
EncodingBase32Decode = 184,
FsMakeDirs = 185,
FsRemoveTree = 186,
FsStat = 187,
FsWalk = 188,
StatsMean = 189,
StatsMedian = 190,
StatsStdev = 191,
}
@@ -342,6 +364,28 @@ impl HostFn {
167 => Self::LogSetLevel,
168 => Self::LogLine,
169 => Self::UuidV4,
170 => Self::DatetimeNow,
171 => Self::DatetimeFromUnix,
172 => Self::DatetimeToUnix,
173 => Self::DatetimeParse,
174 => Self::DatetimeFormat,
175 => Self::DatetimeAdd,
176 => Self::DatetimeInZone,
177 => Self::UrlParse,
178 => Self::UrlQueryParse,
179 => Self::UrlQueryStringify,
180 => Self::TomlParse,
181 => Self::HtmlEscape,
182 => Self::HtmlUnescape,
183 => Self::EncodingBase32Encode,
184 => Self::EncodingBase32Decode,
185 => Self::FsMakeDirs,
186 => Self::FsRemoveTree,
187 => Self::FsStat,
188 => Self::FsWalk,
189 => Self::StatsMean,
190 => Self::StatsMedian,
191 => Self::StatsStdev,
_ => return None,
})
}
@@ -511,6 +555,28 @@ impl HostFn {
"host_log_set_level" | "log_set_level" => Self::LogSetLevel,
"host_log_line" | "log_line" => Self::LogLine,
"host_uuid_v4" | "uuid_v4" => Self::UuidV4,
"host_datetime_now" | "datetime_now" => Self::DatetimeNow,
"host_datetime_from_unix" | "datetime_from_unix" => Self::DatetimeFromUnix,
"host_datetime_to_unix" | "datetime_to_unix" => Self::DatetimeToUnix,
"host_datetime_parse" | "datetime_parse" => Self::DatetimeParse,
"host_datetime_format" | "datetime_format" => Self::DatetimeFormat,
"host_datetime_add" | "datetime_add" => Self::DatetimeAdd,
"host_datetime_in_zone" | "datetime_in_zone" => Self::DatetimeInZone,
"host_url_parse" | "url_parse" => Self::UrlParse,
"host_url_query_parse" | "url_query_parse" => Self::UrlQueryParse,
"host_url_query_stringify" | "url_query_stringify" => Self::UrlQueryStringify,
"host_toml_parse" | "toml_parse" => Self::TomlParse,
"host_html_escape" | "html_escape" => Self::HtmlEscape,
"host_html_unescape" | "html_unescape" => Self::HtmlUnescape,
"host_encoding_base32_encode" | "encoding_base32_encode" => Self::EncodingBase32Encode,
"host_encoding_base32_decode" | "encoding_base32_decode" => Self::EncodingBase32Decode,
"host_make_dirs" | "make_dirs" => Self::FsMakeDirs,
"host_remove_tree" | "remove_tree" => Self::FsRemoveTree,
"host_stat" | "stat" => Self::FsStat,
"host_walk" | "walk" => Self::FsWalk,
"host_stats_mean" | "stats_mean" => Self::StatsMean,
"host_stats_median" | "stats_median" => Self::StatsMedian,
"host_stats_stdev" | "stats_stdev" => Self::StatsStdev,
_ => return None,
})
}
@@ -667,6 +733,28 @@ impl HostFn {
Self::LogSetLevel => "host_log_set_level",
Self::LogLine => "host_log_line",
Self::UuidV4 => "host_uuid_v4",
Self::DatetimeNow => "host_datetime_now",
Self::DatetimeFromUnix => "host_datetime_from_unix",
Self::DatetimeToUnix => "host_datetime_to_unix",
Self::DatetimeParse => "host_datetime_parse",
Self::DatetimeFormat => "host_datetime_format",
Self::DatetimeAdd => "host_datetime_add",
Self::DatetimeInZone => "host_datetime_in_zone",
Self::UrlParse => "host_url_parse",
Self::UrlQueryParse => "host_url_query_parse",
Self::UrlQueryStringify => "host_url_query_stringify",
Self::TomlParse => "host_toml_parse",
Self::HtmlEscape => "host_html_escape",
Self::HtmlUnescape => "host_html_unescape",
Self::EncodingBase32Encode => "host_encoding_base32_encode",
Self::EncodingBase32Decode => "host_encoding_base32_decode",
Self::FsMakeDirs => "host_make_dirs",
Self::FsRemoveTree => "host_remove_tree",
Self::FsStat => "host_stat",
Self::FsWalk => "host_walk",
Self::StatsMean => "host_stats_mean",
Self::StatsMedian => "host_stats_median",
Self::StatsStdev => "host_stats_stdev",
Self::WritebackRecord => "host_writeback_record",
Self::WritebackGet => "host_writeback_get",
Self::WritebackClear => "host_writeback_clear",
@@ -759,7 +847,9 @@ impl HostFn {
Self::EncodingBase64Encode
| Self::EncodingBase64Decode
| Self::EncodingHexEncode
| Self::EncodingHexDecode => &["text"],
| Self::EncodingHexDecode
| Self::EncodingBase32Encode
| Self::EncodingBase32Decode => &["text"],
Self::PathJoin => &["a", "b"],
Self::PathSplit
| Self::PathFileName
@@ -789,6 +879,18 @@ impl HostFn {
Self::LogSetLevel => &["level"],
Self::LogLine => &["level", "text"],
Self::UuidV4 => &[],
Self::StatsMean | Self::StatsMedian | Self::StatsStdev => &["list"],
Self::DatetimeNow => &[],
Self::DatetimeFromUnix => &["unix"],
Self::DatetimeToUnix => &["dt"],
Self::DatetimeParse => &["text"],
Self::DatetimeFormat => &["dt"],
Self::DatetimeAdd => &["dt"],
Self::DatetimeInZone => &["dt", "zone"],
Self::UrlParse | Self::UrlQueryParse => &["text"],
Self::UrlQueryStringify => &["map"],
Self::TomlParse | Self::HtmlEscape | Self::HtmlUnescape => &["text"],
Self::FsMakeDirs | Self::FsRemoveTree | Self::FsStat | Self::FsWalk => &["path"],
Self::WritebackRecord => &["value"],
Self::WritebackGet | Self::WritebackClear => &[],
Self::WritebackList => &[],
@@ -808,7 +910,7 @@ impl HostFn {
Self::OpenAiSseParse => &["echo"],
Self::HttpRequest => &["body", "content_type", "headers"],
Self::DotenvLoad => &["path"],
Self::Exec => &["args"],
Self::Exec => &["args", "capture"],
Self::Solve => &["min", "max"],
Self::Plot => &["steps", "path", "derivative", "grid"],
Self::PlotPoints => &["path", "grid"],
@@ -827,6 +929,11 @@ impl HostFn {
Self::TextReplace => &["count"],
Self::TextPad => &["fill", "align"],
Self::CliParse => &["args"],
Self::DatetimeFromUnix => &["zone"],
Self::DatetimeParse => &["pattern"],
Self::DatetimeFormat => &["style"],
Self::DatetimeAdd => &["days", "hours", "minutes", "seconds"],
Self::LogLine => &["fields"],
_ => &[],
}
}
@@ -855,6 +962,10 @@ pub fn call_host(
HostFn::ListDir => fs::list_dir(ctx, require(bound, "path")?),
HostFn::MakeDir => fs::make_dir(ctx, require(bound, "path")?),
HostFn::Remove => fs::remove(ctx, require(bound, "path")?),
HostFn::FsMakeDirs => fs::make_dirs(ctx, require(bound, "path")?),
HostFn::FsRemoveTree => fs::remove_tree(ctx, require(bound, "path")?),
HostFn::FsStat => fs::stat(ctx, require(bound, "path")?),
HostFn::FsWalk => fs::walk(ctx, require(bound, "path")?),
HostFn::NowUnix => time::now_unix(),
HostFn::NowMs => time::now_ms(),
HostFn::FormatTime => {
@@ -870,7 +981,12 @@ pub fn call_host(
HostFn::Args => sys::args(ctx),
HostFn::Cwd => sys::cwd(ctx),
HostFn::Exit => sys::exit(ctx, require(bound, "code")?),
HostFn::Exec => sys::exec(ctx, require(bound, "cmd")?, bound.get("args")),
HostFn::Exec => sys::exec(
ctx,
require(bound, "cmd")?,
bound.get("args"),
bound.get("capture"),
),
HostFn::JsonParse => json::parse(require(bound, "text")?),
HostFn::JsonStringify => {
json::stringify(require(bound, "value")?, bound.get("indent"))
@@ -1210,6 +1326,11 @@ pub fn call_host(
HostFn::EncodingBase64Decode => encoding::base64_decode(require(bound, "text")?),
HostFn::EncodingHexEncode => encoding::hex_encode(require(bound, "text")?),
HostFn::EncodingHexDecode => encoding::hex_decode(require(bound, "text")?),
HostFn::EncodingBase32Encode => encoding::base32_encode(require(bound, "text")?),
HostFn::EncodingBase32Decode => encoding::base32_decode(require(bound, "text")?),
HostFn::TomlParse => toml_ops::parse(require(bound, "text")?),
HostFn::HtmlEscape => html_ops::escape(require(bound, "text")?),
HostFn::HtmlUnescape => html_ops::unescape(require(bound, "text")?),
HostFn::PathJoin => path::join(require(bound, "a")?, require(bound, "b")?),
HostFn::PathSplit => path::split(require(bound, "path")?),
HostFn::PathFileName => path::file_name(require(bound, "path")?),
@@ -1270,8 +1391,36 @@ pub fn call_host(
HostFn::ListFlatten => super::collection::list_flatten(require(bound, "list")?),
HostFn::CliParse => cli::parse(ctx, bound.get("args")),
HostFn::LogSetLevel => log::set_level(ctx, require(bound, "level")?),
HostFn::LogLine => log::line(ctx, require(bound, "level")?, require(bound, "text")?),
HostFn::LogLine => log::line(
ctx,
require(bound, "level")?,
require(bound, "text")?,
bound.get("fields"),
),
HostFn::StatsMean => stats::mean(require(bound, "list")?),
HostFn::StatsMedian => stats::median(require(bound, "list")?),
HostFn::StatsStdev => stats::stdev(require(bound, "list")?),
HostFn::UuidV4 => uuid_ops::v4(),
HostFn::DatetimeNow => datetime::now(),
HostFn::DatetimeFromUnix => {
datetime::from_unix(require(bound, "unix")?, bound.get("zone"))
}
HostFn::DatetimeToUnix => datetime::to_unix(require(bound, "dt")?),
HostFn::DatetimeParse => datetime::parse(require(bound, "text")?, bound.get("pattern")),
HostFn::DatetimeFormat => datetime::format(require(bound, "dt")?, bound.get("style")),
HostFn::DatetimeAdd => datetime::add(
require(bound, "dt")?,
bound.get("days"),
bound.get("hours"),
bound.get("minutes"),
bound.get("seconds"),
),
HostFn::DatetimeInZone => {
datetime::in_zone(require(bound, "dt")?, require(bound, "zone")?)
}
HostFn::UrlParse => url_ops::parse(require(bound, "text")?),
HostFn::UrlQueryParse => url_ops::query_parse(require(bound, "text")?),
HostFn::UrlQueryStringify => url_ops::query_stringify(require(bound, "map")?),
HostFn::OuterCallLine => Ok(Value::Int(
ctx.call_site_lines
.first()
+71
View File
@@ -53,6 +53,74 @@ pub fn hex_decode(text: &Value) -> Result<Value, String> {
Ok(Value::Text(out))
}
const B32: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZ234567";
pub fn base32_encode(text: &Value) -> Result<Value, String> {
let s = as_text(text, "text")?;
Ok(Value::Text(encode_base32(s.as_bytes())))
}
pub fn base32_decode(text: &Value) -> Result<Value, String> {
let s = as_text(text, "text")?;
let bytes = decode_base32(s)?;
let out = String::from_utf8(bytes).map_err(|e| format!("base32_decode: invalid utf-8: {e}"))?;
Ok(Value::Text(out))
}
fn encode_base32(input: &[u8]) -> String {
let mut out = String::new();
let mut buffer: u64 = 0;
let mut bits = 0u32;
for &b in input {
buffer = (buffer << 8) | (b as u64);
bits += 8;
while bits >= 5 {
bits -= 5;
let idx = ((buffer >> bits) & 31) as usize;
out.push(B32[idx] as char);
}
}
if bits > 0 {
let idx = ((buffer << (5 - bits)) & 31) as usize;
out.push(B32[idx] as char);
}
while out.len() % 8 != 0 {
out.push('=');
}
out
}
fn decode_base32(input: &str) -> Result<Vec<u8>, String> {
let cleaned: String = input
.chars()
.filter(|c| !c.is_whitespace())
.map(|c| c.to_ascii_uppercase())
.collect();
if cleaned.len() % 8 != 0 {
return Err("base32_decode: length must be multiple of 8".into());
}
let mut out = Vec::new();
let mut buffer: u64 = 0;
let mut bits = 0u32;
for c in cleaned.bytes() {
if c == b'=' {
break;
}
let v = match c {
b'A'..=b'Z' => c - b'A',
b'2'..=b'7' => c - b'2' + 26,
_ => return Err(format!("base32_decode: invalid character {:?}", c as char)),
} as u64;
buffer = (buffer << 5) | v;
bits += 5;
if bits >= 8 {
bits -= 8;
out.push(((buffer >> bits) & 0xff) as u8);
}
}
Ok(out)
}
fn from_hex(b: u8) -> Result<u8, String> {
match b {
b'0'..=b'9' => Ok(b - b'0'),
@@ -153,5 +221,8 @@ mod tests {
let hx = hex_encode(&Value::Text("hi".into())).unwrap();
assert_eq!(hx, Value::Text("6869".into()));
assert_eq!(hex_decode(&hx).unwrap(), Value::Text("hi".into()));
let b32 = base32_encode(&Value::Text("hello".into())).unwrap();
assert_eq!(b32, Value::Text("NBSWY3DP".into()));
assert_eq!(base32_decode(&b32).unwrap(), Value::Text("hello".into()));
}
}
+84
View File
@@ -94,6 +94,16 @@ pub fn make_dir(ctx: &HostContext, path: &Value) -> Result<Value, String> {
Ok(Value::None)
}
/// Explicit recursive mkdir (same host behavior as `make_dir`).
pub fn make_dirs(ctx: &HostContext, path: &Value) -> Result<Value, String> {
if !ctx.allow_fs_write() {
return Err("make_dirs denied by host policy".into());
}
let p = ctx.resolve_path(text_path(path)?)?;
fs::create_dir_all(&p).map_err(|e| format!("make_dirs {}: {e}", p.display()))?;
Ok(Value::None)
}
pub fn remove(ctx: &HostContext, path: &Value) -> Result<Value, String> {
if !ctx.allow_fs_write() {
return Err("remove denied by host policy".into());
@@ -107,6 +117,80 @@ pub fn remove(ctx: &HostContext, path: &Value) -> Result<Value, String> {
Ok(Value::None)
}
/// Remove a directory tree. Errors if `path` is a file.
pub fn remove_tree(ctx: &HostContext, path: &Value) -> Result<Value, String> {
if !ctx.allow_fs_write() {
return Err("remove_tree denied by host policy".into());
}
let p = ctx.resolve_path(text_path(path)?)?;
if !p.exists() {
return Err(format!("remove_tree: not found: {}", p.display()));
}
if !p.is_dir() {
return Err(format!("remove_tree: not a directory: {}", p.display()));
}
fs::remove_dir_all(&p).map_err(|e| format!("remove_tree {}: {e}", p.display()))?;
Ok(Value::None)
}
/// `{size, mtime_unix, is_file, is_dir}` for a path.
pub fn stat(ctx: &HostContext, path: &Value) -> Result<Value, String> {
let p = ctx.resolve_path(text_path(path)?)?;
let meta = fs::metadata(&p).map_err(|e| format!("stat {}: {e}", p.display()))?;
let mtime = meta
.modified()
.ok()
.and_then(|t| t.duration_since(std::time::UNIX_EPOCH).ok())
.map(|d| d.as_secs() as i64)
.unwrap_or(0);
Ok(Value::Map(vec![
("size".into(), Value::Int(meta.len() as i64)),
("mtime_unix".into(), Value::Int(mtime)),
("is_file".into(), Value::Bool(meta.is_file())),
("is_dir".into(), Value::Bool(meta.is_dir())),
]))
}
/// Depth-first walk; returns path texts relative to `path` (forward slashes), sorted per directory.
pub fn walk(ctx: &HostContext, path: &Value) -> Result<Value, String> {
let root = ctx.resolve_path(text_path(path)?)?;
if !root.is_dir() {
return Err(format!("walk: not a directory: {}", root.display()));
}
let mut out: Vec<Value> = Vec::new();
walk_dfs(&root, Path::new(""), &mut out)?;
Ok(Value::List(out))
}
fn walk_dfs(abs: &Path, rel: &Path, out: &mut Vec<Value>) -> Result<(), String> {
let mut entries: Vec<_> = fs::read_dir(abs)
.map_err(|e| format!("walk {}: {e}", abs.display()))?
.collect::<Result<Vec<_>, _>>()
.map_err(|e| format!("walk: {e}"))?;
entries.sort_by_key(|e| e.file_name());
for ent in entries {
let name = ent.file_name();
let child_rel = if rel.as_os_str().is_empty() {
Path::new(&name).to_path_buf()
} else {
rel.join(&name)
};
let rel_text = child_rel
.components()
.map(|c| c.as_os_str().to_string_lossy())
.collect::<Vec<_>>()
.join("/");
out.push(Value::Text(rel_text));
let ft = ent
.file_type()
.map_err(|e| format!("walk file_type: {e}"))?;
if ft.is_dir() {
walk_dfs(&ent.path(), &child_rel, out)?;
}
}
Ok(())
}
pub fn copy_file(ctx: &HostContext, src: &Value, dest: &Value) -> Result<Value, String> {
if !ctx.allow_fs_write() {
return Err("copy_file denied by host policy".into());
+71
View File
@@ -0,0 +1,71 @@
//! HTML escape / unescape for Mid2 M8 (`lib/html`).
use crate::value::Value;
fn as_text<'a>(v: &'a Value, label: &str) -> Result<&'a str, String> {
match v {
Value::Text(s) => Ok(s.as_str()),
_ => Err(format!("{label} must be text")),
}
}
pub fn escape(text: &Value) -> Result<Value, String> {
let s = as_text(text, "text")?;
let mut out = String::with_capacity(s.len());
for c in s.chars() {
match c {
'&' => out.push_str("&amp;"),
'<' => out.push_str("&lt;"),
'>' => out.push_str("&gt;"),
'"' => out.push_str("&quot;"),
'\'' => out.push_str("&#x27;"),
_ => out.push(c),
}
}
Ok(Value::Text(out))
}
pub fn unescape(text: &Value) -> Result<Value, String> {
let s = as_text(text, "text")?;
let mut out = String::with_capacity(s.len());
let mut rest = s;
while let Some(i) = rest.find('&') {
out.push_str(&rest[..i]);
rest = &rest[i..];
if let Some(end) = rest.find(';') {
let ent = &rest[..=end];
let ch = match ent {
"&amp;" => Some('&'),
"&lt;" => Some('<'),
"&gt;" => Some('>'),
"&quot;" => Some('"'),
"&#x27;" | "&#39;" | "&apos;" => Some('\''),
_ => None,
};
if let Some(c) = ch {
out.push(c);
rest = &rest[end + 1..];
continue;
}
}
out.push('&');
rest = &rest[1..];
}
out.push_str(rest);
Ok(Value::Text(out))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn roundtrip() {
let e = escape(&Value::Text(r#"a<b>&"c'"#.into())).unwrap();
assert_eq!(
e,
Value::Text("a&lt;b&gt;&amp;&quot;c&#x27;".into())
);
assert_eq!(unescape(&e).unwrap(), Value::Text(r#"a<b>&"c'"#.into()));
}
}
+34 -2
View File
@@ -46,13 +46,32 @@ pub fn set_level(ctx: &mut HostContext, level: &Value) -> Result<Value, String>
}
/// Return a ready-to-print line, or `None` if filtered out.
pub fn line(ctx: &HostContext, level: &Value, text: &Value) -> Result<Value, String> {
/// Optional `fields` map is appended as `key=value` pairs (stable map order).
pub fn line(
ctx: &HostContext,
level: &Value,
text: &Value,
fields: Option<&Value>,
) -> Result<Value, String> {
let lvl = Level::parse(as_text(level, "level")?)?;
let msg = as_text(text, "text")?;
if (lvl as u8) < ctx.log_min_level {
return Ok(Value::None);
}
Ok(Value::Text(format!("{} {}", lvl.label(), msg)))
let mut out = format!("{} {}", lvl.label(), msg);
match fields {
None | Some(Value::None) => {}
Some(Value::Map(entries)) => {
for (k, v) in entries {
out.push(' ');
out.push_str(k);
out.push('=');
out.push_str(&v.as_display());
}
}
Some(_) => return Err("log: fields must be a map".into()),
}
Ok(Value::Text(out))
}
#[cfg(test)]
@@ -67,6 +86,7 @@ mod tests {
&ctx,
&Value::Text("debug".into()),
&Value::Text("x".into()),
None,
)
.unwrap();
assert_eq!(filtered, Value::None);
@@ -75,8 +95,20 @@ mod tests {
&ctx,
&Value::Text("debug".into()),
&Value::Text("x".into()),
None,
)
.unwrap();
assert_eq!(shown, Value::Text("DEBUG x".into()));
let with = line(
&ctx,
&Value::Text("info".into()),
&Value::Text("hi".into()),
Some(&Value::Map(vec![
("user".into(), Value::Text("a".into())),
("n".into(), Value::Int(1)),
])),
)
.unwrap();
assert_eq!(with, Value::Text("INFO hi user=a n=1".into()));
}
}
+6 -1
View File
@@ -9,12 +9,14 @@ pub mod collection;
mod dispatch;
mod cli;
mod csv;
mod datetime;
mod encoding;
pub mod event_bus;
#[cfg(feature = "exec-host")]
pub mod foreign;
mod fs;
mod hash;
mod html_ops;
pub(crate) mod json;
mod log;
pub mod math;
@@ -22,7 +24,11 @@ mod net;
mod path;
mod re;
mod secrets;
mod stats;
mod text_ops;
mod time;
mod toml_ops;
mod url_ops;
mod uuid_ops;
#[cfg(feature = "plugin-host")]
pub mod plugin;
@@ -35,7 +41,6 @@ pub mod subtask;
#[path = "subtask_stub.rs"]
pub mod subtask;
mod sys;
mod time;
pub mod writeback;
pub use dispatch::{call_host, HostFn};
+91 -11
View File
@@ -5,6 +5,7 @@ use std::collections::HashMap;
use std::ffi::{CStr, CString};
use std::os::raw::{c_char, c_int, c_void};
use std::path::PathBuf;
use std::sync::{Mutex, OnceLock};
use libloading::{Library, Symbol};
@@ -59,9 +60,38 @@ thread_local! {
static LIB_PATH_CALL: RefCell<Option<LibPathCall>> = const { RefCell::new(None) };
}
/// Process-wide host pointer while a long-lived plugin call (e.g. `web_listen`) is active.
/// Lets Tokio worker threads invoke `host_query` during HTTP handlers.
struct SendPtr<T>(*mut T);
unsafe impl<T> Send for SendPtr<T> {}
unsafe impl<T> Sync for SendPtr<T> {}
struct SendLibPath(Option<LibPathCall>);
unsafe impl Send for SendLibPath {}
unsafe impl Sync for SendLibPath {}
static GLOBAL_HOST: OnceLock<Mutex<SendPtr<HostContext>>> = OnceLock::new();
/// Process-wide lib-path hook (same lifetime as GLOBAL_HOST).
static GLOBAL_LIB_PATH: OnceLock<Mutex<SendLibPath>> = OnceLock::new();
/// Serialize interpreter re-entry from HTTP worker threads.
static INVOKE_LOCK: OnceLock<Mutex<()>> = OnceLock::new();
fn global_host_slot() -> &'static Mutex<SendPtr<HostContext>> {
GLOBAL_HOST.get_or_init(|| Mutex::new(SendPtr(std::ptr::null_mut())))
}
fn global_lib_slot() -> &'static Mutex<SendLibPath> {
GLOBAL_LIB_PATH.get_or_init(|| Mutex::new(SendLibPath(None)))
}
fn invoke_lock() -> &'static Mutex<()> {
INVOKE_LOCK.get_or_init(|| Mutex::new(()))
}
/// Hook: resolve bare `lib.member` (and longer paths) in the site module's import tree.
#[derive(Clone, Copy)]
pub struct LibPathCall {
pub call: fn(*mut (), &str) -> Result<Value, String>,
pub call: fn(*mut (), &str, Option<&Value>) -> Result<Value, String>,
pub data: *mut (),
}
@@ -69,7 +99,15 @@ pub struct LibPathCall {
pub fn with_lib_path_call<R>(hook: LibPathCall, body: impl FnOnce() -> R) -> R {
LIB_PATH_CALL.with(|slot| {
let prev = slot.replace(Some(hook));
let prev_g = {
let mut g = global_lib_slot().lock().unwrap_or_else(|e| e.into_inner());
std::mem::replace(&mut g.0, Some(hook))
};
let out = body();
{
let mut g = global_lib_slot().lock().unwrap_or_else(|e| e.into_inner());
g.0 = prev_g;
}
slot.replace(prev);
out
})
@@ -252,10 +290,23 @@ unsafe extern "C" fn host_query(
};
let ctx_ptr = CURRENT_HOST.with(|c| c.get());
let ctx_ptr = if ctx_ptr.is_null() {
global_host_slot()
.lock()
.unwrap_or_else(|e| e.into_inner())
.0
} else {
ctx_ptr
};
if ctx_ptr.is_null() {
set_err("host_query: no active host context (call during plugin fn)");
return 1;
}
let _invoke_guard;
let from_worker = CURRENT_HOST.with(|c| c.get().is_null());
if from_worker {
_invoke_guard = invoke_lock().lock().unwrap_or_else(|e| e.into_inner());
}
let ctx = &mut *ctx_ptr;
let args_owned: Option<String> = if args_json.is_null() {
@@ -314,14 +365,26 @@ unsafe extern "C" fn host_query(
.and_then(|v| v.as_str())
.ok_or_else(|| "call_lib_path requires `path`".to_string())?
.to_string();
LIB_PATH_CALL.with(|slot| {
let hook = slot.borrow();
let hook = hook
.as_ref()
.ok_or_else(|| "call_lib_path: no site lib resolver".to_string())?;
let v = (hook.call)(hook.data, &path)?;
value_to_json(&v)
})
let call_args = args.get("args").and_then(|v| {
if v.is_null() {
None
} else {
json_to_value(v).ok()
}
});
let call_args_ref = call_args.as_ref();
let from_tls = LIB_PATH_CALL.with(|slot| slot.borrow().clone());
let hook = if let Some(h) = from_tls {
h
} else {
global_lib_slot()
.lock()
.unwrap_or_else(|e| e.into_inner())
.0
.ok_or_else(|| "call_lib_path: no site lib resolver".to_string())?
};
let v = (hook.call)(hook.data, &path, call_args_ref)?;
value_to_json(&v)
})(),
"record_plot" => (|| {
let raw = args_owned.as_deref().unwrap_or("{}");
@@ -508,9 +571,26 @@ pub fn call_registered(
let mut out_ptr: *mut c_char = std::ptr::null_mut();
let mut err_ptr: *mut c_char = std::ptr::null_mut();
CURRENT_HOST.with(|c| c.set(ctx as *mut HostContext));
// Nesting-safe: restore previous host pointers after this call. Clearing to
// null would break a long-lived outer call (e.g. `web_listen`) so later
// HTTP `invoke` / `host_query` on worker threads see no active context.
let prev_tls = CURRENT_HOST.with(|c| {
let prev = c.get();
c.set(ctx as *mut HostContext);
prev
});
let prev_global = {
let mut g = global_host_slot().lock().unwrap_or_else(|e| e.into_inner());
let prev = g.0;
g.0 = ctx as *mut HostContext;
prev
};
let rc = unsafe { (reg.fn_ptr)(c_args.as_ptr(), &mut out_ptr, &mut err_ptr) };
CURRENT_HOST.with(|c| c.set(std::ptr::null_mut()));
{
let mut g = global_host_slot().lock().unwrap_or_else(|e| e.into_inner());
g.0 = prev_global;
}
CURRENT_HOST.with(|c| c.set(prev_tls));
let err_s = take_c_string(err_ptr);
let out_s = take_c_string(out_ptr);
+2 -1
View File
@@ -9,8 +9,9 @@ pub const ABI_VERSION: u32 = 2;
pub const ABI_VERSION_MIN: u32 = 1;
/// Hook: resolve bare `lib.member` (unused without native plugins).
#[derive(Clone, Copy)]
pub struct LibPathCall {
pub call: fn(*mut (), &str) -> Result<Value, String>,
pub call: fn(*mut (), &str, Option<&Value>) -> Result<Value, String>,
pub data: *mut (),
}
+87
View File
@@ -0,0 +1,87 @@
//! Thin statistics for Mid2 M10 (`lib/stats`). Not a dataframe library.
use crate::value::Value;
fn as_nums(list: &Value) -> Result<Vec<f64>, String> {
let Value::List(xs) = list else {
return Err("stats: list of numbers required".into());
};
let mut out = Vec::with_capacity(xs.len());
for (i, x) in xs.iter().enumerate() {
match x {
Value::Int(n) => out.push(*n as f64),
Value::Num(n) => out.push(*n),
_ => return Err(format!("stats: item {i} must be number")),
}
}
Ok(out)
}
fn num_val(n: f64) -> Value {
if n.fract() == 0.0 && n.abs() < (i64::MAX as f64) {
Value::Int(n as i64)
} else {
Value::Num(n)
}
}
pub fn mean(list: &Value) -> Result<Value, String> {
let xs = as_nums(list)?;
if xs.is_empty() {
return Err("stats.mean: empty list".into());
}
let sum: f64 = xs.iter().sum();
Ok(Value::Num(sum / xs.len() as f64))
}
pub fn median(list: &Value) -> Result<Value, String> {
let mut xs = as_nums(list)?;
if xs.is_empty() {
return Err("stats.median: empty list".into());
}
xs.sort_by(|a, b| a.partial_cmp(b).unwrap());
let n = xs.len();
let m = if n % 2 == 1 {
xs[n / 2]
} else {
(xs[n / 2 - 1] + xs[n / 2]) / 2.0
};
Ok(num_val(m))
}
/// Sample standard deviation (n >= 2), matching common `stdev` semantics.
pub fn stdev(list: &Value) -> Result<Value, String> {
let xs = as_nums(list)?;
if xs.len() < 2 {
return Err("stats.stdev: need at least 2 values".into());
}
let n = xs.len() as f64;
let mean = xs.iter().sum::<f64>() / n;
let var = xs.iter().map(|x| {
let d = x - mean;
d * d
}).sum::<f64>()
/ (n - 1.0);
Ok(Value::Num(var.sqrt()))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn basic() {
let xs = Value::List(vec![
Value::Int(1),
Value::Int(2),
Value::Int(3),
Value::Int(4),
]);
let m = mean(&xs).unwrap();
assert_eq!(m, Value::Num(2.5));
assert_eq!(median(&xs).unwrap(), Value::Num(2.5));
let s = stdev(&xs).unwrap();
let Value::Num(v) = s else { panic!() };
assert!((v - 1.2909944487358056).abs() < 1e-9);
}
}
+44 -6
View File
@@ -108,18 +108,28 @@ pub fn exit(ctx: &HostContext, code: &Value) -> Result<Value, String> {
}
}
pub fn exec(ctx: &HostContext, cmd: &Value, args: Option<&Value>) -> Result<Value, String> {
pub fn exec(
ctx: &HostContext,
cmd: &Value,
args: Option<&Value>,
capture: Option<&Value>,
) -> Result<Value, String> {
#[cfg(not(feature = "exec-host"))]
{
let _ = (ctx, cmd, args);
let _ = (ctx, cmd, args, capture);
return Err("exec unavailable in browser wasm".into());
}
#[cfg(feature = "exec-host")]
{
use std::process::Stdio;
if !ctx.allow_exec() {
return Err("exec denied by host policy".into());
}
let cmd = as_text(cmd, "cmd")?;
let capture = match capture {
None | Some(Value::None) => false,
Some(v) => crate::host::math::as_bool(v),
};
let mut command = Command::new(cmd);
command.current_dir(&ctx.cwd);
match args {
@@ -134,10 +144,38 @@ pub fn exec(ctx: &HostContext, cmd: &Value, args: Option<&Value>) -> Result<Valu
}
Some(_) => return Err("exec args must be list or text".into()),
}
let status = command
.status()
.map_err(|e| format!("exec `{cmd}`: {e}"))?;
Ok(Value::Int(status.code().unwrap_or(1) as i64))
if capture {
command.stdout(Stdio::piped());
command.stderr(Stdio::piped());
let output = command
.output()
.map_err(|e| format!("exec `{cmd}`: {e}"))?;
let code = output.status.code().unwrap_or(1) as i64;
let stdout = String::from_utf8_lossy(&output.stdout).into_owned();
let stderr = String::from_utf8_lossy(&output.stderr).into_owned();
Ok(Value::Map(vec![
("code".into(), Value::Int(code)),
("stdout".into(), Value::Text(clip_exec_io(&stdout))),
("stderr".into(), Value::Text(clip_exec_io(&stderr))),
]))
} else {
let status = command
.status()
.map_err(|e| format!("exec `{cmd}`: {e}"))?;
Ok(Value::Int(status.code().unwrap_or(1) as i64))
}
}
}
const EXEC_IO_CAP: usize = 256 * 1024;
fn clip_exec_io(s: &str) -> String {
let mut it = s.chars();
let head: String = it.by_ref().take(EXEC_IO_CAP).collect();
if it.next().is_some() {
format!("{head}\n…(truncated)")
} else {
head
}
}
+340
View File
@@ -0,0 +1,340 @@
//! Minimal TOML → Marqdo Value for Mid2 M8 (`lib/toml`).
//! Subset: comments, keys, strings, ints, floats, bools, arrays of scalars,
//! `[table]` / dotted tables. No datetime, inline tables, or array-of-tables.
use crate::host::json::json_to_value;
use crate::value::Value;
fn as_text<'a>(v: &'a Value, label: &str) -> Result<&'a str, String> {
match v {
Value::Text(s) => Ok(s.as_str()),
_ => Err(format!("{label} must be text")),
}
}
pub fn parse(text: &Value) -> Result<Value, String> {
let raw = as_text(text, "text")?;
let json = toml_to_json(raw)?;
json_to_value(&json)
}
fn toml_to_json(src: &str) -> Result<serde_json::Value, String> {
let mut root = serde_json::Map::new();
let mut current_path: Vec<String> = Vec::new();
for (lineno, line) in src.lines().enumerate() {
let line_no = lineno + 1;
let stripped = strip_comment(line);
let t = stripped.trim();
if t.is_empty() {
continue;
}
if t.starts_with('[') {
if !t.ends_with(']') {
return Err(format!("toml.parse:{line_no}: bad table header"));
}
if t.starts_with("[[") {
return Err(format!(
"toml.parse:{line_no}: array-of-tables [[...]] not supported"
));
}
let inner = t[1..t.len() - 1].trim();
current_path = split_dotted_keys(inner, line_no)?;
ensure_table_path(&mut root, &current_path, line_no)?;
continue;
}
let (key, val) = t
.split_once('=')
.ok_or_else(|| format!("toml.parse:{line_no}: expected key = value"))?;
let key = key.trim();
if key.is_empty() {
return Err(format!("toml.parse:{line_no}: empty key"));
}
let key_parts = split_dotted_keys(key, line_no)?;
let value = parse_value(val.trim(), line_no)?;
let parent = table_at_path(&mut root, &current_path, line_no)?;
insert_dotted(parent, &key_parts, value, line_no)?;
}
Ok(serde_json::Value::Object(root))
}
fn strip_comment(line: &str) -> String {
// Strip unquoted `#` comments.
let mut out = String::new();
let mut in_str = false;
let mut chars = line.chars().peekable();
while let Some(c) = chars.next() {
if !in_str && c == '#' {
break;
}
if c == '"' {
out.push(c);
in_str = !in_str;
continue;
}
if in_str && c == '\\' {
out.push(c);
if let Some(n) = chars.next() {
out.push(n);
}
continue;
}
out.push(c);
}
out
}
fn split_dotted_keys(s: &str, line_no: usize) -> Result<Vec<String>, String> {
let mut parts = Vec::new();
let mut cur = String::new();
let mut in_quote = false;
let mut chars = s.chars().peekable();
while let Some(c) = chars.next() {
if c == '"' {
in_quote = !in_quote;
continue;
}
if c == '.' && !in_quote {
if cur.is_empty() {
return Err(format!("toml.parse:{line_no}: empty key segment"));
}
parts.push(std::mem::take(&mut cur));
continue;
}
cur.push(c);
}
if in_quote {
return Err(format!("toml.parse:{line_no}: unclosed quoted key"));
}
if !cur.is_empty() {
parts.push(cur);
}
if parts.is_empty() {
return Err(format!("toml.parse:{line_no}: empty path"));
}
Ok(parts)
}
fn ensure_table_path(
root: &mut serde_json::Map<String, serde_json::Value>,
path: &[String],
line_no: usize,
) -> Result<(), String> {
let mut cur = root;
for seg in path {
if !cur.contains_key(seg) {
cur.insert(seg.clone(), serde_json::Value::Object(serde_json::Map::new()));
}
cur = match cur.get_mut(seg) {
Some(serde_json::Value::Object(m)) => m,
Some(_) => {
return Err(format!("toml.parse:{line_no}: {seg} is not a table"));
}
None => unreachable!(),
};
}
Ok(())
}
fn table_at_path<'a>(
root: &'a mut serde_json::Map<String, serde_json::Value>,
path: &[String],
line_no: usize,
) -> Result<&'a mut serde_json::Map<String, serde_json::Value>, String> {
let mut cur = root;
for seg in path {
cur = match cur.get_mut(seg) {
Some(serde_json::Value::Object(m)) => m,
Some(_) => {
return Err(format!("toml.parse:{line_no}: {seg} is not a table"));
}
None => {
return Err(format!("toml.parse:{line_no}: missing table {seg}"));
}
};
}
Ok(cur)
}
fn insert_dotted(
parent: &mut serde_json::Map<String, serde_json::Value>,
parts: &[String],
value: serde_json::Value,
line_no: usize,
) -> Result<(), String> {
if parts.len() == 1 {
if parent.contains_key(&parts[0]) {
return Err(format!(
"toml.parse:{line_no}: duplicate key {}",
parts[0]
));
}
parent.insert(parts[0].clone(), value);
return Ok(());
}
let head = &parts[0];
let entry = parent
.entry(head.clone())
.or_insert_with(|| serde_json::Value::Object(serde_json::Map::new()));
let obj = match entry {
serde_json::Value::Object(m) => m,
_ => {
return Err(format!("toml.parse:{line_no}: {head} is not a table"));
}
};
insert_dotted(obj, &parts[1..], value, line_no)
}
fn parse_value(s: &str, line_no: usize) -> Result<serde_json::Value, String> {
if s.is_empty() {
return Err(format!("toml.parse:{line_no}: empty value"));
}
if s == "true" {
return Ok(serde_json::Value::Bool(true));
}
if s == "false" {
return Ok(serde_json::Value::Bool(false));
}
if s.starts_with('"') {
return Ok(serde_json::Value::String(parse_basic_string(s, line_no)?));
}
if s.starts_with('\'') {
return Ok(serde_json::Value::String(parse_literal_string(s, line_no)?));
}
if s.starts_with('[') {
return parse_array(s, line_no);
}
if let Ok(i) = s.replace('_', "").parse::<i64>() {
return Ok(serde_json::Value::Number(i.into()));
}
if let Ok(f) = s.replace('_', "").parse::<f64>() {
if let Some(n) = serde_json::Number::from_f64(f) {
return Ok(serde_json::Value::Number(n));
}
}
Err(format!("toml.parse:{line_no}: unsupported value {s:?}"))
}
fn parse_basic_string(s: &str, line_no: usize) -> Result<String, String> {
if !s.ends_with('"') || s.len() < 2 {
return Err(format!("toml.parse:{line_no}: bad string"));
}
let inner = &s[1..s.len() - 1];
let mut out = String::new();
let mut chars = inner.chars();
while let Some(c) = chars.next() {
if c == '\\' {
match chars.next() {
Some('n') => out.push('\n'),
Some('t') => out.push('\t'),
Some('r') => out.push('\r'),
Some('\\') => out.push('\\'),
Some('"') => out.push('"'),
Some(other) => {
return Err(format!(
"toml.parse:{line_no}: bad escape \\{other}"
));
}
None => return Err(format!("toml.parse:{line_no}: trailing escape")),
}
} else {
out.push(c);
}
}
Ok(out)
}
fn parse_literal_string(s: &str, line_no: usize) -> Result<String, String> {
if !s.ends_with('\'') || s.len() < 2 {
return Err(format!("toml.parse:{line_no}: bad literal string"));
}
Ok(s[1..s.len() - 1].to_string())
}
fn parse_array(s: &str, line_no: usize) -> Result<serde_json::Value, String> {
if !s.ends_with(']') {
return Err(format!("toml.parse:{line_no}: bad array"));
}
let inner = s[1..s.len() - 1].trim();
if inner.is_empty() {
return Ok(serde_json::Value::Array(vec![]));
}
let mut items = Vec::new();
let mut cur = String::new();
let mut depth = 0i32;
let mut in_str = false;
let mut str_ch = '"';
let mut chars = inner.chars().peekable();
while let Some(c) = chars.next() {
if in_str {
cur.push(c);
if c == '\\' {
if let Some(n) = chars.next() {
cur.push(n);
}
continue;
}
if c == str_ch {
in_str = false;
}
continue;
}
match c {
'"' | '\'' => {
in_str = true;
str_ch = c;
cur.push(c);
}
'[' => {
depth += 1;
cur.push(c);
}
']' => {
depth -= 1;
cur.push(c);
}
',' if depth == 0 => {
items.push(parse_value(cur.trim(), line_no)?);
cur.clear();
}
_ => cur.push(c),
}
}
if !cur.trim().is_empty() {
items.push(parse_value(cur.trim(), line_no)?);
}
Ok(serde_json::Value::Array(items))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parse_simple() {
let t = r#"
# comment
name = "marqdo"
count = 3
ok = true
[pkg]
version = "0.1"
"#;
let v = parse(&Value::Text(t.into())).unwrap();
let Value::Map(entries) = v else { panic!() };
let get = |k: &str| {
entries
.iter()
.find(|(kk, _)| kk == k)
.map(|(_, v)| v.clone())
.unwrap()
};
assert_eq!(get("name"), Value::Text("marqdo".into()));
assert_eq!(get("count"), Value::Int(3));
assert_eq!(get("ok"), Value::Bool(true));
let Value::Map(pkg) = get("pkg") else { panic!() };
assert_eq!(
pkg.iter().find(|(k, _)| k == "version").unwrap().1,
Value::Text("0.1".into())
);
}
}
+254
View File
@@ -0,0 +1,254 @@
//! URL parse / query helpers for Mid2 M7 (`lib/url`).
//! Distinct from `net::url_encode` (single-segment form encoding).
//! Hand-rolled subset (no `url` crate): scheme://[userinfo@]host[:port]/path[?query][#frag].
use crate::value::Value;
fn as_text<'a>(v: &'a Value, label: &str) -> Result<&'a str, String> {
match v {
Value::Text(s) => Ok(s.as_str()),
_ => Err(format!("{label} must be text")),
}
}
fn encode_component(s: &str) -> String {
let mut out = String::new();
for b in s.bytes() {
match b {
b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'_' | b'.' | b'~' => {
out.push(b as char);
}
_ => out.push_str(&format!("%{b:02X}")),
}
}
out
}
fn decode_component(s: &str) -> Result<String, String> {
let bytes = s.as_bytes();
let mut out = Vec::with_capacity(bytes.len());
let mut i = 0;
while i < bytes.len() {
match bytes[i] {
b'+' => {
out.push(b' ');
i += 1;
}
b'%' if i + 2 < bytes.len() => {
let h = |c: u8| -> Result<u8, String> {
match c {
b'0'..=b'9' => Ok(c - b'0'),
b'a'..=b'f' => Ok(c - b'a' + 10),
b'A'..=b'F' => Ok(c - b'A' + 10),
_ => Err(format!("bad percent escape in {s:?}")),
}
};
let v = (h(bytes[i + 1])? << 4) | h(bytes[i + 2])?;
out.push(v);
i += 3;
}
b'%' => return Err(format!("truncated percent escape in {s:?}")),
c => {
out.push(c);
i += 1;
}
}
}
String::from_utf8(out).map_err(|e| format!("url decode utf8: {e}"))
}
/// Parse `scheme://[userinfo@]host[:port]/path[?query][#fragment]`.
pub fn parse(text: &Value) -> Result<Value, String> {
let raw = as_text(text, "text")?;
let (scheme, rest) = raw
.split_once("://")
.ok_or_else(|| format!("url.parse: missing :// in {raw:?}"))?;
if scheme.is_empty()
|| !scheme
.chars()
.all(|c| c.is_ascii_alphanumeric() || c == '+' || c == '-' || c == '.')
{
return Err(format!("url.parse: bad scheme in {raw:?}"));
}
let (before_frag, fragment) = match rest.split_once('#') {
Some((a, b)) => (a, b),
None => (rest, ""),
};
let (before_query, query) = match before_frag.split_once('?') {
Some((a, b)) => (a, b),
None => (before_frag, ""),
};
let (authority, path) = if let Some(i) = before_query.find('/') {
(&before_query[..i], &before_query[i..])
} else {
(before_query, "")
};
let (userinfo, hostport) = if let Some((u, h)) = authority.rsplit_once('@') {
(u, h)
} else {
("", authority)
};
let (host, port) = if hostport.starts_with('[') {
// IPv6 literal [addr]:port
let end = hostport
.find(']')
.ok_or_else(|| format!("url.parse: bad IPv6 host in {raw:?}"))?;
let host = &hostport[1..end];
let rest = &hostport[end + 1..];
if rest.is_empty() {
(host, Value::None)
} else if let Some(p) = rest.strip_prefix(':') {
let n: i64 = p
.parse()
.map_err(|_| format!("url.parse: bad port in {raw:?}"))?;
(host, Value::Int(n))
} else {
return Err(format!("url.parse: bad hostport in {raw:?}"));
}
} else if let Some((h, p)) = hostport.rsplit_once(':') {
if p.chars().all(|c| c.is_ascii_digit()) && !p.is_empty() {
let n: i64 = p
.parse()
.map_err(|_| format!("url.parse: bad port in {raw:?}"))?;
(h, Value::Int(n))
} else {
(hostport, Value::None)
}
} else {
(hostport, Value::None)
};
let path = if path.is_empty() {
"/".to_string()
} else {
path.to_string()
};
Ok(Value::Map(vec![
("scheme".into(), Value::Text(scheme.to_string())),
("userinfo".into(), Value::Text(userinfo.to_string())),
("host".into(), Value::Text(host.to_string())),
("port".into(), port),
("path".into(), Value::Text(path)),
("query".into(), Value::Text(query.to_string())),
("fragment".into(), Value::Text(fragment.to_string())),
]))
}
pub fn query_parse(text: &Value) -> Result<Value, String> {
let raw = as_text(text, "text")?;
let raw = raw.strip_prefix('?').unwrap_or(raw);
let mut map: Vec<(String, Value)> = Vec::new();
if raw.is_empty() {
return Ok(Value::Map(map));
}
for pair in raw.split('&') {
if pair.is_empty() {
continue;
}
let (k, v) = match pair.split_once('=') {
Some((k, v)) => (k, v),
None => (pair, ""),
};
let key = decode_component(k)?;
let val = Value::Text(decode_component(v)?);
if let Some((_, slot)) = map.iter_mut().find(|(kk, _)| *kk == key) {
match slot {
Value::List(xs) => xs.push(val),
other => {
let prev = std::mem::replace(other, Value::None);
*other = Value::List(vec![prev, val]);
}
}
} else {
map.push((key, val));
}
}
Ok(Value::Map(map))
}
pub fn query_stringify(map: &Value) -> Result<Value, String> {
let entries = match map {
Value::Map(m) => m,
Value::None => return Ok(Value::Text(String::new())),
_ => return Err("url.query_stringify: map required".into()),
};
let mut parts: Vec<String> = Vec::new();
for (k, v) in entries {
let key = encode_component(k);
match v {
Value::List(xs) => {
for x in xs {
match x {
Value::Text(s) => {
parts.push(format!("{key}={}", encode_component(s)));
}
Value::Int(n) => {
parts.push(format!("{key}={}", encode_component(&n.to_string())));
}
Value::Bool(b) => {
parts.push(format!("{key}={}", encode_component(&b.to_string())));
}
Value::None => parts.push(format!("{key}=")),
_ => {
return Err("url.query_stringify: list values must be scalar".into())
}
}
}
}
Value::Text(s) => parts.push(format!("{key}={}", encode_component(s))),
Value::Int(n) => parts.push(format!("{key}={}", encode_component(&n.to_string()))),
Value::Bool(b) => parts.push(format!("{key}={}", encode_component(&b.to_string()))),
Value::None => parts.push(format!("{key}=")),
_ => return Err("url.query_stringify: values must be scalar or list".into()),
}
}
Ok(Value::Text(parts.join("&")))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parse_https_query() {
let m = parse(&Value::Text(
"https://user:pass@example.com:8443/a/b?x=1&y=2#frag".into(),
))
.unwrap();
let Value::Map(entries) = m else { panic!() };
let get = |k: &str| {
entries
.iter()
.find(|(kk, _)| kk == k)
.map(|(_, v)| v.clone())
.unwrap()
};
assert_eq!(get("scheme"), Value::Text("https".into()));
assert_eq!(get("host"), Value::Text("example.com".into()));
assert_eq!(get("port"), Value::Int(8443));
assert_eq!(get("path"), Value::Text("/a/b".into()));
assert_eq!(get("query"), Value::Text("x=1&y=2".into()));
assert_eq!(get("fragment"), Value::Text("frag".into()));
assert_eq!(get("userinfo"), Value::Text("user:pass".into()));
}
#[test]
fn query_roundtrip() {
let q = query_parse(&Value::Text("a=1&b=hello%20w&a=2".into())).unwrap();
let s = query_stringify(&q).unwrap();
assert_eq!(s, Value::Text("a=1&a=2&b=hello%20w".into()));
}
#[test]
fn empty_path_becomes_slash() {
let m = parse(&Value::Text("https://example.com".into())).unwrap();
let Value::Map(entries) = m else { panic!() };
let path = entries.iter().find(|(k, _)| k == "path").unwrap().1.clone();
assert_eq!(path, Value::Text("/".into()));
}
}
+43 -17
View File
@@ -195,7 +195,11 @@ impl Interpreter {
}
/// Resolve `lib.member` on the site (entry) module — used by ABI `call_lib_path`.
pub(crate) fn call_site_lib_path(&mut self, path: &str) -> std::result::Result<Value, String> {
pub(crate) fn call_site_lib_path(
&mut self,
path: &str,
args: Option<&Value>,
) -> std::result::Result<Value, String> {
let site = self
.site_module
.ok_or_else(|| "call_lib_path: no site module".to_string())?;
@@ -210,7 +214,13 @@ impl Interpreter {
"call_lib_path: need `lib.member`, got `{path}`"
));
}
self.eval_path_call(site, &parts, &[])
let mut ev_args: Vec<EvArg> = Vec::new();
if let Some(Value::Map(pairs)) = args {
for (k, v) in pairs {
ev_args.push(EvArg::Named(k.clone(), v.clone()));
}
}
self.eval_path_call(site, &parts, &ev_args)
.map_err(|e| e.to_string())
}
@@ -222,10 +232,14 @@ impl Interpreter {
struct HookData {
interp: *mut Interpreter,
}
fn trampoline(data: *mut (), path: &str) -> std::result::Result<Value, String> {
fn trampoline(
data: *mut (),
path: &str,
args: Option<&Value>,
) -> std::result::Result<Value, String> {
let data = unsafe { &mut *(data as *mut HookData) };
let interp = unsafe { &mut *data.interp };
interp.call_site_lib_path(path)
interp.call_site_lib_path(path, args)
}
let mut data = HookData {
interp: self as *mut Interpreter,
@@ -1042,19 +1056,31 @@ fn find_top<'a>(module: &'a Module, name: &str) -> Option<&'a Function> {
/// Resolve `#` object type by name in the entry module or any imported library.
fn find_object_type<'a>(module: &'a Module, name: &str) -> Option<(&'a Module, &'a Function)> {
module
.functions
.iter()
.find(|f| f.name == name && f.is_object())
.map(|f| (module, f))
.or_else(|| {
module.import_modules.values().find_map(|lib| {
lib.functions
.iter()
.find(|f| f.name == name && f.is_object())
.map(|f| (lib, f))
})
})
fn search<'a>(
module: &'a Module,
name: &str,
seen: &mut std::collections::HashSet<*const Module>,
) -> Option<(&'a Module, &'a Function)> {
let key = module as *const Module;
if !seen.insert(key) {
return None;
}
if let Some(f) = module
.functions
.iter()
.find(|f| f.name == name && f.is_object())
{
return Some((module, f));
}
for lib in module.import_modules.values() {
if let Some(hit) = search(lib, name, seen) {
return Some(hit);
}
}
None
}
let mut seen = std::collections::HashSet::new();
search(module, name, &mut seen)
}
/// Resolve `` `obj`.method `` along the inheritance chain (`_type` first, then bases).
+142 -11
View File
@@ -44,6 +44,8 @@ fn is_code_starter(c: char) -> bool {
}
/// `*` / `**` lines that are Marqdo executable (statement / return / else), not narrative Markdown.
///
/// Closing `*` / `**` must not be taken from inside `` `…` `` or `"…"` (GAP-06/07 nested markers).
fn is_marqdo_star_line(trimmed: &str) -> bool {
if trimmed == "*" || trimmed == "****" {
return true;
@@ -52,14 +54,126 @@ fn is_marqdo_star_line(trimmed: &str) -> bool {
if trimmed.trim() == "** **" {
return true;
}
if let Some(rest) = trimmed.strip_prefix("**") {
if let Some(pos) = rest.find("**") {
return rest[pos + 2..].trim().is_empty();
}
}
return matching_bold_close(trimmed).is_some();
}
if !trimmed.starts_with('*') || trimmed.len() < 2 {
return false;
}
trimmed.starts_with('*') && trimmed.ends_with('*') && trimmed.len() >= 2
matching_italic_close(trimmed).is_some()
}
/// Index of the closing `*` for a single-star statement, skipping strings and backticks.
fn matching_italic_close(trimmed: &str) -> Option<usize> {
if trimmed.starts_with("**") {
return None;
}
let bytes = trimmed.as_bytes();
let mut i = 1usize; // skip opening *
let mut in_bt = false;
let mut in_str = false;
while i < bytes.len() {
let c = bytes[i];
if in_str {
if c == b'\\' && i + 1 < bytes.len() {
i += 2;
continue;
}
if c == b'"' {
in_str = false;
}
i += 1;
continue;
}
if in_bt {
if c == b'`' {
in_bt = false;
}
i += 1;
continue;
}
match c {
b'`' => {
in_bt = true;
i += 1;
}
b'"' => {
in_str = true;
i += 1;
}
b'*' => {
// Closing * must be the last non-ws on the line.
if trimmed[i + 1..].trim().is_empty() {
return Some(i);
}
i += 1;
}
_ => i += 1,
}
}
None
}
/// Closing `**` for a bold return, skipping strings and backticks.
fn matching_bold_close(trimmed: &str) -> Option<usize> {
let Some(rest) = trimmed.strip_prefix("**") else {
return None;
};
let bytes = rest.as_bytes();
let mut i = 0usize;
let mut in_bt = false;
let mut in_str = false;
while i + 1 < bytes.len() {
let c = bytes[i];
if in_str {
if c == b'\\' && i + 1 < bytes.len() {
i += 2;
continue;
}
if c == b'"' {
in_str = false;
}
i += 1;
continue;
}
if in_bt {
if c == b'`' {
in_bt = false;
}
i += 1;
continue;
}
match c {
b'`' => {
in_bt = true;
i += 1;
}
b'"' => {
in_str = true;
i += 1;
}
b'*' if bytes[i + 1] == b'*' => {
let after = i + 2;
if rest[after..].trim().is_empty() {
// Position relative to full `trimmed` (after opening `**`).
return Some(2 + i);
}
i += 2;
}
_ => i += 1,
}
}
None
}
/// Headings / calls / finished star lines should escape a narrative paragraph (GAP-06/07 recovery).
fn breaks_comment_paragraph(trimmed: &str) -> bool {
if is_structural_code_line(trimmed) {
return true;
}
if trimmed.starts_with('#') || trimmed.starts_with('>') {
return true;
}
is_marqdo_star_line(trimmed)
}
/// Classify a single logical line (no trailing newline required).
@@ -141,7 +255,7 @@ pub fn classify_source(source: &str) -> Vec<ClassifiedLine> {
in_comment_paragraph = true;
LineKind::Comment
}
LineKind::Code if structural => {
LineKind::Code if structural || breaks_comment_paragraph(trimmed) => {
in_comment_paragraph = false;
LineKind::Code
}
@@ -298,8 +412,8 @@ mod tests {
fn star_line_classification_is_star_terminated() {
// A `*…*` Marqdo statement line must END with `*` to be treated as code.
// If it accidentally ends with a backtick (e.g. `table="t"`), the line
// falls back to narrative Comment and swallows the following paragraph.
// Regression: `store.count` calls after a blank line were being dropped.
// falls back to narrative Comment — but later executable lines recover
// (GAP-06/07: do not swallow the rest of a `##` body).
let star_term = "*c = > `store`.count table=\"t\"*";
assert!(
is_marqdo_star_line(star_term),
@@ -307,6 +421,21 @@ mod tests {
);
assert_eq!(classify_line(star_term), LineKind::Code);
let nested_bt = "*code = > sys.exec cmd=\"python3\" args=`args`*";
assert!(
is_marqdo_star_line(nested_bt),
"args=`args`* must still be a star line"
);
let nested_json = "*args = > json.parse text=[\"scripts/legacy/web_search.py\"]*";
assert!(
is_marqdo_star_line(nested_json),
"text=[…] * must still be a star line"
);
assert!(
is_marqdo_star_line("*x = > f text=`a*b`*"),
"* inside backticks must not steal the closer"
);
let backtick_term = "*c = > `store`.count table=\"t\"`";
assert!(
!is_marqdo_star_line(backtick_term),
@@ -314,9 +443,11 @@ mod tests {
);
assert_eq!(classify_line(backtick_term), LineKind::Comment);
// In paragraph context the misplaced line must not stay Code.
let src = "> `store`.insert table=t rows=`数据`\n\n*c = > `store`.count table=\"t\"`\n> print text=count1-done\n";
// Bad terminator → comment, but following `>` / `*…*` escape the paragraph.
let src = "> `store`.insert table=t rows=`数据`\n\n*c = > `store`.count table=\"t\"`\n> print text=count1-done\n*y = 1*\n";
let lines = classify_source(src);
assert_eq!(lines[2].kind, LineKind::Comment, "bad terminator → comment");
assert_eq!(lines[3].kind, LineKind::Code, "call recovers after bad line");
assert_eq!(lines[4].kind, LineKind::Code, "assign recovers after bad line");
}
}
+31 -6
View File
@@ -924,9 +924,12 @@ fn split_first_token(s: &str) -> (&str, &str) {
}
fn find_next_arg_boundary(after_eq: &str) -> usize {
// Look for ` <ident>=` pattern not inside backticks or quoted strings.
// Look for ` <ident>=` pattern not inside backticks, quoted strings, or
// JSON-ish `[…]` / `{…}` (GAP-07: `text=["a"] other=…`).
let mut in_bt = false;
let mut in_str = false;
let mut bracket_depth: i32 = 0;
let mut brace_depth: i32 = 0;
let mut chars = after_eq.char_indices().peekable();
while let Some((i, c)) = chars.next() {
if in_str {
@@ -945,12 +948,34 @@ fn find_next_arg_boundary(after_eq: &str) -> usize {
}
continue;
}
if c == '`' {
in_bt = true;
continue;
match c {
'`' => {
in_bt = true;
continue;
}
'"' => {
in_str = true;
continue;
}
'[' => {
bracket_depth += 1;
continue;
}
']' => {
bracket_depth = bracket_depth.saturating_sub(1);
continue;
}
'{' => {
brace_depth += 1;
continue;
}
'}' => {
brace_depth = brace_depth.saturating_sub(1);
continue;
}
_ => {}
}
if c == '"' {
in_str = true;
if bracket_depth > 0 || brace_depth > 0 {
continue;
}
if c.is_whitespace() {
+15
View File
@@ -0,0 +1,15 @@
---
title: agent mcp server smoke helpers
description: Tool target for MCP server smoke.
---
## ping
+ `payload`=None
`out` =
| ok | via |
|----|-----|
| true | mcp |
**out**
+32
View File
@@ -0,0 +1,32 @@
---
title: agent mcp server smoke
description: Offline assemble mcp_server + one tool (no serve).
import agent:ext/ai/agent.mq.md
import plugin:lib/plugin.mq.md
import sys:lib/sys.mq.md
import demo:agent-mcp-server-helpers.mq.md
---
# main
*p = > plugin.native_path name="agent"*
1. `p`
> plugin.load path=`p`
2. *
> sys.exit code=1
*srv = > agent.mcp_server name="marqdo-test"*
*srv = > `srv`.tool name="ping" fn="demo.ping" description="ping tool"*
*tools = srv[^tools]*
*n = > len value=`tools`*
1. `n` == 1
> print text=mcp-tool-ok
2. *
> print text=mcp-tool-fail
*t0 = tools[^1]*
*fn = t0[^fn]*
1. `fn` == "demo.ping"
> print text=mcp-fn-ok
2. *
> print text=mcp-fn-fail
+26
View File
@@ -0,0 +1,26 @@
---
title: web db cross-module lib
description: Opens sqlite, inits a tiny table, returns the db handle.
import web:ext/web/web.mq.md
---
## open
*store = > web.db url="sqlite::memory:"*
`fields` =
| name | type |
|------|------|
| title | text |
> `store`.init name=items fields=`fields`
`seed` =
| @ | title |
|---|-------|
| 1 | hello |
> `store`.insert table=items rows=`seed`
**store**
+24
View File
@@ -0,0 +1,24 @@
---
title: web db cross-module smoke
description: Return a web.db handle from an imported module and call select in the entry module.
import web:ext/web/web.mq.md
import plugin:lib/plugin.mq.md
import sys:lib/sys.mq.md
import storemod:web-db-cross-module-lib.mq.md
---
# main
*p = > plugin.native_path name="web"*
1. `p`
> plugin.load path=`p`
2. *
> sys.exit code=1
*store = > storemod.open*
*rows = > `store`.select table="items" limit=10*
*n = > len value=`rows`*
1. `n` >= 1
> print text=cross-db-ok
2. *
> print text=cross-db-fail
+23
View File
@@ -0,0 +1,23 @@
---
title: web hosting live helpers
description: Tiny lib used by hosting live-server invoke route (nested plugin call).
import web:ext/web/web.mq.md
---
## echo
+ `msg`=""
+ `payload`=None
Nested plugin call (`web.page`) while `app.listen` is the outer plugin frame —
must not clear GLOBAL_HOST so a second HTTP invoke still works.
*pg = > web.page title=`msg`*
*title = pg[^title]*
`out` =
| ok | msg | title |
|----|-----|-------|
| true | `msg` | `title` |
**out**
+22
View File
@@ -0,0 +1,22 @@
---
title: web hosting live server
description: Listen with proxy (→ mock upstream) and invoke routes for gold.rs.
import web:ext/web/web.mq.md
import plugin:lib/plugin.mq.md
import sys:lib/sys.mq.md
import demo:web-hosting-helpers.mq.md
---
# main
*p = > plugin.native_path name="web"*
1. `p`
> plugin.load path=`p`
2. *
> sys.exit code=1
*pg = > web.page title="hosting-live"*
*app = > web.app page=`pg` host=127.0.0.1 port=18141*
*app = > `app`.proxy path="/proxy/sse" upstream="http://127.0.0.1:18142/sse" stream=True methods="POST"*
*app = > `app`.invoke path="/api/echo" fn="demo.echo" method="POST" body="json"*
> `app`.listen
+75
View File
@@ -0,0 +1,75 @@
---
title: web proxy/invoke smoke
description: Offline assemble of app.proxy and app.invoke routes (no listen).
import web:ext/web/web.mq.md
import plugin:lib/plugin.mq.md
import sys:lib/sys.mq.md
---
# main
*p = > plugin.native_path name="web"*
1. `p`
> plugin.load path=`p`
2. *
> sys.exit code=1
`proxy_tbl` =
| path | upstream | stream | strip_prefix | methods | headers_from_env | timeout_ms |
|------|----------|--------|--------------|---------|------------------|------------|
| /proxy/chat | $OPENAI_BASE_URL | True | /proxy | POST | OPENAI_API_KEY=Authorization | 60000 |
`invoke_tbl` =
| path | method | fn | body | return |
|------|--------|-----|------|--------|
| /api/echo | POST | demo.echo | json | json |
*pg = > web.page title="proxy-invoke"*
*app = > web.app page=`pg`*
*app = > `app`.configure proxy=`proxy_tbl` invoke=`invoke_tbl`*
*app = > `app`.proxy path="/v1/stream" upstream="https://example.com/v1/chat" stream=True headers_from_env="TOKEN=X-Token"*
*app = > `app`.invoke path="/api/ping" fn="demo.ping" method="GET" body="query"*
*proutes = app[^proxy_routes]*
*chat = proutes[^proxy/chat]*
*up = chat[^upstream]*
1. `up` == "$OPENAI_BASE_URL"
> print text=proxy-table-ok
2. *
> print text=proxy-table-fail
*stream = chat[^stream]*
1. `stream`
> print text=proxy-stream-ok
2. *
> print text=proxy-stream-fail
*hdr = chat[^headers_from_env]*
1. `hdr` == "OPENAI_API_KEY=Authorization"
> print text=proxy-env-ok
2. *
> print text=proxy-env-fail
*v1 = proutes[^v1/stream]*
*v1u = v1[^upstream]*
1. `v1u` == "https://example.com/v1/chat"
> print text=proxy-method-ok
2. *
> print text=proxy-method-fail
*iroutes = app[^invoke_routes]*
*echo = iroutes[^api/echo]*
*efn = echo[^fn]*
1. `efn` == "demo.echo"
> print text=invoke-table-ok
2. *
> print text=invoke-table-fail
*ping = iroutes[^api/ping]*
*pm = ping[^method]*
1. `pm` == "GET"
> print text=invoke-method-ok
2. *
> print text=invoke-method-fail
+259 -8
View File
@@ -189,18 +189,16 @@ sk-live
}
#[test]
fn structure_bare_id_italic_bold() {
fn structure_gap_star_json_body() {
assert_out(
"tests/structure/bare-id-italic-bold.mq.md",
"2
2
quoted
7
ok
hi",
"tests/structure/gap-star-json-body.mq.md",
"1
1
7",
);
}
#[test]
fn structure_footnote_index() {
assert_out(
@@ -1181,6 +1179,83 @@ fn lib_log_uuid_zh_smoke() {
assert_out("tests/lib/日志-标识-烟测.mq.md", "INFO 中文信息\n36");
}
#[test]
fn lib_datetime_smoke() {
assert_out(
"tests/lib/datetime-smoke.mq.md",
"1970-01-01T00:00:00Z\n1970-01-01T08:00:00+08:00\n1577905445\n86400\n1970-01-01",
);
}
#[test]
fn lib_datetime_smoke_zh() {
assert_out(
"tests/lib/日期时间-烟测.mq.md",
"1970-01-01T00:00:00Z\n1970-01-01T08:00:00+08:00\n1577905445",
);
}
#[test]
fn lib_url_smoke() {
assert_out(
"tests/lib/url-smoke.mq.md",
"https\nexample.com\n8443\n/a/b\nx=1&y=2\nfrag\nhi x\n1\n2\na=1&a=2&b=hi%20x",
);
}
#[test]
fn lib_url_smoke_zh() {
assert_out(
"tests/lib/地址-烟测.mq.md",
"https\nexample.com\n/a\nx=1",
);
}
#[test]
fn lib_toml_html_encoding_smoke() {
assert_out(
"tests/lib/toml-html-encoding-smoke.mq.md",
"marqdo\n2\n1\na&lt;b&gt;&amp;&quot;c&#x27;\na<b>&\"c'\nNBSWY3DP\nhello",
);
}
#[test]
fn lib_toml_html_encoding_smoke_zh() {
assert_out(
"tests/lib/配置表-超文本-编码-烟测.mq.md",
"中文\nTrue\na&lt;b&gt;\nNBUQ====",
);
}
#[test]
fn lib_fs_mid2_smoke() {
assert_out(
"tests/lib/fs-mid2-smoke.mq.md",
"1\nTrue\nFalse\n3\na.txt\nnested\nnested/b.txt\nFalse",
);
}
#[test]
fn lib_fs_mid2_smoke_zh() {
assert_out("tests/lib/文件-mid2-烟测.mq.md", "True\n2\nFalse");
}
#[test]
fn lib_stats_log_smoke() {
assert_out(
"tests/lib/stats-log-smoke.mq.md",
"2.5\n2.5\n1.2909944487\nINFO hi user=alice n=3",
);
}
#[test]
fn lib_stats_log_smoke_zh() {
assert_out(
"tests/lib/统计-日志-烟测.mq.md",
"3\nINFO 你好 谁=甲 n=1",
);
}
#[test]
fn lib_re_smoke() {
assert_out(
@@ -1246,6 +1321,16 @@ fn lib_sys_cwd() {
assert_out("tests/lib/sys-cwd.mq.md", "cwd-ok");
}
#[test]
fn lib_sys_exec_capture() {
assert_out(
"tests/lib/sys-exec-capture.mq.md",
"exec-code-ok
capture-code-ok
capture-stdout-ok",
);
}
#[test]
fn lib_sys_dotenv() {
assert_out("tests/lib/sys-dotenv.mq.md", "loaded");
@@ -1952,6 +2037,172 @@ fn ext_web_middleware_live() {
let _ = child.wait();
}
#[test]
fn ext_web_proxy_invoke_smoke() {
let status = Command::new("cargo")
.args(["build", "-p", "marqdo_plugin_web"])
.current_dir(env!("CARGO_MANIFEST_DIR"))
.status()
.expect("build web plugin");
assert!(status.success(), "failed to build marqdo_plugin_web");
assert_out(
"tests/ext/web-proxy-invoke-smoke.mq.md",
"proxy-table-ok
proxy-stream-ok
proxy-env-ok
proxy-method-ok
invoke-table-ok
invoke-method-ok",
);
}
#[test]
fn ext_web_hosting_live() {
// Mock upstream SSE on 18142 + Marqdo web proxy/invoke on 18141.
let status = Command::new("cargo")
.args(["build", "-p", "marqdo_plugin_web"])
.current_dir(env!("CARGO_MANIFEST_DIR"))
.status()
.expect("build web plugin");
assert!(status.success(), "failed to build marqdo_plugin_web");
use std::io::{Read, Write};
use std::net::TcpListener;
let upstream = TcpListener::bind("127.0.0.1:18142").expect("bind mock upstream");
upstream.set_nonblocking(true).ok();
let upstream_thread = std::thread::spawn(move || {
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(20);
loop {
if std::time::Instant::now() > deadline {
break;
}
match upstream.accept() {
Ok((mut sock, _)) => {
let mut buf = [0u8; 4096];
let _ = sock.set_read_timeout(Some(std::time::Duration::from_secs(2)));
let _ = sock.read(&mut buf);
let body = "data: {\"id\":1}\n\ndata: {\"id\":2}\n\n";
let resp = format!(
"HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{}",
body.len(),
body
);
let _ = sock.write_all(resp.as_bytes());
let _ = sock.flush();
}
Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock => {
std::thread::sleep(std::time::Duration::from_millis(50));
}
Err(_) => break,
}
}
});
let script = "tests/ext/web-hosting-live-server.mq.md";
let bin = env!("CARGO_BIN_EXE_marqdo");
let mut child = Command::new(bin)
.args(["run", script])
.current_dir(env!("CARGO_MANIFEST_DIR"))
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.expect("spawn hosting live server");
let base = "http://127.0.0.1:18141";
let ready = std::time::Instant::now();
loop {
if ready.elapsed().as_secs() > 15 {
let _ = child.kill();
panic!("hosting live server did not start in time");
}
let out = Command::new("curl")
.args(["-s", "-o", "/dev/null", "-w", "%{http_code}", &format!("{base}/")])
.output()
.expect("curl probe");
if String::from_utf8_lossy(&out.stdout) == "200" {
break;
}
std::thread::sleep(std::time::Duration::from_millis(300));
}
// Proxy SSE: expect both data frames.
let out = Command::new("curl")
.args([
"-s",
"-N",
"-X",
"POST",
&format!("{base}/proxy/sse"),
"-H",
"Content-Type: application/json",
"-d",
"{}",
])
.output()
.expect("curl proxy");
let body = String::from_utf8_lossy(&out.stdout);
assert!(
body.contains("data: {\"id\":1}") && body.contains("data: {\"id\":2}"),
"proxy sse body={body}"
);
// Invoke twice: demo.echo nests a web.page plugin call; the second request
// must still see GLOBAL_HOST (listen outer frame) — regression for 0.3.7.
for (msg, label) in [("hi", "first"), ("again", "second")] {
let out = Command::new("curl")
.args([
"-s",
"-X",
"POST",
&format!("{base}/api/echo"),
"-H",
"Content-Type: application/json",
"-d",
&format!(r#"{{"msg":"{msg}"}}"#),
])
.output()
.expect("curl invoke");
let body = String::from_utf8_lossy(&out.stdout);
assert!(
body.contains("\"ok\"")
&& body.contains(&format!("\"msg\":\"{msg}\""))
&& !body.contains("no active host context"),
"invoke {label} body={body}"
);
}
let _ = child.kill();
let _ = child.wait();
let _ = upstream_thread.join();
}
#[test]
fn ext_web_db_cross_module_smoke() {
let status = Command::new("cargo")
.args(["build", "-p", "marqdo_plugin_web"])
.current_dir(env!("CARGO_MANIFEST_DIR"))
.status()
.expect("build web plugin");
assert!(status.success(), "failed to build marqdo_plugin_web");
assert_out("tests/ext/web-db-cross-module-smoke.mq.md", "cross-db-ok");
}
#[test]
fn ext_agent_mcp_server_smoke() {
let status = Command::new("cargo")
.args(["build", "-p", "marqdo_plugin_agent"])
.current_dir(env!("CARGO_MANIFEST_DIR"))
.status()
.expect("build agent plugin");
assert!(status.success(), "failed to build marqdo_plugin_agent");
assert_out(
"tests/ext/agent-mcp-server-smoke.mq.md",
"mcp-tool-ok
mcp-fn-ok",
);
}
#[test]
fn ext_web_security_smoke() {
let status = Command::new("cargo")
+23
View File
@@ -0,0 +1,23 @@
---
title: lib/datetime smoke
import dt:lib/datetime.mq.md
---
# main
*m = > dt.from_unix unix=0*
> print text=`m`[^iso]
*sh = > dt.in_zone dt=`m` zone="Asia/Shanghai"*
> print text=`sh`[^iso]
*p = > dt.parse text="2020-01-02T03:04:05+08:00"*
*u = > dt.to_unix dt=`p`*
> print text=`u`
*n = > dt.add dt=`m` days=1*
*nu = > dt.to_unix dt=`n`*
> print text=`nu`
*d = > dt.format dt=`sh` style="date"*
> print text=`d`
+32
View File
@@ -0,0 +1,32 @@
---
title: fs mid2 M9 smoke
import fs:lib/fs.mq.md
---
# main
> fs.make_dirs path="m9-tree/nested"
> fs.write_text path="m9-tree/a.txt" text="A"
> fs.write_text path="m9-tree/nested/b.txt" text="B"
*st = > fs.stat path="m9-tree/a.txt"*
> print text=`st`[^size]
> print text=`st`[^is_file]
> print text=`st`[^is_dir]
*w = > fs.walk path="m9-tree"*
*n = > len `w`*
> print text=`n`
*p0 = > at value=`w` index=0*
*p1 = > at value=`w` index=1*
*p2 = > at value=`w` index=2*
> print text=`p0`
> print text=`p1`
> print text=`p2`
> fs.remove_tree path="m9-tree"
*gone = > fs.exists path="m9-tree"*
> print text=`gone`
+33
View File
@@ -0,0 +1,33 @@
---
title: lib/stats + log fields M10 smoke
import stats:lib/stats.mq.md
import log:lib/log.mq.md
---
# main
`xs` =
| |
|---|
| 1 |
| 2 |
| 3 |
| 4 |
*m = > stats.mean list=`xs`*
> print text=`m`
*med = > stats.median list=`xs`*
> print text=`med`
*s = > stats.stdev list=`xs`*
> print text=`s`
`fld` =
| user | n |
|------|---|
| alice | 3 |
> log.info text="hi" fields=`fld`
+28
View File
@@ -0,0 +1,28 @@
---
title: sys.exec capture=True returns stdout map
description: GAP-09 optional capture.
import sys:lib/sys.mq.md
import json:lib/json.mq.md
---
# main
*code = > sys.exec cmd="true"*
1. `code` == 0
> print text=exec-code-ok
2. *
> print text=exec-code-fail
*args = > json.parse text=["-c","printf hi-capture"]*
*out = > sys.exec cmd="sh" args=args capture=True*
*c = out[^code]*
*s = out[^stdout]*
1. `c` == 0
> print text=capture-code-ok
2. *
> print text=capture-code-fail
1. `s` == "hi-capture"
> print text=capture-stdout-ok
2. *
> print text=capture-stdout-fail
+24
View File
@@ -0,0 +1,24 @@
---
title: lib/toml html encoding M8 smoke
import toml:lib/toml.mq.md
import html:lib/html.mq.md
import enc:lib/encoding.mq.md
---
# main
*cfg = > toml.parse text="name = \"marqdo\"\ncount = 2\n[pkg]\nver = \"1\"\n"*
> print text=`cfg`[^name]
> print text=`cfg`[^count]
*pkg = `cfg`[^pkg]*
> print text=`pkg`[^ver]
*e = > html.escape text="a<b>&\"c'"*
> print text=`e`
*u = > html.unescape text=`e`*
> print text=`u`
*b = > enc.base32_encode text="hello"*
> print text=`b`
*back = > enc.base32_decode text=`b`*
> print text=`back`
+25
View File
@@ -0,0 +1,25 @@
---
title: lib/url smoke
import url:lib/url.mq.md
---
# main
*u = > url.parse text="https://example.com:8443/a/b?x=1&y=2#frag"*
> print text=`u`[^scheme]
> print text=`u`[^host]
> print text=`u`[^port]
> print text=`u`[^path]
> print text=`u`[^query]
> print text=`u`[^fragment]
*q = > url.query_parse text="a=1&b=hi%20x&a=2"*
> print text=`q`[^b]
*as = `q`[^a]*
*a0 = > at value=`as` index=0*
*a1 = > at value=`as` index=1*
> print text=`a0`
> print text=`a1`
*s = > url.query_stringify map=`q`*
> print text=`s`
+12
View File
@@ -0,0 +1,12 @@
---
title: lib/地址 烟测
导入 址:lib/地址.mq.md
---
# main
*u = > 址.解析 内容="https://example.com/a?x=1"*
> print text=`u`[^scheme]
> print text=`u`[^host]
> print text=`u`[^path]
> print text=`u`[^query]
+22
View File
@@ -0,0 +1,22 @@
---
title: 文件 Mid2 M9 烟测
导入 文件:lib/文件.mq.md
---
# main
> 文件.建目录树 路径="m9-zh/子"
> 文件.写文本 路径="m9-zh/a.txt" 内容="A"
*st = > 文件.状态 路径="m9-zh/a.txt"*
> print text=`st`[^is_file]
*w = > 文件.遍历 路径="m9-zh"*
*n = > 长度 `w`*
> print text=`n`
> 文件.删目录树 路径="m9-zh"
*gone = > 文件.存在 路径="m9-zh"*
> print text=`gone`
+16
View File
@@ -0,0 +1,16 @@
---
title: lib/日期时间 烟测
导入 日时:lib/日期时间.mq.md
---
# main
*m = > 日时.自戳 秒=0*
> print text=`m`[^iso]
*sh = > 日时.时区 时刻=`m` 区="Asia/Shanghai"*
> print text=`sh`[^iso]
*p = > 日时.解析 内容="2020-01-02T03:04:05+08:00"*
*u = > 日时.成戳 时刻=`p`*
> print text=`u`
+25
View File
@@ -0,0 +1,25 @@
---
title: lib/统计 日志 M10 烟测
导入 统计:lib/统计.mq.md
导入 日志:lib/日志.mq.md
---
# main
`xs` =
| |
|---|
| 2 |
| 4 |
*m = > 统计.均值 列表=`xs`*
> print text=`m`
`fld` =
| 谁 | n |
|----|---|
| 甲 | 1 |
> 日志.信息 内容="你好" 字段=`fld`
@@ -0,0 +1,18 @@
---
title: lib/配置表 超文本 编码 烟测
导入 配置:lib/配置表.mq.md
导入 超:lib/超文本.mq.md
导入 编:lib/编码.mq.md
---
# main
*cfg = > 配置.解析 内容="name = \"中文\"\nok = true\n"*
> print text=`cfg`[^name]
> print text=`cfg`[^ok]
*e = > 超.转义 内容="a<b>"*
> print text=`e`
*b = > 编.基32编码 内容="hi"*
> print text=`b`
Loaded 100 of 101 files, more files were not shown because too many files have changed in this diff. Show more