Compare commits

...
4 Commits
Author SHA1 Message Date
cflmyandCursor 586975090b release: v1.0.7 — WS/网络边界加固
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-22 15:40:48 +08:00
cflmyandCursor 544805266b fix(wasm): mark marqdo-bridge.js as ESM for Node consumers
The canonical bridge uses ESM exports but, with no package.json in scope,
Node loads untyped .js as CommonJS — tests/wasm/smoke.mjs could not see
applyDomPatch et al (Named export not found). Scope type=module to the js
dir only; marqdo wasm build copies just the single .js file, so nothing
ships differently.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-22 15:27:09 +08:00
cflmyandCursor f3051fb536 fix(test): align P3 gold with quiet-admin gate semantics
b752cd4 stopped auto-gating admin_prefix when admin=false but did not
update the P3 gold, which still expected the removed /admin auto gate
(ng >= 2). Assert the new intent instead: exactly one explicit gate
registered with the declared path.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-22 15:20:48 +08:00
cflmyandCursor 86b183e2d7 fix(web): harden WebSocket and net host boundaries
- WS handshake: same-origin CheckOrigin instead of allow-all
- route_ws require_auth/需要登录: broadcast/room need a session by default,
  explicit opt-out for public feeds; unauthenticated handshakes get 401
- WS limits: 1 MiB per message, 256 replies per connect; transport errors
  return {ok:false, error} instead of an empty success; empty rooms GC'd
- net host: enforce the 8 MiB response cap while reading (incl. error and
  SSE error paths) so a peer cannot exhaust memory
- url_encode: %20 for space (URL segment semantics, not form +)
- docs: sync 网页 (route_ws 需要登录, connect caps) and CHANGELOG
- tests: gold expectations, live fixture opt-out, hub GC, ws auth gate

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-22 15:14:35 +08:00
24 changed files with 280 additions and 38 deletions

No files matched your search

+2
View File
@@ -124,6 +124,8 @@ jobs:
## Marqdo ${{ github.ref_name }}
### 亮点 / Highlights
- **v1.0.7**:WS/网络边界加固(同源握手、require_auth/需要登录、消息上限、HTTP 8 MiB 截断);`url_encode` `%20`
- **v1.0.7**: WS/net boundary hardening (same-origin, require_auth, message caps, 8 MiB HTTP cap); `url_encode` `%20`
- **v1.0.3**:Ubuntu PPA + 扩展 CDN(ext.marqdo.com);CLI/扩展可分发;noble 捆绑 Rust 工具链
- **v1.0.3**: Ubuntu PPA + extension CDN; independent CLI/ext packs; bundled Rust for noble
- **v1.0.1**:修复 Go `libweb` 动态路由 `{param}` 查询代入、详情 Markdown、CSS 数字误报
+26
View File
@@ -1,5 +1,31 @@
# Changelog
## v1.0.7 — 2026-09-22
### Highlights
**WS / 网络边界加固**:WebSocket 握手同源校验、broadcast/room 默认需登录(`require_auth`/`需要登录` 可显式关闭)、消息 1 MiB × 256 条上限、空 room 回收;HTTP 响应 8 MiB 在读流时截断;`url_encode` 空格 `%20`。本版 CLI 与扩展包 **同步为 1.0.7**。
- **安全默认**:浏览器 WS 握手仅同源;有状态 fan-out 端点未登录 401 / 公开直播流显式 `require_auth=False`
- **资源上限**:WS 单条 1 MiB、单连接 256 条;HTTP/SSE 8 MiB 读流截断——对端无法借大响应耗尽内存
- **测试**:P3 金样对齐 quiet-admin 门禁语义;wasm smoke 以 ESM 加载官方 bridge
```bash
# Ubuntu PPA / 源码
sudo add-apt-repository ppa:cflmy/marqdo && sudo apt update && sudo apt install marqdo
git checkout v1.0.7 && cargo build --release
marqdo ext add web
```
### Fixed
- **WS 安全默认**:握手改为同源校验(不再放行任意 Origin);`route_ws` 新增 `require_auth`/`需要登录`,broadcast/room 默认要求登录会话,公开直播流需显式设假,未认证握手 401。
- **WS 资源上限**:单条消息 1 MiB、单连接最多收集 256 条回复;传输错误/超时返回 `{ok:false, error}`,不再伪装空成功;空 room 自动回收(防 URL 派生 room 名耗尽内存)。
- **HTTP 响应上限**:8 MiB 上限在读流时截断(含错误路径与 SSE 错误路径),上游无法借超大响应耗尽内存。
### Changed
- `net.url_encode`:空格编码为 `%20`(原 `+`),URL 段语义下与字面 `+` 消歧义。
- `web`/`网页` 文档同步:`route_ws` 的 `require_auth`/`需要登录` 与 `ws.connect` 的收集上限语义。
## Unreleased
### Added
Generated
+2 -2
View File
@@ -572,7 +572,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "marqdo"
version = "1.0.3"
version = "1.0.7"
dependencies = [
"anyhow",
"assert_cmd",
@@ -603,7 +603,7 @@ dependencies = [
[[package]]
name = "marqdo-wasm"
version = "1.0.3"
version = "1.0.7"
dependencies = [
"marqdo",
"serde_json",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "marqdo"
version = "1.0.3"
version = "1.0.7"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo — Markdown markers as an interpreted programming language"
+4 -4
View File
@@ -98,7 +98,7 @@ marqdo catalog public -o .marqdo
---
## 现状(v1.0.3)
## 现状(v1.0.7)
- 映射与解释器:**v0.3 语法宪法**(`**` 代码 / `*` 返回 / 叙述声明);Phase I 树遍历 + 字节码后端;金样例在 `tests/` 与 `tests/markup-v03/`
- **对象**:`#` = 类型/构造,`##`+ = 函数/方法;见 [objects.md](doc/design/objects.md)
@@ -120,7 +120,7 @@ marqdo catalog public -o .marqdo
- **浏览器 Marqdo(WASM)**:`marqdo wasm build` + 官方 bridge 自启(作者零业务 JS;桥内可含列表/路由/storage/ws/文件/Canvas/音频/Observer/拖放);`lib/browser` + GFM;`web.client_embed`;示例 [browser-hello](examples/browser-hello/) · [browser-app](examples/browser-app/) · [browser-media](examples/browser-media/) · [web-client-site](examples/web-client-site/)([ADR 0002](doc/adr/0002-browser-marqdo-wasm.md) · [D](doc/roadmap/browser-wasm-d.md) · [E](doc/roadmap/browser-wasm-e.md) · [F](doc/roadmap/browser-wasm-f.md))
- 选型:[ADR 0001 — Rust](doc/adr/0001-implementation-language.md) · [ADR 0002 — 浏览器 WASM](doc/adr/0002-browser-marqdo-wasm.md)(C0–C5 完结,见 [roadmap/browser-wasm.md](doc/roadmap/browser-wasm.md))· [ADR 0003 — 异步效应](doc/adr/0003-browser-async-effects.md)
### 如何使用最新 Marqdo(v1.0.3)
### 如何使用最新 Marqdo(v1.0.7)
**安装解释器(任选其一)**
@@ -135,7 +135,7 @@ marqdo catalog public -o .marqdo
```bash
# 1) 源码安装(跟 tag 或 main)
git clone https://github.com/cflmy/marqdo.git && cd marqdo
git checkout v1.0.3 # 或留在 main
git checkout v1.0.7 # 或留在 main
cargo build --release
export PATH="$PWD/target/release:$PATH"
marqdo version
@@ -169,7 +169,7 @@ marqdo wasm build
# → dist/wasm/ … 见 examples/browser-hello/
# 6) Releases:Windows exe/zip/vsix;Linux CLI zip + native `.so` zip
# https://github.com/cflmy/marqdo/releases/tag/v1.0.3
# https://github.com/cflmy/marqdo/releases/tag/v1.0.7
```
开发期也可用 `cargo run -- …` 代替已安装的 `marqdo`:
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "marqdo-wasm"
version = "1.0.3"
version = "1.0.7"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo interpreter as WebAssembly (browser host bridge)"
+5
View File
@@ -0,0 +1,5 @@
{
"name": "marqdo-bridge",
"private": true,
"type": "module"
}
+8
View File
@@ -1,3 +1,11 @@
marqdo (1.0.7-1ppa1) noble; urgency=medium
* Marqdo 1.0.7: WS/net boundary hardening (same-origin handshake,
require_auth/需要登录, message caps, empty-room GC); HTTP 8 MiB cap
enforced while reading; url_encode space -> %20.
-- cflmy <pingan@cflmy.cn> Tue, 22 Sep 2026 15:40:00 +0800
marqdo (1.0.3-1ppa2) noble; urgency=medium
* Fix Launchpad build: dh_clean -Xvendor/ so cargo vendor checksums keep
+1 -1
View File
@@ -1 +1 @@
1.0.6
1.0.7
+4 -3
View File
@@ -800,10 +800,11 @@ Serve `/`, routed pages, `/_part/{id}` (home) and `{path}/_part/{id}` (routes),
+ `room_key`=None
+ `on_message`=None
+ `presence`=False
+ `require_auth`=None
Register a WebSocket endpoint at `path` (e.g. `/live` or `/chat/{id}`). `mode` is `echo` (default), `broadcast` (fan-out text to all sockets on this path), `room` (named room via `room_key`, supports `{id}` from the path), or `drain`. Legacy `echo=False` maps to `drain`. `on_message` is an optional `lib.member` hook (G-WS2): called before fan-out with `message`/`room`/`user`; return `{skip:true}` to drop or `{message:…}` to rewrite. `presence=True` also joins `room.presence` and emits join/leave JSON. Connect from a client with `web.ws.connect`.
Register a WebSocket endpoint at `path` (e.g. `/live` or `/chat/{id}`). `mode` is `echo` (default), `broadcast` (fan-out text to all sockets on this path), `room` (named room via `room_key`, supports `{id}` from the path), or `drain`. Legacy `echo=False` maps to `drain`. `on_message` is an optional `lib.member` hook (G-WS2): called before fan-out with `message`/`room`/`user`; return `{skip:true}` to drop or `{message:…}` to rewrite. `presence=True` also joins `room.presence` and emits join/leave JSON. Stateful `broadcast` and `room` endpoints require a logged-in session by default; set `require_auth=False` only for intentionally public live feeds. Browser handshakes are same-origin. Connect from a client with `web.ws.connect`.
*> web_app_route_ws app=`self` path=`path` echo=`echo` mode=`mode` room_key=`room_key` on_message=`on_message` presence=`presence`*
*> web_app_route_ws app=`self` path=`path` echo=`echo` mode=`mode` room_key=`room_key` on_message=`on_message` presence=`presence` require_auth=`require_auth`*
## auth
+ `users`
@@ -1160,7 +1161,7 @@ WebSocket client helper.
+ `message`=""
+ `headers`=None
Single request–response: connect to `url`, send `message`, collect all server text replies, close. Returns `{ok, messages}`.
Single request–response: connect to `url`, send `message`, collect up to 256 server text replies (each up to 1 MiB), close. Transport errors and timeouts return `{ok:false, error}` instead of looking like an empty successful reply.
**timeout = [timeout_sec](self)**
*> web_ws_connect url=`url` message=`message` headers=`headers` timeout_sec=`timeout`*
+4 -3
View File
@@ -768,10 +768,11 @@ SSR 顶栏刊名 Logo、主题切换钮与目录抽屉钮。交互写在 `client
+ `房间`=None
+ `消息钩子`=None
+ `在场`=假
+ `需要登录`=None
在 `路径`(如 `/live` 或 `/chat/{id}`)注册 WebSocket。`模式` 为 `echo`(默认)、`broadcast`、`room`(需 `房间` 键,可含 `{id}`)或 `drain`。旧参数 `回显=假` 等价于 `drain`。`消息钩子` 为可选 `库.成员`(G-WS2):广播前调用,入参含 `message`/`room`/`user`;返回 `{skip:真}` 丢弃或 `{message:…}` 改写。`在场=真` 时另进 `房间.presence` 并推送 join/leave。客户端用 `网页.实时` 的 `连接` 接入。
在 `路径`(如 `/live` 或 `/chat/{id}`)注册 WebSocket。`模式` 为 `echo`(默认)、`broadcast`、`room`(需 `房间` 键,可含 `{id}`)或 `drain`。旧参数 `回显=假` 等价于 `drain`。`消息钩子` 为可选 `库.成员`(G-WS2):广播前调用,入参含 `message`/`room`/`user`;返回 `{skip:真}` 丢弃或 `{message:…}` 改写。`在场=真` 时另进 `房间.presence` 并推送 join/leave。有状态的 `broadcast` 与 `room` 端点默认需登录会话;仅对有意公开的直播流设 `需要登录=假`。浏览器握手同源。客户端用 `网页.实时` 的 `连接` 接入。
*> web_app_route_ws app=`self` path=`路径` echo=`回显` mode=`模式` room_key=`房间` on_message=`消息钩子` presence=`在场`*
*> web_app_route_ws app=`self` path=`路径` echo=`回显` mode=`模式` room_key=`房间` on_message=`消息钩子` presence=`在场` require_auth=`需要登录`*
## 鉴权
+ `用户表`
@@ -1128,7 +1129,7 @@ WebSocket 客户端工具。
+ `消息`=""
+ `headers`=None
单次请求-响应:连接 `地址`,发送 `消息`,收集服务器全部文本回复后关闭。返回 `{ok, messages}`。
单次请求-响应:连接 `地址`,发送 `消息`,最多收集 256 条服务器文本回复(每条 ≤1 MiB)后关闭。传输错误与超时返回 `{ok:false, error}`,不再像空的成功回复。
**timeout = [timeout_sec](self)**
*> web_ws_connect url=`地址` message=`消息` headers=`headers` timeout_sec=`timeout`*
+2 -2
View File
@@ -9,7 +9,7 @@ English HTTP(S) wrappers. Optional headers / content_type via host aliases.
+ `url`
+ `headers`=None
HTTPS supported. Optional named arg `headers=` (map).
HTTPS supported. Optional named arg `headers=` (map). Responses are capped at 8 MiB before decoding, so an upstream cannot exhaust the interpreter's memory.
*> host_http_get url=`url` headers=`headers`*
@@ -56,7 +56,7 @@ Optional `body=` / `content_type=` / `headers=`.
## url_encode
+ `text`
Percent-encode a URL segment.
Percent-encode a URL segment. Spaces become `%20` (not form/query `+`), so literal `+` remains unambiguous in paths.
*> host_url_encode text=`text`*
+19 -2
View File
@@ -4,6 +4,7 @@ import (
"encoding/json"
"fmt"
"net/http"
"net/url"
"strings"
"github.com/gorilla/websocket"
@@ -12,8 +13,19 @@ import (
"github.com/marqdo/marqdo/plugins/web/internal/ws"
)
var wsUpgrader = websocket.Upgrader{
CheckOrigin: func(r *http.Request) bool { return true },
const maxWSMessageBytes = 1 << 20
var wsUpgrader = websocket.Upgrader{CheckOrigin: sameOrigin}
// Browsers send Origin for a cross-site WebSocket handshake. Permit clients
// without Origin, but reject browser origins that differ from this application.
func sameOrigin(r *http.Request) bool {
origin := r.Header.Get("Origin")
if origin == "" {
return true
}
u, err := url.Parse(origin)
return err == nil && strings.EqualFold(u.Host, r.Host)
}
func (st *state) mountWSRoutes(mux *http.ServeMux) {
@@ -57,11 +69,16 @@ func pathValuesOf(r *http.Request, routePath string) map[string]string {
}
func (st *state) handleWS(w http.ResponseWriter, r *http.Request, path string, spec ws.RouteSpec) {
if spec.RequireAuth && session.UsernameFromCookie(r.Header.Get("Cookie")) == "" {
http.Error(w, "WebSocket authentication required", http.StatusUnauthorized)
return
}
conn, err := wsUpgrader.Upgrade(w, r, nil)
if err != nil {
return
}
defer conn.Close()
conn.SetReadLimit(maxWSMessageBytes)
vals := pathValuesOf(r, path)
switch spec.Mode {
@@ -0,0 +1,55 @@
package httpx
import (
"net/http"
"net/http/httptest"
"testing"
"github.com/marqdo/marqdo/plugins/web/internal/ws"
)
func TestHandleWSRejectsUnauthenticatedHandshake(t *testing.T) {
st := &state{}
r := httptest.NewRequest("GET", "http://app.example/live", nil)
w := httptest.NewRecorder()
st.handleWS(w, r, "/live", ws.RouteSpec{Mode: ws.ModeBroadcast, RequireAuth: true})
if w.Code != http.StatusUnauthorized {
t.Fatalf("unauthenticated ws handshake code=%d, want 401", w.Code)
}
}
func TestHandleWSAllowsPublicRouteThroughAuthGate(t *testing.T) {
st := &state{}
r := httptest.NewRequest("GET", "http://app.example/live", nil)
w := httptest.NewRecorder()
// Public route: must not 401. Upgrade then fails on the recorder (no
// Hijacker), which is fine — only the auth gate is under test.
st.handleWS(w, r, "/live", ws.RouteSpec{Mode: ws.ModeBroadcast, RequireAuth: false})
if w.Code == http.StatusUnauthorized {
t.Fatalf("public ws route unexpectedly requires authentication")
}
}
func TestSameOriginWebSocketHandshake(t *testing.T) {
cases := []struct {
name string
origin string
want bool
}{
{name: "non-browser client", want: true},
{name: "same origin", origin: "https://app.example", want: true},
{name: "different origin", origin: "https://evil.example", want: false},
{name: "malformed origin", origin: "://bad", want: false},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
r := httptest.NewRequest("GET", "http://app.example/live", nil)
if tc.origin != "" {
r.Header.Set("Origin", tc.origin)
}
if got := sameOrigin(r); got != tc.want {
t.Fatalf("sameOrigin(%q)=%v, want %v", tc.origin, got, tc.want)
}
})
}
}
+13 -1
View File
@@ -9,6 +9,11 @@ import (
"github.com/gorilla/websocket"
)
const (
maxMessageBytes = 1 << 20
maxMessages = 256
)
// Connect dials url, sends one text message, collects text replies until close or timeout.
// Mirrors Rust ws::connect; errors are returned as {ok:false, error}.
func Connect(url, message string, headers map[string]any, timeoutSec uint64) map[string]any {
@@ -29,6 +34,7 @@ func Connect(url, message string, headers map[string]any, timeoutSec uint64) map
return map[string]any{"ok": false, "error": fmt.Sprintf("ws connect: %v", err)}
}
defer conn.Close()
conn.SetReadLimit(maxMessageBytes)
if err := conn.SetWriteDeadline(time.Now().Add(timeout)); err != nil {
return map[string]any{"ok": false, "error": fmt.Sprintf("ws send: %v", err)}
@@ -45,9 +51,15 @@ func Connect(url, message string, headers map[string]any, timeoutSec uint64) map
}
mt, data, err := conn.ReadMessage()
if err != nil {
break
if websocket.IsCloseError(err, websocket.CloseNormalClosure, websocket.CloseGoingAway) {
break
}
return map[string]any{"ok": false, "error": fmt.Sprintf("ws receive: %v", err)}
}
if mt == websocket.TextMessage {
if len(messages) == maxMessages {
return map[string]any{"ok": false, "error": "ws receive: too many messages"}
}
messages = append(messages, string(data))
}
if mt == websocket.CloseMessage {
+15
View File
@@ -42,7 +42,22 @@ func unsubscribe(path string, ch chan string) {
}
room.mu.Lock()
delete(room.subs, ch)
empty := len(room.subs) == 0
room.mu.Unlock()
if !empty {
return
}
// A room can be derived from a URL path. Remove empty rooms so a stream of
// one-off room names cannot leave metadata allocated for the process lifetime.
globalHub.mu.Lock()
if globalHub.room[path] == room {
room.mu.Lock()
if len(room.subs) == 0 {
delete(globalHub.room, path)
}
room.mu.Unlock()
}
globalHub.mu.Unlock()
}
// JoinRoom subscribes to a named room (G-WS1). Same backing store as path broadcast.
+32
View File
@@ -35,6 +35,38 @@ func TestBroadcastFanout(t *testing.T) {
unsubscribe(path, ch2)
}
func TestUnsubscribeRemovesEmptyRoom(t *testing.T) {
path := "/t-empty-room-gc"
ch := subscribe(path)
unsubscribe(path, ch)
globalHub.mu.Lock()
_, ok := globalHub.room[path]
globalHub.mu.Unlock()
if ok {
t.Fatalf("empty room %q was not garbage-collected", path)
}
}
func TestResubscribeDuringCleanupKeepsRoom(t *testing.T) {
path := "/t-room-rejoin"
ch := subscribe(path)
ch2 := subscribe(path)
unsubscribe(path, ch)
unsubscribe(path, ch2)
ch3 := subscribe(path)
Publish(path, "still-alive")
if got := readOne(ch3); got != "still-alive" {
t.Fatalf("re-subscribed chan got %q", got)
}
unsubscribe(path, ch3)
globalHub.mu.Lock()
_, ok := globalHub.room[path]
globalHub.mu.Unlock()
if ok {
t.Fatalf("empty room %q was not garbage-collected after rejoin", path)
}
}
func readOne(ch chan string) string {
select {
case s := <-ch:
+19 -4
View File
@@ -9,10 +9,11 @@ import (
// RouteSpec is one ws_routes entry (mode + optional room_key / on_message / presence).
type RouteSpec struct {
Mode Mode
RoomKey string
OnMessage string
Presence bool
Mode Mode
RoomKey string
OnMessage string
Presence bool
RequireAuth bool
}
// RouteWS registers path → {mode, room_key?, on_message?, presence?} on app.ws_routes.
@@ -28,6 +29,12 @@ func RouteWS(appBag map[string]any, path string, args map[string]any) (map[strin
}
onMsg := OnMessageFromArgs(args)
presence := PresenceFromArgs(args)
// Stateful fan-out routes are private by default. Echo and drain remain
// useful for public health checks and demos; authors can opt out explicitly.
requireAuth := mode == ModeBroadcast || mode == ModeRoom
if v, ok := first(args, "require_auth", "需要登录"); ok && v != nil {
requireAuth = boolish(v)
}
out := clone(appBag)
wsRoutes := map[string]any{}
if r, ok := out["ws_routes"].(map[string]any); ok {
@@ -43,6 +50,9 @@ func RouteWS(appBag map[string]any, path string, args map[string]any) (map[strin
if presence {
spec["presence"] = true
}
if requireAuth {
spec["require_auth"] = true
}
wsRoutes[path] = spec
out["ws_routes"] = wsRoutes
return out, nil
@@ -165,6 +175,11 @@ func parseRouteSpec(spec any) RouteSpec {
} else if p, ok := t["在场"]; ok {
out.Presence = boolish(p)
}
if a, ok := t["require_auth"]; ok {
out.RequireAuth = boolish(a)
} else if a, ok := t["需要登录"]; ok {
out.RequireAuth = boolish(a)
}
case string:
out.Mode = ParseMode(t)
default:
+21
View File
@@ -26,6 +26,27 @@ func TestRouteWSBroadcastMode(t *testing.T) {
if room["mode"] != "broadcast" {
t.Fatalf("bag mode=%v", room["mode"])
}
if room["require_auth"] != true {
t.Fatalf("broadcast must require authentication by default: %v", room)
}
}
func TestRouteWSPublicBroadcastOptOut(t *testing.T) {
a := app.New(nil)
out, err := ws.RouteWS(a, "/live", map[string]any{
"mode": "broadcast",
"require_auth": false,
})
if err != nil {
t.Fatal(err)
}
spec, err := ws.RouteSpecOf(out, "/live")
if err != nil {
t.Fatal(err)
}
if spec.RequireAuth {
t.Fatalf("explicit public route unexpectedly requires authentication: %+v", spec)
}
}
func TestRouteWSEchoDefault(t *testing.T) {
+1 -1
View File
@@ -258,7 +258,7 @@ static int register_core(void) {
if (host_register((char *)"web_app_sitemap", (char *)"app,path,base,table,loc,limit,items", web_app_sitemap) != 0) return 1;
if (host_register((char *)"web_app_robots", (char *)"app,body,sitemap", web_app_robots) != 0) return 1;
if (host_register((char *)"web_sitemap_build", (char *)"base,items", web_sitemap_build) != 0) return 1;
if (host_register((char *)"web_app_route_ws", (char *)"app,path,echo,mode,room_key,on_message,presence", web_app_route_ws) != 0) return 1;
if (host_register((char *)"web_app_route_ws", (char *)"app,path,echo,mode,room_key,on_message,presence,require_auth", web_app_route_ws) != 0) return 1;
if (host_register((char *)"web_ws_connect", (char *)"url,message,headers,timeout_sec", web_ws_connect) != 0) return 1;
if (host_register((char *)"web_cache_new", (char *)"url", web_cache_new) != 0) return 1;
if (host_register((char *)"web_cache_get", (char *)"url,key", web_cache_get) != 0) return 1;
+41 -10
View File
@@ -1,6 +1,6 @@
//! HTTP host primitives (HTTP and HTTPS via ureq).
use std::io::Write;
use std::io::{Read, Write};
#[cfg(feature = "net-host")]
use std::io::{BufRead, BufReader};
#[cfg(feature = "net-host")]
@@ -32,6 +32,26 @@ fn headers_from_value(headers: Option<&Value>) -> Result<Vec<(String, String)>,
}
}
/// Maximum response body accepted by the text HTTP helpers.
///
/// Read the limit from the stream, rather than after converting it to a `String`:
/// an untrusted peer must never be able to allocate an arbitrary response in memory.
#[cfg(feature = "net-host")]
const MAX_HTTP_BODY_BYTES: usize = 8 * 1024 * 1024;
#[cfg(feature = "net-host")]
fn read_response_text<R: Read>(reader: R) -> Result<String, String> {
let mut bytes = Vec::new();
reader
.take(MAX_HTTP_BODY_BYTES as u64 + 1)
.read_to_end(&mut bytes)
.map_err(|e| format!("http read body: {e}"))?;
if bytes.len() > MAX_HTTP_BODY_BYTES {
return Err("http response exceeds 8MiB limit".into());
}
Ok(String::from_utf8_lossy(&bytes).into_owned())
}
#[cfg(feature = "net-host")]
fn http_exchange(
method: &str,
@@ -74,16 +94,11 @@ fn http_exchange(
match resp {
Ok(r) => {
let status = r.status();
let text = r
.into_string()
.map_err(|e| format!("http read body: {e}"))?;
if text.len() > 8 * 1024 * 1024 {
return Err("http response exceeds 8MiB limit".into());
}
let text = read_response_text(r.into_reader())?;
Ok((status, text))
}
Err(ureq::Error::Status(code, r)) => {
let text = r.into_string().unwrap_or_default();
let text = read_response_text(r.into_reader())?;
Ok((code, text))
}
Err(e) => Err(format!("http {method} {url}: {e}")),
@@ -177,7 +192,9 @@ pub fn url_encode(text: &Value) -> Result<Value, String> {
b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'_' | b'.' | b'~' => {
(b as char).to_string()
}
b' ' => "+".into(),
// `url_encode` documents a URL *segment*. `+` is form/query
// encoding, where it is ambiguous with a literal plus in a path.
b' ' => "%20".into(),
_ => format!("%{b:02X}"),
})
.collect(),
@@ -809,7 +826,7 @@ pub fn http_post_sse(
]))
}
Err(ureq::Error::Status(code, r)) => {
let text = r.into_string().unwrap_or_default();
let text = read_response_text(r.into_reader())?;
Ok(Value::Map(vec![
("status".into(), Value::Int(code as i64)),
(
@@ -842,6 +859,20 @@ mod tests {
.any(|(k, v)| k == "Authorization" && v == "Bearer x"));
}
#[cfg(feature = "net-host")]
#[test]
fn response_limit_stops_before_unbounded_allocation() {
let body = vec![b'x'; MAX_HTTP_BODY_BYTES + 1];
let err = read_response_text(std::io::Cursor::new(body)).unwrap_err();
assert_eq!(err, "http response exceeds 8MiB limit");
}
#[test]
fn url_segment_encoding_keeps_space_and_plus_distinct() {
let encoded = url_encode(&Value::Text("a b+c".into())).unwrap();
assert!(matches!(encoded, Value::Text(text) if text == "a%20b%2Bc"));
}
#[test]
fn openai_sse_fixture_to_events() {
let fixture = "\
+2 -1
View File
@@ -90,7 +90,8 @@ import json:lib/json.mq.md
**app = > `app`.gate path="/write*" roles="admin,author"**
**gates = [gates](app)**
**ng = > len value=`gates`**
1. `ng` >= 2
**gpath = [path]([1](gates))**
1. `ng` == 1 and `gpath` == "/write*"
> print text=rbac-gate-ok
2. *
> print text=rbac-gate-fail
+1 -1
View File
@@ -16,5 +16,5 @@ import sys:lib/sys.mq.md
**pg = > web.page title="ws-live"**
**app = > web.app page=`pg` port=18112**
**app = > `app`.configure access_log=True**
**app = > `app`.route_ws path="/room" mode="broadcast"**
**app = > `app`.route_ws path="/room" mode="broadcast" require_auth=False**
> `app`.listen
+1 -1
View File
@@ -3735,7 +3735,7 @@ callable-ok",
#[test]
fn lib_net_encode() {
assert_out("tests/lib/net-encode.mq.md", "a+b");
assert_out("tests/lib/net-encode.mq.md", "a%20b");
}
#[test]