ext/web W5: multipart upload receive and download

Add app.upload/download routes over # storage, offline media validate/save
helpers, form type=file + enctype, and curl live gold coverage.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
cflmyandCursor committed 2026-08-28 11:01:18 +08:00
1 parent 034f52b7e7
commit a900fd550f
18 files changed
+1214 -15

No files matched your search

Generated
+33
View File
@@ -191,6 +191,7 @@ dependencies = [
"matchit",
"memchr",
"mime",
"multer",
"percent-encoding",
"pin-project-lite",
"serde_core",
@@ -537,6 +538,15 @@ version = "1.18.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "252afb9ae5eaa683babdc6a068b3f5726eb19e05070c731f9b2a23a7c3e8ed34"
[[package]]
name = "encoding_rs"
version = "0.8.35"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75030f3c4f45dafd7586dd6780965a8c7e8e285a5ecb86713e63a79c5b2766f3"
dependencies = [
"cfg-if",
]
[[package]]
name = "errno"
version = "0.3.14"
@@ -1132,6 +1142,23 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "multer"
version = "3.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "83e87776546dc87511aa5ee218730c92b666d7264ab6ed41f9d215af9cd5224b"
dependencies = [
"bytes",
"encoding_rs",
"futures-util",
"http",
"httparse",
"memchr",
"mime",
"spin",
"version_check",
]
[[package]]
name = "normalize-line-endings"
version = "0.3.0"
@@ -1777,6 +1804,12 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "spin"
version = "0.9.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3763264f6b73151db08c50ff20d7d8a0b8796e021cdea7ceedad07b80155fa0e"
[[package]]
name = "stringprep"
version = "0.1.5"
+2 -2
View File
@@ -117,7 +117,7 @@
| A2 | **中间件管道**(`app.configure` / `应用.装配`:CORS、安全头、压缩、请求体限制统一挂载) | ✅ **已实现(W1)**:`middleware.rs` 装配 GFM 表并 `app.layer` 挂到 axum | `plugins/web` | **P0 ✅** |
| A3 | **CORS 支持**(`Access-Control-Allow-Origin` 可配) | ✅ **已实现(W1)**:`跨域` 表 `|允许来源|方法|头|暴露头|凭证|` | `plugins/web` | P1 ✅ |
| A4 | **安全响应头**(CSP / X-Frame-Options / X-Content-Type-Options / HSTS / Referrer-Policy / 去 `X-Powered-By`) | ✅ **已实现(W1)**:`安全` 表 `|头|值|` | `plugins/web` | P1 ✅ |
| A5 | **文件上传接收**(multipart extractor → 校验类型/大小 → 落盘)与**下载**(`Content-Disposition`) | 头像/附件/图片上传是博客/CMS 刚需 | `plugins/web` | P1 |
| A5 | **文件上传接收**(multipart extractor → 校验类型/大小 → 落盘)与**下载**(`Content-Disposition`) | ✅ **已实现(W5)**:`app.upload` / `app.download`;`file:` storage;离线 + curl live 金样 | `plugins/web` | P1 ✅ |
| A6 | **HTTPS / TLS**(rustls 终止或反代提示) | 生产部署安全 | `plugins/web`(或文档指引反代) | P2 |
| A7 | **gzip / br 压缩** + **ETag / Cache-Control 缓存头** | gzip ✅ **已实现(W1,`压缩=真`)**;ETag/Cache-Control 待补 | `plugins/web`(tower-http 现成) | P2 |
| A8 | **访问日志**(请求方法/路径/状态/耗时) | 排障、审计 | `plugins/web`(tower-http TraceLayer) | P2 |
@@ -179,7 +179,7 @@
| **W2** | 数据层深化 | 连接池/busy_timeout/WAL + 事务 API + 结果集裸查询 + 分页(offset) + OR/IN/BETWEEN + 聚合 count | B1 B2 B3 B4 B5 B8 ✅ **已完成** |
| **W3** | 安全硬化 | 密码哈希 + CSRF + 会话持久化 + CSPRNG session id + cookie 增强 + 登录限速/锁定 | C1–C7 |
| **W4** | 内容站点标配 | SEO 元数据 + sitemap/robots + RSS + Markdown 渲染 + 分页导航 UI + 标签/分类 | D1–D6 |
| **W5** | 上传与媒体 | multipart 文件上传接收 + 落盘 + 下载 + 图片库 | A5 D9 |
| **W5** | 上传与媒体 | multipart 文件上传接收 + 落盘 + 下载 + 图片库 | A5 D9 ✅ **A5 done**(D9 相册 UI 后续) |
| **W6** | 进阶 | 全文搜索(FTS5) + 迁移机制 + 评论系统 + 草稿/发布 + WS 广播 | B6 B9 D7 D8 + WS 增强 |
### 4.2 边界判定要点(严格遵守既有约束)
+3 -2
View File
@@ -2,7 +2,7 @@
| | |
|---|---|
| 状态 | **W0–W4 + 安全硬化 + 内容标配 W4c 已落地** |
| 状态 | **W0–W5 + 安全硬化 + 内容标配 W4c 已落地** |
| 日期 | 2026-08-28 |
| **锁定设计** | [design/ext-web.md](../design/ext-web.md) |
| 相关 | [ext-cli.md](../design/ext-cli.md) · [ext-abi.md](../design/ext-abi.md) · [ext-llm.md](../design/ext-llm.md) |
@@ -30,8 +30,9 @@
| **W3.5** | 人写面首版:`web.assemble` + `` `表`.`字段` `` + `examples/man-write-site` | **done**(编排袋仍可再瘦) |
| **W4** | Postgres;Redis / S3 驱动(见 [ext-web-drivers.md](../design/ext-web-drivers.md)) | **done** |
| **W4c** | 内容标配:SEO / Markdown / RSS / 分页 UI(见 [web-net-capabilities.md](../design/web-net-capabilities.md)) | **done** |
| **W5** | 上传与媒体:multipart 接收 + 落盘 + 下载(见 [web-net-capabilities.md](../design/web-net-capabilities.md) A5) | **done** |
验收金样:`tests/ext/web-smoke.mq.md`、`tests/ext/web-security-smoke.mq.md`、`tests/ext/web-drivers-smoke.mq.md`、`ext_cli_add_web`。
验收金样:`tests/ext/web-smoke.mq.md`、`tests/ext/web-security-smoke.mq.md`、`tests/ext/web-drivers-smoke.mq.md`、`tests/ext/web-upload-smoke.mq.md`、`ext_cli_add_web`。
---
+53 -1
View File
@@ -417,7 +417,7 @@ Tables stay as data; `configure` assembles them. 配置即数据、装配即函
## listen
Serve `/`, routed pages, `/_part/{id}` (home) and `{path}/_part/{id}` (routes), `/_form/{id}` (from mounts + page embeds), optional `/static` (or custom mount), and optional `/admin`.
Serve `/`, routed pages, `/_part/{id}` (home) and `{path}/_part/{id}` (routes), `/_form/{id}` (from mounts + page embeds), optional `/static` (or custom mount), optional `/admin`, and any `upload` / `download` routes.
**> web_listen app=`self`**
@@ -450,6 +450,27 @@ Register an RSS 2.0 feed at `path` (e.g. `/feed.xml`) backed by a DB table.
**> web_app_route_rss app=`self` path=`path` table=`table` limit=`limit` order=`order` title=`title` link=`link` description=`description`**
## upload
+ `path`=/_upload
+ `field`=file
+ `storage`
+ `prefix`=uploads/
+ `max_bytes`=5242880
+ `types`=None
Mount `POST path` for multipart file upload. `storage` is a `# storage` handle or `file:…` / `s3://…` URL. Optional `types` is a `|type|ext|` / `|类型|扩展名|` allowlist (or a MIME CSV string). Success returns JSON `{ok,key,size,content_type}`.
**> web_app_upload app=`self` path=`path` field=`field` storage=`storage` prefix=`prefix` max_bytes=`max_bytes` types=`types`**
## download
+ `path`=/_media/{*key}
+ `storage`
+ `disposition`=attachment
Mount `GET path` to stream an object. Path must capture `key` (use `{*key}` for nested keys). `disposition` is `attachment` or `inline`.
**> web_app_download app=`self` path=`path` storage=`storage` disposition=`disposition`**
# auth
+ `users`
+ `session_ttl`=3600
@@ -569,6 +590,37 @@ Provide either `body` (text) or `path` (local file to upload).
*url = self[^url]*
**> web_storage_list url=`url` prefix=`prefix`**
# media
+ `storage`=None
Offline helpers for upload validation and saving into `# storage` (also used by HTTP `app.upload`).
> ensure_plugin
**> web_media_new storage=`storage`**
## validate
+ `filename`
+ `content_type`="application/octet-stream"
+ `size`
+ `max_bytes`=5242880
+ `types`=None
**> web_upload_validate filename=`filename` content_type=`content_type` size=`size` max_bytes=`max_bytes` types=`types`**
## save
+ `path`
+ `key`=None
+ `content_type`="application/octet-stream"
+ `prefix`=uploads/
+ `storage`=None
*st = storage*
1. `st` == None
*st = self[^storage]*
2. *
**> web_upload_save storage=`st` path=`path` key=`key` content_type=`content_type` prefix=`prefix`**
# ws
+ `timeout_sec`=30
+53 -1
View File
@@ -399,7 +399,7 @@ SEO / OpenGraph 元数据表(`|键|值|`)。如 `title`、`description`、`c
## 监听
提供 `/`、路由页、首页 `/_part/{id}`、路由 `{路径}/_part/{id}`、`/_form/{id}`、可选静态目录,以及可选 `/admin`。
提供 `/`、路由页、首页 `/_part/{id}`、路由 `{路径}/_part/{id}`、`/_form/{id}`、可选静态目录、可选 `/admin`,以及已挂载的 `上传` / `下载` 路由。
**> web_listen app=`self`**
@@ -432,6 +432,27 @@ SEO / OpenGraph 元数据表(`|键|值|`)。如 `title`、`description`、`c
**> web_app_route_rss app=`self` path=`路径` table=`表` limit=`上限` order=`排序` title=`标题` link=`链接` description=`描述`**
## 上传
+ `路径`=/_upload
+ `字段`=file
+ `存储`
+ `前缀`=uploads/
+ `最大字节`=5242880
+ `类型`=None
挂载 `POST 路径` 接收 multipart 文件。`存储` 为 `# 存储` 句柄或 `file:…` / `s3://…` URL。可选 `类型` 为 `|类型|扩展名|` 允许表(或 MIME CSV)。成功返回 JSON `{ok,key,size,content_type}`。
**> web_app_upload app=`self` path=`路径` field=`字段` storage=`存储` prefix=`前缀` max_bytes=`最大字节` types=`类型`**
## 下载
+ `路径`=/_media/{*key}
+ `存储`
+ `处置`=attachment
挂载 `GET 路径` 流式返回对象。路径须捕获 `key`(嵌套键用 `{*key}`)。`处置` 为 `attachment` 或 `inline`。
**> web_app_download app=`self` path=`路径` storage=`存储` disposition=`处置`**
# 鉴权
+ `用户表`
+ `会话时长`=3600
@@ -551,6 +572,37 @@ SEO / OpenGraph 元数据表(`|键|值|`)。如 `title`、`description`、`c
*url = self[^url]*
**> web_storage_list url=`url` prefix=`前缀`**
# 媒体
+ `存储`=None
上传校验与落盘辅助(HTTP `应用.上传` 亦用同一套规则)。
> 确保插件
**> web_media_new storage=`存储`**
## 校验
+ `文件名`
+ `类型`="application/octet-stream"
+ `大小`
+ `最大字节`=5242880
+ `允许`=None
**> web_upload_validate filename=`文件名` content_type=`类型` size=`大小` max_bytes=`最大字节` types=`允许`**
## 保存
+ `路径`
+ `键`=None
+ `类型`="application/octet-stream"
+ `前缀`=uploads/
+ `存储`=None
*st = 存储*
1. `st` == None
*st = self[^storage]*
2. *
**> web_upload_save storage=`st` path=`路径` key=`键` content_type=`类型` prefix=`前缀`**
# 实时
+ `超时秒`=30
+1 -1
View File
@@ -14,7 +14,7 @@ crate-type = ["cdylib"]
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "sync", "time", "signal"] }
axum = { version = "0.8", features = ["ws", "json", "query"] }
axum = { version = "0.8", features = ["ws", "json", "query", "multipart"] }
tower-http = { version = "0.6", features = ["fs", "cors", "compression-gzip", "limit", "set-header", "trace", "propagate-header"] }
rusqlite = { version = "0.32", features = ["bundled"] }
tokio-tungstenite = "0.24"
+40 -3
View File
@@ -488,8 +488,17 @@ pub fn render_body(
));
}
body.push_str(&format!(
"<form method=\"post\" action=\"{}\">",
esc(&post_to)
"<form method=\"post\" action=\"{}\"{}>",
esc(&post_to),
if fields.iter().any(|f| {
f.get("type")
.and_then(|v| v.as_str())
.is_some_and(|t| t.eq_ignore_ascii_case("file"))
}) {
" enctype=\"multipart/form-data\""
} else {
""
}
));
if let Some(token) = csrf.filter(|s| !s.is_empty()) {
body.push_str(&format!(
@@ -541,7 +550,7 @@ pub fn render_body(
));
} else {
let input_type = match ty {
"number" | "email" | "url" | "checkbox" => ty,
"number" | "email" | "url" | "checkbox" | "file" => ty,
_ => "text",
};
if ty == "checkbox" {
@@ -552,6 +561,12 @@ pub fn render_body(
if checked { " checked" } else { "" },
req_attr
));
} else if ty == "file" {
body.push_str(&format!(
"<input type=\"file\" name=\"{}\"{}/>",
esc(name),
req_attr
));
} else {
body.push_str(&format!(
"<input type=\"{}\" name=\"{}\" value=\"{}\"{}/>",
@@ -658,3 +673,25 @@ pub fn submit(form: &Value, data: &Value, db_url: &str) -> Result<Value, String>
"redirect": redirect,
}))
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn file_field_adds_multipart_enctype() {
let form = json!({
"table": "x",
"fields": [
{"name": "title", "label": "Title", "type": "text"},
{"name": "file", "label": "File", "type": "file"},
]
});
let html = render(&form, "up", None, None, None);
assert!(html.contains("enctype=\"multipart/form-data\""), "{html}");
assert!(html.contains("type=\"file\""), "{html}");
assert!(html.contains("name=\"file\""), "{html}");
}
}
+211 -2
View File
@@ -5,9 +5,10 @@ use std::net::SocketAddr;
use std::path::PathBuf;
use std::sync::Arc;
use axum::extract::{Form, Path, Query, State};
use axum::extract::{Form, Multipart, Path, Query, State};
use axum::http::{header, HeaderValue, StatusCode};
use axum::response::{Html, IntoResponse, Redirect, Response};
use axum::routing::get;
use axum::routing::{get, post};
use axum::Router;
use serde::Deserialize;
use serde_json::{json, Map, Value};
@@ -21,6 +22,23 @@ use crate::rate_limit;
use crate::render;
use crate::rss;
use crate::session;
use crate::storage;
use crate::upload;
#[derive(Clone)]
pub struct UploadRoute {
pub field: String,
pub storage_url: String,
pub prefix: String,
pub max_bytes: u64,
pub types: Option<Value>,
}
#[derive(Clone)]
pub struct DownloadRoute {
pub storage_url: String,
pub disposition: String,
}
#[derive(Clone)]
pub struct AppState {
@@ -34,6 +52,8 @@ pub struct AppState {
auth_users: Option<Value>,
session_ttl: u64,
cookie_secure: bool,
upload_routes: HashMap<String, UploadRoute>,
download_routes: HashMap<String, DownloadRoute>,
}
fn collect_page_forms(
@@ -73,6 +93,8 @@ pub fn listen(
cookie_secure: bool,
ws_routes: HashMap<String, bool>,
rss_routes: HashMap<String, Value>,
upload_routes: HashMap<String, UploadRoute>,
download_routes: HashMap<String, DownloadRoute>,
middleware: &Middleware,
) -> Result<Value, String> {
let mut form_owners = HashMap::new();
@@ -96,6 +118,8 @@ pub fn listen(
auth_users,
session_ttl,
cookie_secure,
upload_routes: upload_routes.clone(),
download_routes: download_routes.clone(),
});
let mut app = Router::new()
@@ -196,6 +220,32 @@ pub fn listen(
);
}
let mut upload_paths: Vec<String> = upload_routes.keys().cloned().collect();
upload_paths.sort();
for path in upload_paths {
let route_path = path.clone();
app = app.route(
&path,
post(move |State(st): State<Arc<AppState>>, headers: axum::http::HeaderMap, multipart: Multipart| {
let route_path = route_path.clone();
async move { upload_post(st, headers, route_path, multipart).await }
}),
);
}
let mut download_paths: Vec<String> = download_routes.keys().cloned().collect();
download_paths.sort();
for path in download_paths {
let route_path = path.clone();
app = app.route(
&path,
get(move |State(st): State<Arc<AppState>>, Path(params): Path<HashMap<String, String>>| {
let route_path = route_path.clone();
async move { download_get(st, &route_path, params) }
}),
);
}
let mount = normalize_static_mount(static_mount);
if let Some(dir) = static_dir {
if !dir.is_dir() {
@@ -489,6 +539,165 @@ fn validate_csrf_post(
Ok((sid, csrf, set_cookie, posted))
}
fn json_err(status: StatusCode, msg: &str) -> Response {
let body = json!({ "ok": false, "error": msg });
(status, axum::Json(body)).into_response()
}
async fn upload_post(
st: Arc<AppState>,
headers: axum::http::HeaderMap,
route_path: String,
mut multipart: Multipart,
) -> Response {
let cfg = match st.upload_routes.get(&route_path) {
Some(c) => c.clone(),
None => return json_err(StatusCode::NOT_FOUND, "upload route not found"),
};
let had_cookie = cookie_from_headers(&headers).is_some();
let (sid, _csrf, set_cookie) = resolve_session(&headers);
let mut csrf_token = String::new();
let mut file_name = String::new();
let mut file_ct = String::from("application/octet-stream");
let mut file_bytes: Option<Vec<u8>> = None;
loop {
let field = match multipart.next_field().await {
Ok(Some(f)) => f,
Ok(None) => break,
Err(e) => return json_err(StatusCode::BAD_REQUEST, &format!("multipart: {e}")),
};
let name = field.name().unwrap_or("").to_string();
if name == "_csrf" {
csrf_token = field.text().await.unwrap_or_default();
continue;
}
if name != cfg.field {
// Drain unrelated fields.
let _ = field.bytes().await;
continue;
}
file_name = field
.file_name()
.map(|s| s.to_string())
.unwrap_or_default();
if let Some(ct) = field.content_type() {
file_ct = ct.to_string();
}
match field.bytes().await {
Ok(b) => file_bytes = Some(b.to_vec()),
Err(e) => return json_err(StatusCode::BAD_REQUEST, &format!("read file: {e}")),
}
}
// Existing session cookie → require CSRF; brand-new anonymous upload → open.
if had_cookie && !session::validate_csrf(&sid, &csrf_token) {
return json_err(StatusCode::FORBIDDEN, "Invalid or missing CSRF token");
}
let bytes = match file_bytes {
Some(b) if !b.is_empty() => b,
_ => return json_err(StatusCode::BAD_REQUEST, "missing file field"),
};
let size = bytes.len() as u64;
let check = match upload::validate(
&file_name,
&file_ct,
size,
cfg.max_bytes,
cfg.types.as_ref(),
) {
Ok(v) => v,
Err(e) => return json_err(StatusCode::BAD_REQUEST, &e),
};
if check.get("ok").and_then(|v| v.as_bool()) != Some(true) {
let err = check
.get("error")
.and_then(|v| v.as_str())
.unwrap_or("validation failed");
return json_err(StatusCode::BAD_REQUEST, err);
}
let ct = check
.get("content_type")
.and_then(|v| v.as_str())
.unwrap_or(&file_ct);
let key = match upload::make_key(&cfg.prefix, &file_name) {
Ok(k) => k,
Err(e) => return json_err(StatusCode::BAD_REQUEST, &e),
};
let saved = match storage::put_bytes(&cfg.storage_url, &key, &bytes, ct) {
Ok(v) => v,
Err(e) => return json_err(StatusCode::INTERNAL_SERVER_ERROR, &e),
};
let mut body = json!({
"ok": true,
"key": key,
"size": size,
"content_type": ct,
});
if let Some(obj) = body.as_object_mut() {
if let Some(sz) = saved.get("size") {
obj.insert("size".into(), sz.clone());
}
}
let mut resp = (StatusCode::OK, axum::Json(body)).into_response();
append_set_cookie(&mut resp, set_cookie);
resp
}
fn download_get(
st: Arc<AppState>,
route_path: &str,
params: HashMap<String, String>,
) -> Response {
let cfg = match st.download_routes.get(route_path) {
Some(c) => c.clone(),
None => return json_err(StatusCode::NOT_FOUND, "download route not found"),
};
let key = params
.get("key")
.or_else(|| params.values().next())
.map(|s| s.trim_start_matches('/').to_string())
.unwrap_or_default();
if key.is_empty() {
return json_err(StatusCode::BAD_REQUEST, "missing key");
}
let got = match storage::read_bytes(&cfg.storage_url, &key) {
Ok(Some(v)) => v,
Ok(None) => return json_err(StatusCode::NOT_FOUND, "not found"),
Err(e) => return json_err(StatusCode::INTERNAL_SERVER_ERROR, &e),
};
let (bytes, content_type, filename) = got;
let disp = if cfg.disposition.eq_ignore_ascii_case("inline") {
"inline"
} else {
"attachment"
};
let safe_name = filename.replace('"', "_");
let mut resp = (
StatusCode::OK,
[
(
header::CONTENT_TYPE,
HeaderValue::from_str(&content_type)
.unwrap_or_else(|_| HeaderValue::from_static("application/octet-stream")),
),
(
header::CONTENT_DISPOSITION,
HeaderValue::from_str(&format!("{disp}; filename=\"{safe_name}\""))
.unwrap_or_else(|_| HeaderValue::from_static("attachment")),
),
],
bytes,
)
.into_response();
let _ = &mut resp;
resp
}
fn admin_gate(st: &AppState, cookie_header: Option<&str>) -> Option<Response> {
if !st.admin {
return Some(Html(admin_shell(st, "Admin", None, "<p class=\"flash err\">Admin is disabled for this app.</p>")).into_response());
+212
View File
@@ -16,6 +16,7 @@ mod rss;
mod session;
mod storage;
mod table;
mod upload;
mod ws;
use crate::table::as_css_named;
@@ -444,6 +445,131 @@ web_ffi!(web_app_route_rss, |args: &Value| {
Ok(app)
});
fn storage_url_arg(args: &Value, key: &str) -> Result<String, String> {
if let Some(s) = arg_str_opt(args, key) {
return Ok(s.to_string());
}
let v = args
.get(key)
.ok_or_else(|| format!("missing `{key}`"))?;
if let Some(u) = v.get("url").and_then(|x| x.as_str()) {
return Ok(u.to_string());
}
Err(format!("`{key}` must be a storage url string or storage handle"))
}
web_ffi!(web_app_upload, |args: &Value| {
let mut app = args.get("app").cloned().unwrap_or(json!({}));
let path = normalize_route_path(arg_str(args, "path").unwrap_or("/_upload"))?;
let field = arg_str_opt(args, "field").unwrap_or("file").to_string();
let storage_url = storage_url_arg(args, "storage")?;
let prefix = arg_str_opt(args, "prefix").unwrap_or("uploads/").to_string();
let max_bytes = args
.get("max_bytes")
.and_then(|v| v.as_u64().or_else(|| v.as_i64().map(|i| i as u64)))
.unwrap_or(5_242_880);
let types = args.get("types").cloned().filter(|v| !matches!(v, Value::Null));
let obj = app
.as_object_mut()
.ok_or_else(|| "app must be a map".to_string())?;
let mut upload_routes = obj
.get("upload_routes")
.and_then(|v| v.as_object())
.cloned()
.unwrap_or_default();
upload_routes.insert(
path,
json!({
"field": field,
"storage_url": storage_url,
"prefix": prefix,
"max_bytes": max_bytes,
"types": types,
}),
);
obj.insert("upload_routes".into(), Value::Object(upload_routes));
Ok(app)
});
web_ffi!(web_app_download, |args: &Value| {
let mut app = args.get("app").cloned().unwrap_or(json!({}));
let path = arg_str(args, "path").unwrap_or("/_media/{*key}");
let path = {
let s = path.trim();
if s.is_empty() {
return Err("download path is empty".into());
}
let mut p = if s.starts_with('/') {
s.to_string()
} else {
format!("/{s}")
};
while p.len() > 1 && p.ends_with('/') {
p.pop();
}
p
};
if !path.contains('{') {
return Err("download path must include a `{key}` or `{*key}` capture".into());
}
let storage_url = storage_url_arg(args, "storage")?;
let disposition = arg_str_opt(args, "disposition")
.unwrap_or("attachment")
.to_string();
let obj = app
.as_object_mut()
.ok_or_else(|| "app must be a map".to_string())?;
let mut download_routes = obj
.get("download_routes")
.and_then(|v| v.as_object())
.cloned()
.unwrap_or_default();
download_routes.insert(
path,
json!({
"storage_url": storage_url,
"disposition": disposition,
}),
);
obj.insert("download_routes".into(), Value::Object(download_routes));
Ok(app)
});
web_ffi!(web_upload_validate, |args: &Value| {
let filename = arg_str(args, "filename")?;
let content_type = arg_str_opt(args, "content_type").unwrap_or("application/octet-stream");
let size = args
.get("size")
.and_then(|v| v.as_u64().or_else(|| v.as_i64().map(|i| i as u64)))
.ok_or_else(|| "missing `size`".to_string())?;
let max_bytes = args
.get("max_bytes")
.and_then(|v| v.as_u64().or_else(|| v.as_i64().map(|i| i as u64)))
.unwrap_or(5_242_880);
let types = args.get("types");
upload::validate(filename, content_type, size, max_bytes, types)
});
web_ffi!(web_upload_save, |args: &Value| {
let storage_url = if let Ok(u) = storage_url_arg(args, "storage") {
u
} else if let Ok(u) = storage_url_arg(args, "url") {
u
} else {
arg_str(args, "storage_url")?.to_string()
};
let path = arg_str(args, "path")?;
let key = arg_str_opt(args, "key");
let content_type = arg_str_opt(args, "content_type");
let prefix = arg_str_opt(args, "prefix");
upload::save(&storage_url, key, path, content_type, prefix)
});
web_ffi!(web_media_new, |args: &Value| {
let storage = args.get("storage").cloned().unwrap_or(Value::Null);
Ok(json!({ "_type": "media", "storage": storage }))
});
web_ffi!(web_compose_form, |args: &Value| {
let page = args.get("page").cloned().unwrap_or(json!({}));
let form = args
@@ -1049,6 +1175,8 @@ web_ffi!(web_listen, |args: &Value| {
session_ttl,
ws_routes,
rss_routes,
upload_routes,
download_routes,
middleware,
cookie_secure,
) = if args.get("page").is_some() || args.get("host").is_some() {
@@ -1094,6 +1222,8 @@ web_ffi!(web_listen, |args: &Value| {
session_ttl,
HashMap::new(),
HashMap::new(),
HashMap::new(),
HashMap::new(),
middleware::Middleware::default(),
cookie_secure,
)
@@ -1149,6 +1279,63 @@ web_ffi!(web_listen, |args: &Value| {
rss_routes.insert(k.clone(), v.clone());
}
}
let mut upload_routes = HashMap::new();
if let Some(obj) = app.get("upload_routes").and_then(|v| v.as_object()) {
for (k, v) in obj {
let field = v
.get("field")
.and_then(|x| x.as_str())
.unwrap_or("file")
.to_string();
let storage_url = v
.get("storage_url")
.and_then(|x| x.as_str())
.unwrap_or("")
.to_string();
let prefix = v
.get("prefix")
.and_then(|x| x.as_str())
.unwrap_or("uploads/")
.to_string();
let max_bytes = v
.get("max_bytes")
.and_then(|x| x.as_u64().or_else(|| x.as_i64().map(|i| i as u64)))
.unwrap_or(5_242_880);
let types = v.get("types").cloned().filter(|t| !matches!(t, Value::Null));
upload_routes.insert(
k.clone(),
http::UploadRoute {
field,
storage_url,
prefix,
max_bytes,
types,
},
);
}
}
let mut download_routes = HashMap::new();
if let Some(obj) = app.get("download_routes").and_then(|v| v.as_object()) {
for (k, v) in obj {
let storage_url = v
.get("storage_url")
.and_then(|x| x.as_str())
.unwrap_or("")
.to_string();
let disposition = v
.get("disposition")
.and_then(|x| x.as_str())
.unwrap_or("attachment")
.to_string();
download_routes.insert(
k.clone(),
http::DownloadRoute {
storage_url,
disposition,
},
);
}
}
let static_dir = app
.get("static_dir")
.and_then(|v| v.as_str())
@@ -1204,6 +1391,8 @@ web_ffi!(web_listen, |args: &Value| {
session_ttl,
ws_routes,
rss_routes,
upload_routes,
download_routes,
middleware,
cookie_secure,
)
@@ -1230,6 +1419,8 @@ web_ffi!(web_listen, |args: &Value| {
cookie_secure,
ws_routes,
rss_routes,
upload_routes,
download_routes,
&middleware,
)
});
@@ -1319,6 +1510,27 @@ pub unsafe extern "C" fn marqdo_plugin_init(host: *const MarqdoHostApi) -> c_int
"app,path,table,limit,order,title,link,description",
web_app_route_rss as PluginFn,
),
(
"web_app_upload",
"app,path,field,storage,prefix,max_bytes,types",
web_app_upload as PluginFn,
),
(
"web_app_download",
"app,path,storage,disposition",
web_app_download as PluginFn,
),
(
"web_upload_validate",
"filename,content_type,size,max_bytes,types",
web_upload_validate as PluginFn,
),
(
"web_upload_save",
"storage,path,key,content_type,prefix",
web_upload_save as PluginFn,
),
("web_media_new", "storage", web_media_new as PluginFn),
(
"web_style",
"name,table",
+57 -3
View File
@@ -84,17 +84,71 @@ pub fn put(
) -> Result<Value, String> {
let bytes = load_bytes(body, path)?;
let ct = content_type.unwrap_or("application/octet-stream");
put_bytes(url, key, &bytes, ct)
}
/// Persist raw bytes (multipart upload / binary save).
pub fn put_bytes(
url: &str,
key: &str,
bytes: &[u8],
content_type: &str,
) -> Result<Value, String> {
let ct = if content_type.trim().is_empty() {
"application/octet-stream"
} else {
content_type.trim()
};
if is_file(url) {
let root = file_root(url)?;
let dest = path_for(&root, key)?;
if let Some(parent) = dest.parent() {
fs::create_dir_all(parent).map_err(|e| e.to_string())?;
}
fs::write(&dest, &bytes).map_err(|e| e.to_string())?;
fs::write(&dest, bytes).map_err(|e| e.to_string())?;
let _ = fs::write(ctype_path(&dest), ct);
Ok(json!({ "ok": true, "key": key, "size": bytes.len() }))
Ok(json!({ "ok": true, "key": key, "size": bytes.len(), "content_type": ct }))
} else if is_s3(url) {
s3_put_http(url, key, &bytes, ct)
s3_put_http(url, key, bytes, ct)
} else {
Err("unknown storage backend".into())
}
}
/// Read object bytes for HTTP download. Returns `(bytes, content_type, filename)`.
pub fn read_bytes(url: &str, key: &str) -> Result<Option<(Vec<u8>, String, String)>, String> {
let filename = key
.rsplit('/')
.next()
.filter(|s| !s.is_empty())
.unwrap_or("download")
.to_string();
if is_file(url) {
let root = file_root(url)?;
let dest = path_for(&root, key)?;
if !dest.is_file() {
return Ok(None);
}
let bytes = fs::read(&dest).map_err(|e| e.to_string())?;
let ct = fs::read_to_string(ctype_path(&dest))
.unwrap_or_else(|_| "application/octet-stream".into());
Ok(Some((bytes, ct, filename)))
} else if is_s3(url) {
let cfg = S3Conf::parse(url)?;
let target = cfg.object_url(key)?;
let (status, ct, body) = cfg.request_bytes("GET", &target)?;
if status == 404 {
return Ok(None);
}
if !(200..300).contains(&status) {
return Err(format!("s3 get HTTP {status}"));
}
let content_type = if ct.trim().is_empty() {
"application/octet-stream".into()
} else {
ct
};
Ok(Some((body, content_type, filename)))
} else {
Err("unknown storage backend".into())
}
+264
View File
@@ -0,0 +1,264 @@
//! HTTP upload validation and safe object keys (W5).
use serde_json::{json, Value};
use std::fs;
use std::time::{SystemTime, UNIX_EPOCH};
/// Validate size / MIME / extension against an optional allowlist table.
///
/// `types` may be:
/// - `null` / missing → allow any
/// - string CSV of MIME types (`image/png,image/jpeg`)
/// - GFM-style rows: `|类型|扩展名|` / `|type|ext|` (ext may be comma-separated)
pub fn validate(
filename: &str,
content_type: &str,
size: u64,
max_bytes: u64,
types: Option<&Value>,
) -> Result<Value, String> {
if size > max_bytes {
return Ok(json!({
"ok": false,
"error": format!("file too large: {size} > {max_bytes}"),
}));
}
let name = filename.trim();
if name.is_empty() {
return Ok(json!({ "ok": false, "error": "empty filename" }));
}
if name.contains("..") || name.contains('/') || name.contains('\\') {
// Basename only is expected from clients; reject path fragments.
return Ok(json!({ "ok": false, "error": "unsafe filename" }));
}
let ct = content_type.trim().to_ascii_lowercase();
let ct = ct.split(';').next().unwrap_or(&ct).trim();
let ext = name
.rsplit('.')
.next()
.filter(|e| !e.is_empty() && *e != name)
.unwrap_or("")
.to_ascii_lowercase();
if let Some(t) = types {
if !types_allow(t, ct, &ext) {
return Ok(json!({
"ok": false,
"error": format!("type not allowed: content_type={ct} ext={ext}"),
}));
}
}
Ok(json!({
"ok": true,
"filename": name,
"content_type": if ct.is_empty() { "application/octet-stream" } else { ct },
"size": size,
"ext": ext,
}))
}
fn types_allow(types: &Value, content_type: &str, ext: &str) -> bool {
match types {
Value::Null => true,
Value::String(s) => {
let s = s.trim();
if s.is_empty() {
return true;
}
s.split(',')
.map(|p| p.trim().to_ascii_lowercase())
.filter(|p| !p.is_empty())
.any(|p| p == content_type)
}
Value::Array(rows) => {
if rows.is_empty() {
return true;
}
for row in rows {
if row_allows(row, content_type, ext) {
return true;
}
}
false
}
Value::Object(obj) => {
// Shapes:
// 1) Columnar GFM: `{类型: [mime…], 扩展名: [ext…]}` (Marqdo default)
// 2) Single allowlist row: `{类型|type, 扩展名|ext}` with string values
// 3) MIME→ext map: `{ "image/png": "png", … }`
let col_mime = obj
.get("类型")
.or_else(|| obj.get("type"))
.or_else(|| obj.get("mime"));
let col_ext = obj
.get("扩展名")
.or_else(|| obj.get("ext"))
.or_else(|| obj.get("extension"));
if let (Some(Value::Array(mimes)), Some(exts_v)) = (col_mime, col_ext) {
let exts: Vec<String> = match exts_v {
Value::Array(a) => a
.iter()
.map(|x| x.as_str().unwrap_or("").to_ascii_lowercase())
.collect(),
Value::String(s) => vec![s.to_ascii_lowercase()],
_ => Vec::new(),
};
if mimes.is_empty() {
return true;
}
for (i, mime_v) in mimes.iter().enumerate() {
let mime = mime_v
.as_str()
.unwrap_or("")
.trim()
.to_ascii_lowercase();
let row_ext = exts.get(i).map(|s| s.as_str()).unwrap_or("");
let mime_ok = mime.is_empty() || mime == content_type;
let ext_ok = if row_ext.is_empty() {
true
} else {
row_ext
.split(',')
.map(|e| e.trim().trim_start_matches('.'))
.any(|e| e == ext)
};
if mime_ok && ext_ok {
return true;
}
}
return false;
}
if col_mime.is_some() || col_ext.is_some() {
return row_allows(&Value::Object(obj.clone()), content_type, ext);
}
if obj.is_empty() {
return true;
}
for (mime, exts_v) in obj {
let mime = mime.trim().to_ascii_lowercase();
let exts = match exts_v {
Value::String(s) => s.clone(),
Value::Array(a) => a
.iter()
.filter_map(|x| x.as_str())
.collect::<Vec<_>>()
.join(","),
_ => String::new(),
};
let exts = exts.trim().to_ascii_lowercase();
let mime_ok = mime.is_empty() || mime == content_type;
let ext_ok = if exts.is_empty() {
true
} else {
exts.split(',')
.map(|e| e.trim().trim_start_matches('.'))
.any(|e| e == ext)
};
if mime_ok && ext_ok {
return true;
}
}
false
}
_ => true,
}
}
fn row_allows(row: &Value, content_type: &str, ext: &str) -> bool {
let Some(obj) = row.as_object() else {
if let Some(s) = row.as_str() {
return s.trim().eq_ignore_ascii_case(content_type);
}
return false;
};
let mime = obj
.get("类型")
.or_else(|| obj.get("type"))
.or_else(|| obj.get("mime"))
.and_then(|v| v.as_str())
.unwrap_or("")
.trim()
.to_ascii_lowercase();
let exts = obj
.get("扩展名")
.or_else(|| obj.get("ext"))
.or_else(|| obj.get("extension"))
.and_then(|v| v.as_str())
.unwrap_or("")
.trim()
.to_ascii_lowercase();
let mime_ok = mime.is_empty() || mime == content_type;
let ext_ok = if exts.is_empty() {
true
} else {
exts.split(',')
.map(|e| e.trim().trim_start_matches('.'))
.any(|e| e == ext)
};
mime_ok && ext_ok
}
/// Build `prefix` + timestamp-random + sanitized basename.
pub fn make_key(prefix: &str, filename: &str) -> Result<String, String> {
let base = filename
.rsplit(['/', '\\'])
.next()
.unwrap_or(filename)
.trim();
if base.is_empty() {
return Err("empty filename".into());
}
let safe: String = base
.chars()
.map(|c| {
if c.is_ascii_alphanumeric() || c == '.' || c == '-' || c == '_' {
c
} else {
'_'
}
})
.collect();
if safe.contains("..") {
return Err("unsafe filename".into());
}
let mut pref = prefix.trim().trim_start_matches('/').replace('\\', "/");
if !pref.is_empty() && !pref.ends_with('/') {
pref.push('/');
}
if pref.contains("..") {
return Err("prefix must not contain `..`".into());
}
let secs = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
let mut rnd = [0u8; 4];
getrandom::getrandom(&mut rnd).map_err(|e| e.to_string())?;
let hex = hex::encode(rnd);
Ok(format!("{pref}{secs}-{hex}-{safe}"))
}
/// Offline save: read local `path` bytes into storage under `key` (or auto key).
pub fn save(
storage_url: &str,
key: Option<&str>,
path: &str,
content_type: Option<&str>,
prefix: Option<&str>,
) -> Result<Value, String> {
let bytes = fs::read(path).map_err(|e| format!("read `{path}`: {e}"))?;
let filename = path
.rsplit(['/', '\\'])
.next()
.unwrap_or("upload.bin");
let ct = content_type
.filter(|s| !s.is_empty())
.unwrap_or("application/octet-stream");
let object_key = match key.filter(|k| !k.is_empty()) {
Some(k) => k.to_string(),
None => make_key(prefix.unwrap_or("uploads/"), filename)?,
};
crate::storage::put_bytes(storage_url, &object_key, &bytes, ct)
}
+1
View File
@@ -0,0 +1 @@
not-allowed
+1
View File
@@ -0,0 +1 @@
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+1
View File
@@ -0,0 +1 @@
hello-upload
+2
View File
@@ -0,0 +1,2 @@
‰PNG


After

Width:  |  Height:  |  Size: 8 B

+31
View File
@@ -0,0 +1,31 @@
---
title: web upload live server (W5)
description: Listen with upload + download routes for curl gold.
import web:ext/web/web.mq.md
import plugin:lib/plugin.mq.md
import sys:lib/sys.mq.md
---
# main
*p = > plugin.native_path name="web"*
1. `p`
> plugin.load path=`p`
2. *
> sys.exit code=1
*blob = > web.storage url="file:tests/ext/web-fixtures/data/upload-live-blobs"*
`types` =
| 类型 | 扩展名 |
|------|--------|
| text/plain | txt |
| image/png | png |
*pg = > web.page title="upload-live"*
*app = > web.app page=`pg` port=18101*
*app = > `app`.configure body_limit=1048576*
*app = > `app`.upload path="/_upload" field="file" storage=`blob` prefix="live/" max_bytes=1024 types=`types`*
*app = > `app`.download path="/_media/{*key}" storage=`blob` disposition="attachment"*
> `app`.listen
+77
View File
@@ -0,0 +1,77 @@
---
title: web upload smoke (W5)
description: Offline validate + save into file: storage.
import web:ext/web/web.mq.md
import plugin:lib/plugin.mq.md
import sys:lib/sys.mq.md
---
# main
*p = > plugin.native_path name="web"*
1. `p`
> plugin.load path=`p`
2. *
> sys.exit code=1
`types` =
| 类型 | 扩展名 |
|------|--------|
| image/png | png |
| text/plain | txt |
*m = > web.media*
*ok = > `m`.validate filename="a.png" content_type="image/png" size=100 max_bytes=1000 types=`types`*
*ook = ok[^ok]*
1. `ook`
> print text=validate-ok
2. *
> print text=validate-fail
*bad = > `m`.validate filename="a.exe" content_type="application/octet-stream" size=100 max_bytes=1000 types=`types`*
*bok = bad[^ok]*
1. `bok`
> print text=reject-type-fail
2. *
> print text=reject-type-ok
*big = > `m`.validate filename="a.png" content_type="image/png" size=9000 max_bytes=1000 types=`types`*
*gok = big[^ok]*
1. `gok`
> print text=reject-size-fail
2. *
> print text=reject-size-ok
*blob = > web.storage url="file:tests/ext/web-fixtures/data/upload-blobs"*
*m = > web.media storage=`blob`*
*saved = > `m`.save path="tests/ext/web-fixtures/upload/sample.txt" content_type="text/plain" prefix="smoke/"*
*sok = saved[^ok]*
*skey = saved[^key]*
1. `sok`
> print text=save-ok
2. *
> print text=save-fail
*got = > `blob`.get key=`skey`*
*body = got[^body]*
1. `body` == "hello-upload"
> print text=save-roundtrip-ok
2. *
> print text=save-roundtrip-fail
<!-- form file field renders with multipart enctype -->
`fields` =
| name | label | type |
|------|-------|------|
| title | Title | text |
| file | File | file |
*frm = > web.form table="x" fields=`fields`*
*html = > `frm`.render id="up"*
1. `html`
> print text=form-file-ok
2. *
> print text=form-file-fail
+172
View File
@@ -1765,6 +1765,178 @@ s3-open-ok",
);
}
#[test]
fn ext_web_upload_smoke() {
let status = Command::new("cargo")
.args(["build", "-p", "marqdo_plugin_web"])
.current_dir(env!("CARGO_MANIFEST_DIR"))
.status()
.expect("build web plugin");
assert!(status.success(), "failed to build marqdo_plugin_web");
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/ext/web-fixtures/data/upload-blobs");
let _ = std::fs::remove_dir_all(&root);
assert_out(
"tests/ext/web-upload-smoke.mq.md",
"validate-ok
reject-type-ok
reject-size-ok
save-ok
save-roundtrip-ok
form-file-ok",
);
}
#[test]
fn ext_web_upload_live() {
let status = Command::new("cargo")
.args(["build", "-p", "marqdo_plugin_web"])
.current_dir(env!("CARGO_MANIFEST_DIR"))
.status()
.expect("build web plugin");
assert!(status.success(), "failed to build marqdo_plugin_web");
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/ext/web-fixtures/data/upload-live-blobs");
let _ = std::fs::remove_dir_all(&root);
let script = "tests/ext/web-upload-live-server.mq.md";
let bin = env!("CARGO_BIN_EXE_marqdo");
let mut child = Command::new(bin)
.args(["run", script])
.current_dir(env!("CARGO_MANIFEST_DIR"))
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.expect("spawn upload live server");
let base = "http://127.0.0.1:18101";
let ready = std::time::Instant::now();
loop {
if ready.elapsed().as_secs() > 10 {
let _ = child.kill();
panic!("upload live server did not start in time");
}
let out = Command::new("curl")
.args([
"-s",
"-o",
"/dev/null",
"-w",
"%{http_code}",
&format!("{base}/"),
])
.output()
.expect("curl probe");
if String::from_utf8_lossy(&out.stdout) == "200" {
break;
}
std::thread::sleep(std::time::Duration::from_millis(300));
}
let fixture = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/ext/web-fixtures/upload/sample.txt");
// Happy path multipart upload.
let out = Command::new("curl")
.args([
"-s",
"-w",
"\n%{http_code}",
"-F",
&format!("file=@{};type=text/plain", fixture.display()),
&format!("{base}/_upload"),
])
.output()
.expect("curl upload");
let body = String::from_utf8_lossy(&out.stdout);
let (json_part, code) = body.rsplit_once('\n').unwrap_or((&body, ""));
assert_eq!(code.trim(), "200", "upload body={body}");
assert!(
json_part.contains("\"ok\":true") && json_part.contains("\"key\""),
"upload json={json_part}"
);
let key = {
let v: serde_json::Value =
serde_json::from_str(json_part).expect("upload json parse");
v.get("key")
.and_then(|k| k.as_str())
.expect("key")
.to_string()
};
// Download with Content-Disposition.
let out = Command::new("curl")
.args([
"-s",
"-D",
"-",
"-o",
"/tmp/marqdo-upload-dl.txt",
&format!("{base}/_media/{key}"),
])
.output()
.expect("curl download");
let headers = String::from_utf8_lossy(&out.stdout).to_ascii_lowercase();
assert!(
headers.contains("content-disposition: attachment"),
"download headers={headers}"
);
let dl = std::fs::read_to_string("/tmp/marqdo-upload-dl.txt").expect("dl body");
assert_eq!(dl, "hello-upload");
// Reject wrong type.
let bad = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/ext/web-fixtures/upload/bad.bin");
let _ = std::fs::write(&bad, b"not-allowed");
let out = Command::new("curl")
.args([
"-s",
"-o",
"/dev/null",
"-w",
"%{http_code}",
"-F",
&format!("file=@{};type=application/octet-stream", bad.display()),
&format!("{base}/_upload"),
])
.output()
.expect("curl bad type");
assert_eq!(
String::from_utf8_lossy(&out.stdout).trim(),
"400",
"bad type should 400"
);
// Oversize (> max_bytes=1024).
let big = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/ext/web-fixtures/upload/big.txt");
let _ = std::fs::write(&big, vec![b'x'; 2048]);
let out = Command::new("curl")
.args([
"-s",
"-o",
"/dev/null",
"-w",
"%{http_code}",
"-F",
&format!("file=@{};type=text/plain", big.display()),
&format!("{base}/_upload"),
])
.output()
.expect("curl oversize");
assert_eq!(
String::from_utf8_lossy(&out.stdout).trim(),
"400",
"oversize should 400"
);
let _ = child.kill();
let _ = child.wait();
}
#[test]
fn ext_web_db_w2_smoke() {
// W2 data layer: transactions, connection pooling, pagination, query