refactor: 精简 — 删除归档的 Rust web 插件(14,235 行)

审计结论:plugins/web-rust-archive/ 不是 Cargo workspace 成员、默认不构建、
无任何脚本/代码引用,官方 web 插件早已是 Go 版(plugins/web/ + libweb.so)。
仅作为"参考"留在树里,占 Rust 代码量约 18%。

删除内容:
- plugins/web-rust-archive/(31 个文件,14,235 行 Rust,528K)
- 更新三处指向:Cargo.toml 注释、plugins/web/README.md、ext-web-go-rewrite.md,
  说明历史保留在 git 中
- CHANGELOG Unreleased 记录此次精简

验证:cargo build --release 通过;cargo test --release 565 passed / 0 failed。

这是「不要让冗余代码毁坏 marqdo 设计」的第一步;后续按审计报告继续:
HostFn 元数据表合一(~400L)、plugin ABI 共用 SDK(~550L)、
catalog 复用 knowledge 助手(~140L)、view 路由表化(~130L)、
as_text 19→1(~90L)、view/html 字符串拆分与去重(~300L)。
This commit is contained in:
cflmy committed 2026-10-22 15:09:31 +08:00
1 parent 5a0f4ed747
commit 13012cac0e
30 files changed
+13 -14280

No files matched your search

+6
View File
@@ -4,10 +4,16 @@
### Added
- **M1 `()` 内调用语法**:`加(1)` / `加(1, 2)` / `加(加(1 2), 10)` / `加一 号(31)`(callee 可含空格,解决 D8)/ `txt.str_trim(" x ")`。`)` 是参数终止符,逗号可选,支持嵌套。括号形参数值按表达式解析,空格形(`> f k=v`)仍按文本,`source=print(2+2)` 不受影响。
### Fixed
- `加(31)` 曾被解析成 `unknown function 加(31`(末个位置实参吞掉 `)`)。
### Changed
- **精简**:删除 `plugins/web-rust-archive/`(14,235 行 Rust,非 workspace 成员、默认不构建;Go `libweb` 已取代)。Git 历史仍可追溯。
## v1.5.2 — 2026-10-18
### Highlights
+2 -2
View File
@@ -63,8 +63,8 @@ members = [
"plugins/linalg",
"crates/marqdo-wasm",
]
# Official web plugin is Go (`plugins/web/`); Rust sources kept as
# `plugins/web-rust-archive/` for reference (not a workspace member).
# Official web plugin is Go (`plugins/web/`). The legacy Rust web plugin was
# deleted from the tree (see CHANGELOG); git history keeps it if ever needed.
resolver = "2"
[dev-dependencies]
+2 -1
View File
@@ -347,7 +347,8 @@ plugins/web/ # Go module(替换原 Rust crate)
scripts/build-web-plugin.sh # go build -buildmode=c-shared -o …/libweb.so
```
原 Rust 源码:`plugins/web-rust-archive/`(**已移出 Cargo workspace**;参考用,默认不构建)。
原 Rust 源码:曾置于 `plugins/web-rust-archive/`(已移出 Cargo workspace,参考用);
该目录已于 v1.6 精简时从工作树移除,历史保留在 git 中。
### 9.2 产物与解析
-32
View File
@@ -1,32 +0,0 @@
[package]
name = "marqdo_plugin_web"
version = "0.1.0"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo web plugin (ABI v2) — HTTP, SQLite, page assemble"
publish = false
[lib]
name = "web"
crate-type = ["cdylib"]
[dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "sync", "time", "signal"] }
axum = { version = "0.8", features = ["ws", "json", "query", "multipart"] }
tower-http = { version = "0.6", features = ["fs", "cors", "compression-gzip", "limit", "set-header", "trace", "propagate-header"] }
rusqlite = { version = "0.32", features = ["bundled"] }
tokio-tungstenite = "0.24"
futures-util = "0.3"
argon2 = "0.5"
getrandom = "0.2"
pulldown-cmark = "0.12"
postgres = { version = "0.19", features = ["with-serde_json-1"] }
redis = { version = "0.27", default-features = true }
ureq = { version = "=2.9.7", default-features = false, features = ["tls"] }
reqwest = { version = "0.12", default-features = false, features = ["rustls-tls", "stream", "json"] }
bytes = "1"
hmac = "0.12"
sha2 = "0.10"
hex = "0.4"
-9
View File
@@ -1,9 +0,0 @@
# web-rust-archive
Former Rust/axum implementation of the Marqdo web plugin (`marqdo_plugin_web`).
**Status (W-G13):** archived reference only. The official plugin is Go under
[`plugins/web/`](../web/). This crate is **not** a Cargo workspace member and is
not built by CI or `marqdo ext add web`.
Do not evolve features here — see [ext-web-go-rewrite.md](../../doc/design/ext-web-go-rewrite.md) C5 / ADR 0004.
-607
View File
@@ -1,607 +0,0 @@
//! Site icons, `<head>` link/script assembly, and image HTML assembly.
//! Tables stay data; assembly is a function — 配置即数据、装配即函数.
use serde_json::{json, Map, Value};
use std::path::{Path, PathBuf};
use crate::table::normalize_ref;
#[derive(Clone, Debug)]
pub struct HeadLink {
pub rel: String,
pub href: String,
pub type_: String,
pub sizes: String,
pub media: String,
pub as_: String,
pub crossorigin: String,
/// When `Some(true)`, emit `defer` on classic scripts (not modules).
pub defer: Option<bool>,
pub async_: bool,
/// Query `?v=` bump for cache busting.
pub version: String,
}
#[derive(Clone, Debug)]
pub struct IconRoute {
pub url: String,
pub path: PathBuf,
pub content_type: String,
}
fn cell_str(v: &Value) -> String {
match v {
Value::Null => String::new(),
Value::String(s) => s.clone(),
Value::Bool(b) => b.to_string(),
Value::Number(n) => n.to_string(),
other => other.to_string(),
}
}
fn pick<'a>(m: &'a Map<String, Value>, keys: &[&str]) -> Option<&'a Value> {
keys.iter().find_map(|k| m.get(*k))
}
fn cell_boolish(v: &Value) -> Option<bool> {
match v {
Value::Bool(b) => Some(*b),
Value::Number(n) => Some(n.as_i64().unwrap_or(0) != 0),
Value::String(s) => {
let s = s.trim();
if s.is_empty() {
return None;
}
if matches!(
s,
"1" | "true" | "True" | "yes" | "on" | "真" | "是"
) {
Some(true)
} else if matches!(
s,
"0" | "false" | "False" | "no" | "off" | "假" | "否"
) {
Some(false)
} else {
None
}
}
_ => None,
}
}
fn row_bool(m: &Map<String, Value>, keys: &[&str]) -> Option<bool> {
pick(m, keys).and_then(cell_boolish)
}
fn row_str(m: &Map<String, Value>, keys: &[&str]) -> String {
pick(m, keys)
.map(|v| normalize_ref(&cell_str(v)))
.unwrap_or_default()
.trim()
.to_string()
}
/// Append `?v=` / `&v=` for cache busting.
pub fn href_with_version(href: &str, version: &str) -> String {
let version = version.trim();
if version.is_empty() {
return href.to_string();
}
if href.contains('?') {
format!("{href}&v={version}")
} else {
format!("{href}?v={version}")
}
}
fn esc(s: &str) -> String {
s.replace('&', "&amp;")
.replace('<', "&lt;")
.replace('>', "&gt;")
.replace('"', "&quot;")
}
/// Infer Content-Type from file extension.
pub fn mime_for_path(path: &str) -> &'static str {
let lower = path.to_ascii_lowercase();
if lower.ends_with(".ico") {
"image/x-icon"
} else if lower.ends_with(".png") {
"image/png"
} else if lower.ends_with(".svg") {
"image/svg+xml"
} else if lower.ends_with(".webp") {
"image/webp"
} else if lower.ends_with(".jpg") || lower.ends_with(".jpeg") {
"image/jpeg"
} else if lower.ends_with(".gif") {
"image/gif"
} else if lower.ends_with(".webmanifest") || lower.ends_with(".json") {
"application/manifest+json"
} else if lower.ends_with(".css") {
"text/css; charset=utf-8"
} else if lower.ends_with(".js") || lower.ends_with(".mjs") {
"text/javascript; charset=utf-8"
} else {
"application/octet-stream"
}
}
fn rows_of(table: &Value) -> Vec<Map<String, Value>> {
match table {
Value::Array(a) => a
.iter()
.filter_map(|v| v.as_object().cloned())
.collect(),
Value::Object(m) => {
// Columnar GFM: { 关系: […], 地址: […] }
let keys: Vec<String> = m.keys().cloned().collect();
if keys.is_empty() {
return Vec::new();
}
let len = m
.values()
.filter_map(|v| v.as_array().map(|a| a.len()))
.max()
.unwrap_or(0);
if len == 0 {
return vec![m.clone()];
}
let mut out = Vec::with_capacity(len);
for i in 0..len {
let mut row = Map::new();
for k in &keys {
if let Some(Value::Array(a)) = m.get(k) {
row.insert(k.clone(), a.get(i).cloned().unwrap_or(Value::Null));
} else if i == 0 {
row.insert(k.clone(), m.get(k).cloned().unwrap_or(Value::Null));
}
}
out.push(row);
}
out
}
_ => Vec::new(),
}
}
/// Normalize a Head resource table → list of HeadLink.
pub fn as_head_links(table: &Value) -> Vec<HeadLink> {
let mut out = Vec::new();
for m in rows_of(table) {
let rel = row_str(&m, &["关系", "rel", "Rel"]);
let href = row_str(
&m,
&["地址", "href", "src", "url", "Href", "Src", "URL"],
);
if href.is_empty() {
continue;
}
let rel = if rel.is_empty() {
"stylesheet".into()
} else {
rel
};
out.push(HeadLink {
rel,
href,
type_: row_str(&m, &["类型", "type", "Type"]),
sizes: row_str(&m, &["尺寸", "sizes", "Sizes"]),
media: row_str(&m, &["媒体", "media", "Media"]),
as_: row_str(&m, &["作为", "as", "As"]),
crossorigin: row_str(&m, &["跨域", "crossorigin", "crossOrigin"]),
defer: row_bool(&m, &["推迟", "defer", "Defer"]),
async_: row_bool(&m, &["异步", "async", "Async"]).unwrap_or(false),
version: row_str(&m, &["版本", "version", "Version", "v"]),
});
}
out
}
pub fn head_links_to_json(links: &[HeadLink]) -> Value {
Value::Array(
links
.iter()
.map(|l| {
let mut o = json!({
"rel": l.rel,
"href": l.href,
"type": l.type_,
"sizes": l.sizes,
"media": l.media,
"as": l.as_,
"crossorigin": l.crossorigin,
"async": l.async_,
"version": l.version,
});
if let Some(d) = l.defer {
o.as_object_mut()
.unwrap()
.insert("defer".into(), json!(d));
}
o
})
.collect(),
)
}
pub fn head_links_from_json(v: &Value) -> Vec<HeadLink> {
let Some(arr) = v.as_array() else {
return as_head_links(v);
};
let mut out = Vec::new();
for item in arr {
let Some(m) = item.as_object() else {
continue;
};
let href = row_str(m, &["href", "src", "url", "地址"]);
if href.is_empty() {
continue;
}
out.push(HeadLink {
rel: row_str(m, &["rel", "关系"]),
href,
type_: row_str(m, &["type", "类型"]),
sizes: row_str(m, &["sizes", "尺寸"]),
media: row_str(m, &["media", "媒体"]),
as_: row_str(m, &["as", "作为"]),
crossorigin: row_str(m, &["crossorigin", "跨域"]),
defer: row_bool(m, &["defer", "推迟"]),
async_: row_bool(m, &["async", "异步"]).unwrap_or(false),
version: row_str(m, &["version", "版本", "v"]),
});
}
out
}
/// Render HeadLink list to HTML (link / script tags).
pub fn render_head_links(links: &[HeadLink]) -> String {
render_head_links_with_version(links, "")
}
pub fn render_head_links_with_version(links: &[HeadLink], asset_version: &str) -> String {
let mut s = String::new();
for l in links {
let rel = l.rel.trim().to_ascii_lowercase();
let ver = if !l.version.trim().is_empty() {
l.version.as_str()
} else {
asset_version
};
let href = href_with_version(&l.href, ver);
if rel == "script" || rel == "module" {
let mut tag = String::from("<script");
let is_module = rel == "module" || l.type_ == "module";
if is_module {
tag.push_str(" type=\"module\"");
} else if !l.type_.is_empty() {
tag.push_str(&format!(" type=\"{}\"", esc(&l.type_)));
}
// Modules are deferred by browsers; classic scripts honor explicit defer.
if !is_module && l.defer == Some(true) {
tag.push_str(" defer");
}
if l.async_ {
tag.push_str(" async");
}
if !l.crossorigin.is_empty() {
tag.push_str(&format!(" crossorigin=\"{}\"", esc(&l.crossorigin)));
}
tag.push_str(&format!(" src=\"{}\"></script>", esc(&href)));
s.push_str(&tag);
continue;
}
let mut tag = format!("<link rel=\"{}\" href=\"{}\"", esc(&l.rel), esc(&href));
if !l.type_.is_empty() {
tag.push_str(&format!(" type=\"{}\"", esc(&l.type_)));
}
if !l.sizes.is_empty() {
tag.push_str(&format!(" sizes=\"{}\"", esc(&l.sizes)));
}
if !l.media.is_empty() {
tag.push_str(&format!(" media=\"{}\"", esc(&l.media)));
}
if !l.as_.is_empty() {
tag.push_str(&format!(" as=\"{}\"", esc(&l.as_)));
}
if !l.crossorigin.is_empty() {
tag.push_str(&format!(" crossorigin=\"{}\"", esc(&l.crossorigin)));
}
tag.push_str("/>");
s.push_str(&tag);
}
s
}
/// Module-level preview: assemble head table → HTML string.
pub fn make_head_html(table: &Value) -> String {
render_head_links(&as_head_links(table))
}
/// Normalize icons table → (icons json array, site_head links, file routes).
pub fn normalize_icons(table: &Value) -> (Value, Vec<HeadLink>, Vec<IconRoute>) {
let mut icons = Vec::new();
let mut head = Vec::new();
let mut routes = Vec::new();
let mut saw_favicon_url = false;
for m in rows_of(table) {
let path = row_str(&m, &["路径", "path", "file", "Path"]);
let mut rel = row_str(&m, &["关系", "rel", "Rel"]);
if rel.is_empty() {
rel = "icon".into();
}
let mut type_ = row_str(&m, &["类型", "type", "Type"]);
let sizes = row_str(&m, &["尺寸", "sizes", "Sizes"]);
let mut url = row_str(&m, &["地址", "url", "href", "Href", "URL"]);
if type_.is_empty() && !path.is_empty() {
type_ = mime_for_path(&path).to_string();
}
if url.is_empty() {
if !path.is_empty() {
let name = Path::new(&path)
.file_name()
.and_then(|s| s.to_str())
.unwrap_or("icon");
let lower = path.to_ascii_lowercase();
if rel.to_ascii_lowercase().contains("icon") && lower.ends_with(".ico") {
url = "/favicon.ico".into();
} else {
url = format!("/icons/{name}");
}
} else {
continue;
}
}
if url == "/favicon.ico" {
saw_favicon_url = true;
}
icons.push(json!({
"path": path,
"rel": rel,
"type": type_,
"sizes": sizes,
"url": url,
}));
if !url.is_empty() {
head.push(HeadLink {
rel: rel.clone(),
href: url.clone(),
type_: type_.clone(),
sizes: sizes.clone(),
media: String::new(),
as_: String::new(),
crossorigin: String::new(),
defer: None,
async_: false,
version: String::new(),
});
}
if !path.is_empty() && !url.is_empty() {
routes.push(IconRoute {
url: url.clone(),
path: PathBuf::from(&path),
content_type: if type_.is_empty() {
mime_for_path(&path).to_string()
} else {
type_.clone()
},
});
}
}
// Ensure browser default path if we have an icon file but no /favicon.ico.
if !saw_favicon_url {
if let Some(first) = icons.iter().find(|v| {
v.get("rel")
.and_then(|r| r.as_str())
.map(|r| r.to_ascii_lowercase().contains("icon"))
.unwrap_or(false)
&& v.get("path")
.and_then(|p| p.as_str())
.map(|p| !p.is_empty())
.unwrap_or(false)
}) {
let path = first.get("path").and_then(|v| v.as_str()).unwrap_or("");
let type_ = first
.get("type")
.and_then(|v| v.as_str())
.unwrap_or_else(|| mime_for_path(path));
routes.push(IconRoute {
url: "/favicon.ico".into(),
path: PathBuf::from(path),
content_type: type_.to_string(),
});
head.insert(
0,
HeadLink {
rel: "icon".into(),
href: "/favicon.ico".into(),
type_: type_.to_string(),
sizes: first
.get("sizes")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
media: String::new(),
as_: String::new(),
crossorigin: String::new(),
defer: None,
async_: false,
version: String::new(),
},
);
}
}
(Value::Array(icons), head, routes)
}
/// Convention: if static_dir has favicon.ico|png|svg and icons empty, register defaults.
pub fn convention_favicon(static_dir: &Path) -> Option<(Vec<HeadLink>, Vec<IconRoute>)> {
for name in ["favicon.ico", "favicon.png", "favicon.svg"] {
let p = static_dir.join(name);
if p.is_file() {
let ct = mime_for_path(name).to_string();
let head = vec![HeadLink {
rel: "icon".into(),
href: "/favicon.ico".into(),
type_: ct.clone(),
sizes: String::new(),
media: String::new(),
as_: String::new(),
crossorigin: String::new(),
defer: None,
async_: false,
version: String::new(),
}];
let routes = vec![IconRoute {
url: "/favicon.ico".into(),
path: p,
content_type: ct,
}];
return Some((head, routes));
}
}
None
}
/// Merge site_head into a page value (dedupe by href+rel).
pub fn merge_site_head(page: &mut Value, site_head: &[HeadLink]) {
if site_head.is_empty() {
return;
}
let obj = match page.as_object_mut() {
Some(o) => o,
None => return,
};
let mut existing = obj
.get("head")
.map(head_links_from_json)
.unwrap_or_default();
for link in site_head {
let dup = existing.iter().any(|e| {
e.href == link.href && e.rel.eq_ignore_ascii_case(&link.rel)
});
if !dup {
existing.push(link.clone());
}
}
obj.insert("head".into(), head_links_to_json(&existing));
}
/// GFM image table → HTML fragment.
pub fn make_images_html(table: &Value) -> String {
let mut figures = String::new();
for m in rows_of(table) {
let src = row_str(&m, &["源", "src", "Src", "url", "URL"]);
if src.is_empty() {
continue;
}
let alt = row_str(&m, &["替代", "alt", "Alt"]);
let title = row_str(&m, &["标题", "title", "Title"]);
let class = row_str(&m, &["类", "class", "Class"]);
let href = row_str(&m, &["链接", "href", "link", "Href", "Link"]);
let width = row_str(&m, &["宽度", "width", "Width"]);
let height = row_str(&m, &["高度", "height", "Height"]);
let mut loading = row_str(&m, &["加载", "loading", "Loading"]);
if loading.is_empty() {
loading = "lazy".into();
}
let caption = row_str(&m, &["图注", "caption", "Caption"]);
let mut img = format!("<img src=\"{}\" alt=\"{}\"", esc(&src), esc(&alt));
if !title.is_empty() {
img.push_str(&format!(" title=\"{}\"", esc(&title)));
}
if !width.is_empty() {
img.push_str(&format!(" width=\"{}\"", esc(&width)));
}
if !height.is_empty() {
img.push_str(&format!(" height=\"{}\"", esc(&height)));
}
img.push_str(&format!(" loading=\"{}\"/>", esc(&loading)));
let inner = if href.is_empty() {
img
} else {
format!("<a href=\"{}\">{}</a>", esc(&href), img)
};
let fig_class = if class.is_empty() {
"mq-img".into()
} else {
format!("mq-img {}", class)
};
figures.push_str(&format!("<figure class=\"{}\">{}", fig_class, inner));
if !caption.is_empty() {
figures.push_str(&format!("<figcaption>{}</figcaption>", esc(&caption)));
}
figures.push_str("</figure>");
}
if figures.is_empty() {
return String::new();
}
format!("<div class=\"mq-images\" data-slot=\"main\">{figures}</div>")
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn images_basic() {
let t = json!([
{"源": "/a.png", "替代": "A", "类": "logo", "加载": "eager"}
]);
let h = make_images_html(&t);
assert!(h.contains("mq-images"));
assert!(h.contains("src=\"/a.png\""));
assert!(h.contains("loading=\"eager\""));
assert!(h.contains("class=\"mq-img logo\""));
}
#[test]
fn head_script_and_icon() {
let t = json!([
{"关系": "icon", "地址": "/favicon.ico", "类型": "image/x-icon"},
{"关系": "script", "地址": "/static/a.js"}
]);
let h = make_head_html(&t);
assert!(h.contains("rel=\"icon\""));
assert!(h.contains("<script src=\"/static/a.js\"></script>"));
assert!(!h.contains(" defer"), "{h}");
}
#[test]
fn head_script_defer_async_version() {
let t = json!([
{"关系": "script", "地址": "/static/a.js", "推迟": true, "版本": "3"},
{"关系": "module", "地址": "/static/b.js", "版本": "9"},
{"关系": "stylesheet", "地址": "/static/t.css", "版本": "1"},
]);
let h = make_head_html(&t);
assert!(h.contains("<script defer src=\"/static/a.js?v=3\"></script>"), "{h}");
assert!(h.contains("type=\"module\""), "{h}");
assert!(h.contains("src=\"/static/b.js?v=9\""), "{h}");
assert!(h.contains("href=\"/static/t.css?v=1\""), "{h}");
}
#[test]
fn head_asset_version_fallback() {
let links = as_head_links(&json!([
{"关系": "script", "地址": "/static/a.js", "推迟": "真"},
]));
let h = render_head_links_with_version(&links, "2026-09-04");
assert!(
h.contains("<script defer src=\"/static/a.js?v=2026-09-04\"></script>"),
"{h}"
);
}
}
-242
View File
@@ -1,242 +0,0 @@
//! `# cache` — Redis or in-process memory (design ext-web-drivers).
use serde_json::{json, Value};
use std::collections::HashMap;
use std::sync::Mutex;
use std::time::{SystemTime, UNIX_EPOCH};
struct Entry {
value: String,
/// Absolute expiry unix secs; `None` = no TTL.
exp: Option<u64>,
}
struct MemStore {
data: HashMap<String, Entry>,
}
enum Backend {
Memory(MemStore),
/// Redis URL — connections opened per call (simple; enough for W4).
RedisUrl(String),
}
static HANDLES: Mutex<Option<HashMap<String, Backend>>> = Mutex::new(None);
fn now() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0)
}
fn with_map<T>(f: impl FnOnce(&mut HashMap<String, Backend>) -> T) -> T {
let mut g = HANDLES.lock().unwrap_or_else(|e| e.into_inner());
let m = g.get_or_insert_with(HashMap::new);
f(m)
}
fn is_memory(url: &str) -> bool {
let u = url.trim().to_ascii_lowercase();
u.is_empty() || u == "memory:" || u == "memory://" || u.starts_with("memory:")
}
fn is_redis(url: &str) -> bool {
let u = url.trim().to_ascii_lowercase();
u.starts_with("redis://") || u.starts_with("rediss://")
}
/// Open (or reuse) a cache handle. Returns `{ _type, url }`.
pub fn open(url: &str) -> Result<Value, String> {
let url = if url.trim().is_empty() {
"memory:"
} else {
url.trim()
};
if !is_memory(url) && !is_redis(url) {
return Err(format!(
"cache url must be `memory:` or `redis://…`, got `{url}`"
));
}
with_map(|m| {
if !m.contains_key(url) {
let backend = if is_memory(url) {
Backend::Memory(MemStore {
data: HashMap::new(),
})
} else {
Backend::RedisUrl(url.to_string())
};
m.insert(url.to_string(), backend);
}
});
Ok(json!({ "_type": "cache", "url": url }))
}
fn prune_mem(store: &mut MemStore) {
let t = now();
store.data.retain(|_, e| e.exp.map(|x| x > t).unwrap_or(true));
}
fn with_backend<T>(url: &str, f: impl FnOnce(&mut Backend) -> Result<T, String>) -> Result<T, String> {
with_map(|m| {
let b = m
.get_mut(url)
.ok_or_else(|| format!("cache `{url}` not opened — call cache.new first"))?;
f(b)
})
}
pub fn get(url: &str, key: &str) -> Result<Value, String> {
with_backend(url, |b| match b {
Backend::Memory(store) => {
prune_mem(store);
match store.data.get(key) {
Some(e) => Ok(json!({ "ok": true, "value": e.value })),
None => Ok(json!({ "ok": false })),
}
}
Backend::RedisUrl(u) => redis_get(u, key),
})
}
pub fn set(url: &str, key: &str, value: &str, ttl_sec: Option<u64>) -> Result<Value, String> {
with_backend(url, |b| match b {
Backend::Memory(store) => {
prune_mem(store);
let exp = ttl_sec.filter(|t| *t > 0).map(|t| now() + t);
store.data.insert(
key.to_string(),
Entry {
value: value.to_string(),
exp,
},
);
Ok(json!({ "ok": true }))
}
Backend::RedisUrl(u) => redis_set(u, key, value, ttl_sec),
})
}
pub fn del(url: &str, key: &str) -> Result<Value, String> {
with_backend(url, |b| match b {
Backend::Memory(store) => {
let ok = store.data.remove(key).is_some();
Ok(json!({ "ok": ok }))
}
Backend::RedisUrl(u) => redis_del(u, key),
})
}
pub fn exists(url: &str, key: &str) -> Result<Value, String> {
with_backend(url, |b| match b {
Backend::Memory(store) => {
prune_mem(store);
Ok(json!({ "ok": store.data.contains_key(key) }))
}
Backend::RedisUrl(u) => redis_exists(u, key),
})
}
pub fn ttl(url: &str, key: &str) -> Result<Value, String> {
with_backend(url, |b| match b {
Backend::Memory(store) => {
prune_mem(store);
match store.data.get(key) {
Some(e) => {
let left = match e.exp {
Some(x) => x.saturating_sub(now()) as i64,
None => -1,
};
Ok(json!({ "ok": true, "ttl": left }))
}
None => Ok(json!({ "ok": false, "ttl": -2 })),
}
}
Backend::RedisUrl(u) => redis_ttl(u, key),
})
}
fn redis_get(url: &str, key: &str) -> Result<Value, String> {
let client = redis::Client::open(url).map_err(|e| e.to_string())?;
let mut con = client.get_connection().map_err(|e| e.to_string())?;
let v: Option<String> = redis::cmd("GET")
.arg(key)
.query(&mut con)
.map_err(|e| e.to_string())?;
match v {
Some(s) => Ok(json!({ "ok": true, "value": s })),
None => Ok(json!({ "ok": false })),
}
}
fn redis_set(url: &str, key: &str, value: &str, ttl_sec: Option<u64>) -> Result<Value, String> {
let client = redis::Client::open(url).map_err(|e| e.to_string())?;
let mut con = client.get_connection().map_err(|e| e.to_string())?;
match ttl_sec.filter(|t| *t > 0) {
Some(t) => {
redis::cmd("SETEX")
.arg(key)
.arg(t)
.arg(value)
.query::<()>(&mut con)
.map_err(|e| e.to_string())?;
}
None => {
redis::cmd("SET")
.arg(key)
.arg(value)
.query::<()>(&mut con)
.map_err(|e| e.to_string())?;
}
}
Ok(json!({ "ok": true }))
}
fn redis_del(url: &str, key: &str) -> Result<Value, String> {
let client = redis::Client::open(url).map_err(|e| e.to_string())?;
let mut con = client.get_connection().map_err(|e| e.to_string())?;
let n: i64 = redis::cmd("DEL")
.arg(key)
.query(&mut con)
.map_err(|e| e.to_string())?;
Ok(json!({ "ok": n > 0 }))
}
fn redis_exists(url: &str, key: &str) -> Result<Value, String> {
let client = redis::Client::open(url).map_err(|e| e.to_string())?;
let mut con = client.get_connection().map_err(|e| e.to_string())?;
let n: i64 = redis::cmd("EXISTS")
.arg(key)
.query(&mut con)
.map_err(|e| e.to_string())?;
Ok(json!({ "ok": n > 0 }))
}
fn redis_ttl(url: &str, key: &str) -> Result<Value, String> {
let client = redis::Client::open(url).map_err(|e| e.to_string())?;
let mut con = client.get_connection().map_err(|e| e.to_string())?;
let t: i64 = redis::cmd("TTL")
.arg(key)
.query(&mut con)
.map_err(|e| e.to_string())?;
Ok(json!({ "ok": t >= -1, "ttl": t }))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn memory_roundtrip() {
let url = "memory:test-cache";
open(url).unwrap();
set(url, "a", "1", Some(60)).unwrap();
let g = get(url, "a").unwrap();
assert_eq!(g["ok"], true);
assert_eq!(g["value"], "1");
assert_eq!(exists(url, "a").unwrap()["ok"], true);
del(url, "a").unwrap();
assert_eq!(get(url, "a").unwrap()["ok"], false);
}
}
-248
View File
@@ -1,248 +0,0 @@
//! Page assemble: compose_components / compose_main.
use serde_json::{json, Map, Value};
use crate::table::{
as_bind, as_compose, as_css_named, bind_table_name, normalize_ref, normalize_slot,
parse_site_path, SitePath,
};
pub fn compose_components(
page: &Value,
layout: &Value,
mut call_lib: impl FnMut(&str) -> Result<Value, String>,
) -> Result<Value, String> {
let mut obj = page.as_object().cloned().unwrap_or_default();
let rows = as_compose(layout);
let mut css = obj
.get("styles_css")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string();
let mut slot_class = obj
.get("slot_class")
.and_then(|v| v.as_object())
.cloned()
.unwrap_or_default();
let mut parts = Map::new();
let mut compose_out = Vec::new();
for row in rows.as_array().cloned().unwrap_or_default() {
let src_raw = row.get("src").and_then(|v| v.as_str()).unwrap_or("");
let style_raw = row.get("style").and_then(|v| v.as_str()).unwrap_or("");
if src_raw.is_empty() {
continue;
}
let (name, bind_table) = match parse_site_path(src_raw) {
SitePath::LibMember { lib, member } => {
let t = call_lib(&format!("{lib}.{member}"))?;
(member, t)
}
SitePath::Plain(s) => {
let t = call_lib(&format!("{s}.{s}"))?;
(s, t)
}
SitePath::DbField { .. } => {
return Err(format!("component cell cannot be a db field: {src_raw}"));
}
};
let slot = normalize_slot(&name);
let style_name = if style_raw.is_empty() {
String::new()
} else {
match parse_site_path(style_raw) {
SitePath::LibMember { lib, member } => {
let st = call_lib(&format!("{lib}.{member}"))?;
css.push_str(&as_css_named(&member, &st));
member
}
SitePath::Plain(s) => s,
SitePath::DbField { .. } => normalize_ref(style_raw),
}
};
let binds = as_bind(&bind_table);
match slot.as_str() {
"nav" => obj.insert("nav".into(), binds.clone()),
"sidebar" => obj.insert("sidebar".into(), binds.clone()),
"footer" => obj.insert("footer".into(), binds.clone()),
_ => obj.insert("main".into(), binds.clone()),
};
if !style_name.is_empty() {
slot_class.insert(slot.clone(), json!(style_name));
}
compose_out.push(json!({ "src": name, "style": style_name, "slot": slot }));
let mut part = Map::new();
part.insert("_type".into(), json!("page"));
part.insert("slot".into(), json!(slot));
part.insert("fragment".into(), json!(slot));
match slot.as_str() {
"nav" => {
part.insert("nav".into(), binds);
}
"sidebar" => {
part.insert("sidebar".into(), binds);
}
"footer" => {
part.insert("footer".into(), binds);
}
_ => {
part.insert("main".into(), binds);
}
}
parts.insert(name, Value::Object(part));
}
obj.insert("compose".into(), Value::Array(compose_out));
if !css.is_empty() {
obj.insert("styles_css".into(), json!(css));
}
if !slot_class.is_empty() {
obj.insert("slot_class".into(), Value::Object(slot_class));
}
if !parts.is_empty() {
obj.insert("parts".into(), Value::Object(parts));
}
Ok(Value::Object(obj))
}
pub fn compose_main(
page: &Value,
main_table: &Value,
mut call_lib: impl FnMut(&str) -> Result<Value, String>,
) -> Result<Value, String> {
let mut obj = page.as_object().cloned().unwrap_or_default();
let raw = as_bind(main_table);
let arr = raw.as_array().cloned().unwrap_or_default();
let mut binds = Vec::new();
let mut css = obj
.get("styles_css")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string();
for b in arr {
let front = b
.get("front")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string();
let back_raw = b.get("back").and_then(|v| v.as_str()).unwrap_or("");
let css_raw = b.get("css").and_then(|v| v.as_str()).unwrap_or("");
let back = match parse_site_path(back_raw) {
SitePath::DbField { table, field } => format!("{table}.{field}"),
SitePath::Plain(s) => s,
SitePath::LibMember { lib, member } => {
// Try a module member (e.g. `theme.hero`); if the lib is unknown,
// treat `a.b` as a DB field `a.b` (table `a`, column `b`).
match call_lib(&format!("{lib}.{member}")) {
Ok(v) => match v {
Value::String(s) => s,
other => other.to_string(),
},
Err(_) => format!("{lib}.{member}"),
}
}
};
let css_name = match parse_site_path(css_raw) {
SitePath::LibMember { lib, member } => {
let st = call_lib(&format!("{lib}.{member}"))?;
css.push_str(&as_css_named(&member, &st));
member
}
SitePath::Plain(s) => s,
SitePath::DbField { .. } => normalize_ref(css_raw),
};
binds.push(json!({ "front": front, "back": back, "css": css_name }));
}
if let Some(t) = bind_table_name(&binds) {
obj.insert("table".into(), json!(t));
}
obj.insert("main".into(), Value::Array(binds.clone()));
if !css.is_empty() {
obj.insert("styles_css".into(), json!(css));
}
let mut parts = obj
.get("parts")
.and_then(|v| v.as_object())
.cloned()
.unwrap_or_default();
let mut part = Map::new();
part.insert("_type".into(), json!("page"));
part.insert("slot".into(), json!("main"));
part.insert("fragment".into(), json!("main"));
part.insert("main".into(), Value::Array(binds));
if let Some(intro) = obj.get("intro") {
part.insert("intro".into(), intro.clone());
}
parts.insert("index".into(), Value::Object(part));
obj.insert("parts".into(), Value::Object(parts));
Ok(Value::Object(obj))
}
/// Embed a form into the page main slot (`form_id` → POST `/_form/{id}`).
///
/// Optional `form_target` (CSS id / `#id`) mounts the form **inside** intro HTML
/// at that element instead of appending a sibling `.site-form` (GAP-11).
pub fn compose_form(
page: &Value,
form: &Value,
form_id: &str,
form_target: Option<&str>,
) -> Result<Value, String> {
let id = form_id.trim();
if id.is_empty() {
return Err("compose_form requires non-empty `id`".into());
}
let mut obj = page.as_object().cloned().unwrap_or_default();
let mut frm = form.clone();
if let Some(m) = frm.as_object_mut() {
// Quieter meta when sitting inside a page shell.
m.entry("show_meta".to_string()).or_insert(json!(false));
if !m.contains_key("title") {
m.insert("title".into(), json!(id));
}
}
obj.insert("form_id".into(), json!(id));
obj.insert("form".into(), frm.clone());
let target = form_target
.map(str::trim)
.filter(|s| !s.is_empty())
.map(|s| s.to_string());
if let Some(ref t) = target {
obj.insert("form_target".into(), json!(t));
} else {
obj.remove("form_target");
}
let mut forms = obj
.get("forms")
.and_then(|v| v.as_object())
.cloned()
.unwrap_or_default();
forms.insert(id.to_string(), frm.clone());
obj.insert("forms".into(), Value::Object(forms));
let mut parts = obj
.get("parts")
.and_then(|v| v.as_object())
.cloned()
.unwrap_or_default();
let mut part = Map::new();
part.insert("_type".into(), json!("page"));
part.insert("slot".into(), json!("main"));
part.insert("fragment".into(), json!("main"));
part.insert("form_id".into(), json!(id));
part.insert("form".into(), frm);
if let Some(ref t) = target {
part.insert("form_target".into(), json!(t));
}
if let Some(intro) = obj.get("intro") {
part.insert("intro".into(), intro.clone());
}
parts.insert(format!("form_{id}"), Value::Object(part));
obj.insert("parts".into(), Value::Object(parts));
Ok(Value::Object(obj))
}
File diff suppressed because it is too large. Load diff
-676
View File
@@ -1,676 +0,0 @@
//! Postgres backend for `# db` (same author-facing API as SQLite).
use crate::driver::{db_kind, rewrite_placeholders_pg, DbKind};
use postgres::types::ToSql;
use postgres::{Client, NoTls, Row};
use serde_json::{json, Map, Value};
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
static POOL: Mutex<Option<HashMap<String, Arc<Mutex<Client>>>>> = Mutex::new(None);
static TXNS: Mutex<Option<HashMap<String, (String, Arc<Mutex<Client>>)>>> = Mutex::new(None);
pub fn is_postgres(url: &str) -> bool {
matches!(db_kind(url), DbKind::Postgres)
}
fn pooled(url: &str) -> Result<Arc<Mutex<Client>>, String> {
let mut guard = POOL.lock().unwrap_or_else(|e| e.into_inner());
let pool = guard.get_or_insert_with(HashMap::new);
if let Some(c) = pool.get(url) {
return Ok(c.clone());
}
let client = Client::connect(url, NoTls).map_err(|e| format!("postgres connect: {e}"))?;
let arc = Arc::new(Mutex::new(client));
pool.insert(url.to_string(), arc.clone());
Ok(arc)
}
fn conn_for(url: &str, txn_id: Option<&str>) -> Result<Arc<Mutex<Client>>, String> {
if let Some(tid) = txn_id {
let guard = TXNS.lock().unwrap_or_else(|e| e.into_inner());
match guard.as_ref().and_then(|m| m.get(tid)) {
Some((_, c)) => Ok(c.clone()),
None => Err(format!("unknown transaction `{tid}`")),
}
} else {
pooled(url)
}
}
fn next_txn_id() -> String {
use std::time::{SystemTime, UNIX_EPOCH};
let nanos = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.subsec_nanos())
.unwrap_or(0);
format!("pg-txn-{}-{nanos}", std::process::id())
}
fn ident(s: &str) -> Result<&str, String> {
let s = s.trim();
if s.is_empty()
|| !s
.chars()
.all(|c| c.is_ascii_alphanumeric() || c == '_')
{
return Err(format!("bad identifier `{s}`"));
}
Ok(s)
}
fn cell_str(v: &Value) -> String {
match v {
Value::Null => String::new(),
Value::String(s) => s.clone(),
Value::Bool(b) => b.to_string(),
Value::Number(n) => n.to_string(),
other => other.to_string(),
}
}
fn pg_type(t: &str) -> &'static str {
match t.trim().to_ascii_lowercase().as_str() {
"int" | "integer" => "INTEGER",
"real" | "float" | "double" => "DOUBLE PRECISION",
"blob" | "bytea" => "BYTEA",
"bool" | "boolean" => "BOOLEAN",
"timestamp" | "datetime" | "timestamptz" | "审计" => "TIMESTAMPTZ",
_ => "TEXT",
}
}
fn to_pg_box(v: &Value) -> Box<dyn ToSql + Sync + Send> {
match v {
Value::Null => Box::new(None::<String>),
Value::Bool(b) => Box::new(*b),
Value::Number(n) => {
if let Some(i) = n.as_i64() {
Box::new(i)
} else if let Some(f) = n.as_f64() {
Box::new(f)
} else {
Box::new(n.to_string())
}
}
Value::String(s) => Box::new(s.clone()),
other => Box::new(other.to_string()),
}
}
fn row_to_json(row: &Row) -> Value {
let mut m = Map::new();
for (i, col) in row.columns().iter().enumerate() {
let name = col.name().to_string();
let v: Value = if let Ok(x) = row.try_get::<_, Option<i64>>(i) {
x.map(|n| json!(n)).unwrap_or(Value::Null)
} else if let Ok(x) = row.try_get::<_, Option<f64>>(i) {
x.map(|n| json!(n)).unwrap_or(Value::Null)
} else if let Ok(x) = row.try_get::<_, Option<bool>>(i) {
x.map(|b| json!(b)).unwrap_or(Value::Null)
} else if let Ok(x) = row.try_get::<_, Option<String>>(i) {
x.map(Value::String).unwrap_or(Value::Null)
} else {
Value::Null
};
m.insert(name, v);
}
Value::Object(m)
}
fn exec_sql(client: &mut Client, sql: &str, params: &[Value]) -> Result<u64, String> {
let sql = rewrite_placeholders_pg(sql);
let boxes: Vec<Box<dyn ToSql + Sync + Send>> = params.iter().map(to_pg_box).collect();
let refs: Vec<&(dyn ToSql + Sync)> = boxes
.iter()
.map(|b| b.as_ref() as &(dyn ToSql + Sync))
.collect();
client
.execute(sql.as_str(), refs.as_slice())
.map_err(|e| e.to_string())
}
fn query_sql(client: &mut Client, sql: &str, params: &[Value]) -> Result<Vec<Value>, String> {
let sql = rewrite_placeholders_pg(sql);
let boxes: Vec<Box<dyn ToSql + Sync + Send>> = params.iter().map(to_pg_box).collect();
let refs: Vec<&(dyn ToSql + Sync)> = boxes
.iter()
.map(|b| b.as_ref() as &(dyn ToSql + Sync))
.collect();
let rows = client
.query(sql.as_str(), refs.as_slice())
.map_err(|e| e.to_string())?;
Ok(rows.iter().map(row_to_json).collect())
}
/// Equality-map / filter-table → `(AND exprs with ?, vals)`.
fn parse_where_simple(where_v: Option<&Value>) -> Result<(Vec<String>, Vec<Value>), String> {
let Some(w) = where_v else {
return Ok((vec![], vec![]));
};
let w = crate::table::as_rows(w);
let mut exprs = Vec::new();
let mut vals = Vec::new();
match w {
Value::Array(arr) => {
for item in arr {
let Value::Object(m) = item else {
continue;
};
let field = m
.get("field")
.or_else(|| m.get("字段"))
.or_else(|| m.get("列"))
.map(cell_str)
.unwrap_or_default();
if !field.is_empty() {
let col = ident(&field)?;
let op = m
.get("op")
.or_else(|| m.get("操作"))
.map(cell_str)
.unwrap_or_else(|| "=".into());
let op = match op.trim().to_ascii_lowercase().as_str() {
"" | "=" | "eq" | "等于" => "=",
"!=" | "<>" | "ne" | "不等于" => "!=",
">" | "gt" | "大于" => ">",
">=" | "gte" => ">=",
"<" | "lt" | "小于" => "<",
"<=" | "lte" => "<=",
"like" | "包含" => "LIKE",
other => return Err(format!("unsupported where op `{other}`")),
};
let val = m
.get("value")
.or_else(|| m.get("值"))
.cloned()
.unwrap_or(Value::Null);
exprs.push(format!("\"{col}\" {op} ?"));
vals.push(val);
} else {
for (k, v) in &m {
if k == "@" || k == "行" || k == "row" || k.starts_with('_') {
continue;
}
let col = ident(k)?;
exprs.push(format!("\"{col}\" = ?"));
vals.push(v.clone());
}
}
}
}
_ => {}
}
Ok((exprs, vals))
}
fn parse_fk(raw: &str) -> Result<(String, String), String> {
let s = raw.trim();
if s.is_empty() {
return Err("empty foreign key".into());
}
if let Some((t, rest)) = s.split_once('(') {
let col = rest.trim().trim_end_matches(')').trim();
let t = ident(t)?.to_string();
let col = if col.is_empty() {
"id".into()
} else {
ident(col)?.to_string()
};
return Ok((t, col));
}
if let Some((t, c)) = s.split_once('.') {
return Ok((ident(t)?.to_string(), ident(c)?.to_string()));
}
Ok((ident(s)?.to_string(), "id".into()))
}
fn utc_now_iso() -> String {
use std::time::{SystemTime, UNIX_EPOCH};
let secs = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
let days = secs / 86400;
let tod = secs % 86400;
let h = tod / 3600;
let m = (tod % 3600) / 60;
let s = tod % 60;
let z = days as i64 + 719468;
let era = if z >= 0 { z } else { z - 146096 } / 146097;
let doe = (z - era * 146097) as u64;
let yoe = (doe - doe / 1460 + doe / 36524 - doe / 146096) / 365;
let y = yoe as i64 + era * 400;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let d = doy - (153 * mp + 2) / 5 + 1;
let mo = if mp < 10 { mp + 3 } else { mp - 9 };
let y = if mo <= 2 { y + 1 } else { y };
format!("{y:04}-{mo:02}-{d:02}T{h:02}:{m:02}:{s:02}Z")
}
fn column_names(url: &str, table: &str, txn_id: Option<&str>) -> Result<Vec<String>, String> {
let table = ident(table)?;
let conn = conn_for(url, txn_id)?;
let mut c = conn.lock().unwrap_or_else(|e| e.into_inner());
let rows = query_sql(
&mut c,
"SELECT column_name FROM information_schema.columns WHERE table_name = ? ORDER BY ordinal_position",
&[json!(table)],
)?;
Ok(rows
.into_iter()
.filter_map(|r| {
r.get("column_name")
.or_else(|| r.get("COLUMN_NAME"))
.and_then(|v| v.as_str())
.map(|s| s.to_string())
})
.collect())
}
pub fn init(url: &str, table: &str, fields: &Value) -> Result<Value, String> {
let table = ident(table)?;
let cols = crate::table::as_fields(fields);
let arr = cols.as_array().ok_or("fields must be a list")?;
if arr.is_empty() {
return Err("empty fields".into());
}
let mut parts = Vec::new();
let mut has_pk = false;
let mut index_sql = Vec::new();
for c in arr {
let name = c
.get("name")
.or_else(|| c.get("字段"))
.map(cell_str)
.unwrap_or_default();
let name = ident(&name)?;
let ty = c
.get("type")
.or_else(|| c.get("类型"))
.map(cell_str)
.unwrap_or_else(|| "text".into());
let nullable = c
.get("nullable")
.or_else(|| c.get("可空"))
.and_then(|v| match v {
Value::Bool(b) => Some(*b),
Value::String(s) => {
Some(matches!(s.as_str(), "true" | "True" | "1" | "是" | "可"))
}
_ => None,
})
.unwrap_or(true);
let unique = c
.get("unique")
.or_else(|| c.get("唯一"))
.and_then(|v| match v {
Value::Bool(b) => Some(*b),
Value::String(s) => {
Some(matches!(s.as_str(), "true" | "True" | "1" | "yes" | "是" | "唯一"))
}
_ => None,
})
.unwrap_or(false);
let index = c
.get("index")
.or_else(|| c.get("索引"))
.and_then(|v| match v {
Value::Bool(b) => Some(*b),
Value::String(s) => {
Some(matches!(s.as_str(), "true" | "True" | "1" | "yes" | "是" | "索引"))
}
_ => None,
})
.unwrap_or(false);
let part = if name == "id" && !has_pk && pg_type(&ty) == "INTEGER" {
has_pk = true;
format!("\"{name}\" SERIAL PRIMARY KEY")
} else if name == "id" && !has_pk {
has_pk = true;
format!("\"{name}\" {} PRIMARY KEY", pg_type(&ty))
} else {
let mut col = format!("\"{name}\" {}", pg_type(&ty));
if !nullable {
col.push_str(" NOT NULL");
}
if unique {
col.push_str(" UNIQUE");
}
let fk_raw = c
.get("fk")
.or_else(|| c.get("外键"))
.map(cell_str)
.unwrap_or_default();
if !fk_raw.is_empty() {
let (ref_table, ref_col) = parse_fk(&fk_raw)?;
col.push_str(&format!(" REFERENCES \"{ref_table}\"(\"{ref_col}\")"));
}
col
};
parts.push(part);
if (index || unique) && name != "id" {
let kind = if unique { "UNIQUE " } else { "" };
index_sql.push(format!(
"CREATE {kind}INDEX IF NOT EXISTS \"idx_{table}_{name}\" ON \"{table}\" (\"{name}\")"
));
}
}
let sql = format!(
"CREATE TABLE IF NOT EXISTS \"{table}\" ({})",
parts.join(", ")
);
let conn = pooled(url)?;
let mut c = conn.lock().unwrap_or_else(|e| e.into_inner());
c.batch_execute(&sql).map_err(|e| e.to_string())?;
for stmt in index_sql {
let _ = c.batch_execute(&stmt);
}
Ok(json!({ "_type": "db_table", "name": table, "url": url }))
}
pub fn insert(url: &str, table: &str, rows: &Value, txn_id: Option<&str>) -> Result<Value, String> {
let table = ident(table)?;
let cols_present = column_names(url, table, txn_id).unwrap_or_default();
let has_created = cols_present.iter().any(|c| c == "created_at");
let has_updated = cols_present.iter().any(|c| c == "updated_at");
let rows = crate::table::as_rows(rows);
let arr = rows.as_array().ok_or("rows must be a list")?;
let conn = conn_for(url, txn_id)?;
let mut c = conn.lock().unwrap_or_else(|e| e.into_inner());
let mut n = 0i64;
let now = utc_now_iso();
for row in arr {
let obj = row.as_object().ok_or("row must be a map")?;
let mut cols = Vec::new();
let mut ph = Vec::new();
let mut vals = Vec::new();
let mut seen_created = false;
let mut seen_updated = false;
for (k, v) in obj {
if k == "@" || k == "行" || k == "row" {
continue;
}
let _ = ident(k)?;
if k == "created_at" {
seen_created = true;
}
if k == "updated_at" {
seen_updated = true;
}
cols.push(format!("\"{k}\""));
ph.push("?");
vals.push(v.clone());
}
if has_created && !seen_created {
cols.push("\"created_at\"".into());
ph.push("?");
vals.push(json!(now.clone()));
}
if has_updated && !seen_updated {
cols.push("\"updated_at\"".into());
ph.push("?");
vals.push(json!(now.clone()));
}
if cols.is_empty() {
continue;
}
let sql = format!(
"INSERT INTO \"{table}\" ({}) VALUES ({})",
cols.join(", "),
ph.join(", ")
);
exec_sql(&mut c, &sql, &vals)?;
n += 1;
}
Ok(json!({ "ok": true, "inserted": n }))
}
pub fn select_order(
url: &str,
table: &str,
limit: i64,
where_v: Option<&Value>,
order: Option<&str>,
offset: Option<i64>,
txn_id: Option<&str>,
) -> Result<Value, String> {
let table = ident(table)?;
let (exprs, mut vals) = parse_where_simple(where_v)?;
let mut sql = format!("SELECT * FROM \"{table}\"");
let where_sql = if exprs.is_empty() {
String::new()
} else {
format!(" WHERE {}", exprs.join(" AND "))
};
sql.push_str(&where_sql);
if let Some(order) = order.map(str::trim).filter(|s| !s.is_empty()) {
let mut order_exprs = Vec::new();
for part in order.split(',').map(str::trim).filter(|p| !p.is_empty()) {
let (col, dir) = match part.strip_prefix('-') {
Some(c) => (c, " DESC"),
None => (part, ""),
};
order_exprs.push(format!("\"{}\"{}", ident(col)?, dir));
}
if !order_exprs.is_empty() {
sql.push_str(" ORDER BY ");
sql.push_str(&order_exprs.join(", "));
}
}
sql.push_str(" LIMIT ?");
vals.push(json!(limit));
if let Some(off) = offset {
if off > 0 {
sql.push_str(" OFFSET ?");
vals.push(json!(off));
}
}
let conn = conn_for(url, txn_id)?;
let mut c = conn.lock().unwrap_or_else(|e| e.into_inner());
let rows = query_sql(&mut c, &sql, &vals)?;
if offset.is_some() {
let mut count_sql = format!("SELECT COUNT(*) AS count FROM \"{table}\"");
count_sql.push_str(&where_sql);
let count_vals: Vec<Value> = vals
.iter()
.take(exprs.len())
.cloned()
.collect();
let total = query_sql(&mut c, &count_sql, &count_vals)?
.first()
.and_then(|r| r.get("count"))
.and_then(|v| v.as_i64())
.unwrap_or(rows.len() as i64);
Ok(json!({ "rows": rows, "total": total }))
} else {
Ok(json!({ "rows": rows }))
}
}
pub fn get(url: &str, table: &str, id: &str, txn_id: Option<&str>) -> Result<Value, String> {
let table = ident(table)?;
let sql = format!("SELECT * FROM \"{table}\" WHERE \"id\" = ? LIMIT 1");
let conn = conn_for(url, txn_id)?;
let mut c = conn.lock().unwrap_or_else(|e| e.into_inner());
let rows = query_sql(&mut c, &sql, &[json!(id)])?;
Ok(rows.into_iter().next().unwrap_or(Value::Null))
}
pub fn update(
url: &str,
table: &str,
id: &str,
row: &Value,
txn_id: Option<&str>,
) -> Result<Value, String> {
let table = ident(table)?;
let cols_present = column_names(url, table, txn_id).unwrap_or_default();
let has_updated = cols_present.iter().any(|c| c == "updated_at");
let obj = row.as_object().ok_or("row must be a map")?;
let mut sets = Vec::new();
let mut vals = Vec::new();
let mut seen_updated = false;
for (k, v) in obj {
if k == "id" || k == "@" || k == "行" || k == "row" {
continue;
}
let _ = ident(k)?;
if k == "updated_at" {
seen_updated = true;
}
sets.push(format!("\"{k}\" = ?"));
vals.push(v.clone());
}
if has_updated && !seen_updated {
sets.push("\"updated_at\" = ?".into());
vals.push(json!(utc_now_iso()));
}
if sets.is_empty() {
return Ok(json!({ "ok": true, "updated": 0 }));
}
vals.push(json!(id));
let sql = format!(
"UPDATE \"{table}\" SET {} WHERE \"id\" = ?",
sets.join(", ")
);
let conn = conn_for(url, txn_id)?;
let mut c = conn.lock().unwrap_or_else(|e| e.into_inner());
let n = exec_sql(&mut c, &sql, &vals)?;
Ok(json!({ "ok": true, "updated": n }))
}
pub fn delete(url: &str, table: &str, id: &str, txn_id: Option<&str>) -> Result<Value, String> {
let table = ident(table)?;
let sql = format!("DELETE FROM \"{table}\" WHERE \"id\" = ?");
let conn = conn_for(url, txn_id)?;
let mut c = conn.lock().unwrap_or_else(|e| e.into_inner());
let n = exec_sql(&mut c, &sql, &[json!(id)])?;
Ok(json!({ "ok": true, "deleted": n }))
}
pub fn exec(url: &str, sql: &str, args: Option<&Value>, txn_id: Option<&str>) -> Result<Value, String> {
let vals: Vec<Value> = match args {
Some(Value::Array(a)) => a.clone(),
_ => Vec::new(),
};
let conn = conn_for(url, txn_id)?;
let mut c = conn.lock().unwrap_or_else(|e| e.into_inner());
let n = exec_sql(&mut c, sql, &vals)?;
Ok(json!({ "ok": true, "changes": n }))
}
pub fn query(url: &str, sql: &str, args: Option<&Value>, txn_id: Option<&str>) -> Result<Value, String> {
let vals: Vec<Value> = match args {
Some(Value::Array(a)) => a.clone(),
_ => Vec::new(),
};
let conn = conn_for(url, txn_id)?;
let mut c = conn.lock().unwrap_or_else(|e| e.into_inner());
let rows = query_sql(&mut c, sql, &vals)?;
Ok(json!({ "rows": rows, "count": rows.len() }))
}
pub fn count(
url: &str,
table: &str,
where_v: Option<&Value>,
txn_id: Option<&str>,
) -> Result<Value, String> {
let table = ident(table)?;
let (exprs, vals) = parse_where_simple(where_v)?;
let mut sql = format!("SELECT COUNT(*) AS count FROM \"{table}\"");
if !exprs.is_empty() {
sql.push_str(&format!(" WHERE {}", exprs.join(" AND ")));
}
let conn = conn_for(url, txn_id)?;
let mut c = conn.lock().unwrap_or_else(|e| e.into_inner());
let rows = query_sql(&mut c, &sql, &vals)?;
let n = rows
.first()
.and_then(|r| r.get("count"))
.and_then(|v| v.as_i64())
.unwrap_or(0);
Ok(json!({ "count": n }))
}
pub fn begin(url: &str) -> Result<Value, String> {
let conn = pooled(url)?;
{
let mut c = conn.lock().unwrap_or_else(|e| e.into_inner());
c.batch_execute("BEGIN").map_err(|e| e.to_string())?;
}
let id = next_txn_id();
{
let mut guard = TXNS.lock().unwrap_or_else(|e| e.into_inner());
guard
.get_or_insert_with(HashMap::new)
.insert(id.clone(), (url.to_string(), conn));
}
Ok(json!({ "_type": "txn", "txn": id, "url": url, "事务": id, "地址": url }))
}
pub fn commit(txn_id: &str) -> Result<Value, String> {
let (_, conn) = take_txn(txn_id)?;
let mut c = conn.lock().unwrap_or_else(|e| e.into_inner());
c.batch_execute("COMMIT").map_err(|e| e.to_string())?;
Ok(json!({ "ok": true }))
}
pub fn rollback(txn_id: &str) -> Result<Value, String> {
let (_, conn) = take_txn(txn_id)?;
let mut c = conn.lock().unwrap_or_else(|e| e.into_inner());
c.batch_execute("ROLLBACK").map_err(|e| e.to_string())?;
Ok(json!({ "ok": true }))
}
fn take_txn(txn_id: &str) -> Result<(String, Arc<Mutex<Client>>), String> {
let mut guard = TXNS.lock().unwrap_or_else(|e| e.into_inner());
guard
.as_mut()
.and_then(|m| m.remove(txn_id))
.ok_or_else(|| format!("unknown transaction `{txn_id}`"))
}
pub fn list_tables(url: &str) -> Result<Vec<String>, String> {
let conn = pooled(url)?;
let mut c = conn.lock().unwrap_or_else(|e| e.into_inner());
let rows = query_sql(
&mut c,
"SELECT tablename AS name FROM pg_catalog.pg_tables WHERE schemaname = 'public' ORDER BY tablename",
&[],
)?;
Ok(rows
.iter()
.filter_map(|r| r.get("name").and_then(|v| v.as_str()).map(|s| s.to_string()))
.collect())
}
pub fn table_info(url: &str, table: &str) -> Result<Vec<crate::db::ColumnInfo>, String> {
let table = ident(table)?;
let conn = pooled(url)?;
let mut c = conn.lock().unwrap_or_else(|e| e.into_inner());
let rows = query_sql(
&mut c,
"SELECT column_name AS name, data_type AS sql_type, \
CASE WHEN is_nullable = 'NO' THEN true ELSE false END AS notnull \
FROM information_schema.columns \
WHERE table_schema = 'public' AND table_name = ? \
ORDER BY ordinal_position",
&[json!(table)],
)?;
Ok(rows
.iter()
.map(|r| crate::db::ColumnInfo {
name: r.get("name").and_then(|v| v.as_str()).unwrap_or("").into(),
sql_type: r
.get("sql_type")
.and_then(|v| v.as_str())
.unwrap_or("text")
.into(),
notnull: r.get("notnull").and_then(|v| v.as_bool()).unwrap_or(false),
pk: r.get("name").and_then(|v| v.as_str()) == Some("id"),
})
.collect())
}
-54
View File
@@ -1,54 +0,0 @@
//! Driver URL schemes for `# db` / `# cache` / `# storage` (design ext-web-drivers).
/// Which SQL backend a `# db` URL selects.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum DbKind {
Sqlite,
Postgres,
}
pub fn db_kind(url: &str) -> DbKind {
let u = url.trim().to_ascii_lowercase();
if u.starts_with("postgres://") || u.starts_with("postgresql://") {
DbKind::Postgres
} else {
DbKind::Sqlite
}
}
/// Rewrite SQL written with `?` placeholders into Postgres `$1,$2,…` form.
pub fn rewrite_placeholders_pg(sql: &str) -> String {
let mut out = String::with_capacity(sql.len() + 8);
let mut n = 0usize;
let mut chars = sql.chars().peekable();
while let Some(c) = chars.next() {
if c == '?' {
n += 1;
out.push('$');
out.push_str(&n.to_string());
} else {
out.push(c);
}
}
out
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn kind_from_url() {
assert_eq!(db_kind("sqlite:a.db"), DbKind::Sqlite);
assert_eq!(db_kind("postgres://u:p@h/db"), DbKind::Postgres);
assert_eq!(db_kind("postgresql://localhost/x"), DbKind::Postgres);
}
#[test]
fn rewrite_qmark() {
assert_eq!(
rewrite_placeholders_pg("SELECT * FROM t WHERE a = ? AND b IN (?, ?)"),
"SELECT * FROM t WHERE a = $1 AND b IN ($2, $3)"
);
}
}
-701
View File
@@ -1,701 +0,0 @@
//! Form fields / validate / render / submit (design §5.5).
use serde_json::{json, Map, Value};
use crate::db;
use crate::table::normalize_ref;
const FIELD_KEYS: &[&str] = &["字段", "name", "field", "列"];
const LABEL_KEYS: &[&str] = &["标签", "label"];
const TYPE_KEYS: &[&str] = &["类型", "type"];
const REQ_KEYS: &[&str] = &["必填", "required"];
const DEF_KEYS: &[&str] = &["默认", "default"];
const RULE_FIELD_KEYS: &[&str] = &["字段", "field", "name"];
const RULE_KEYS: &[&str] = &["规则", "rule"];
const MSG_KEYS: &[&str] = &["消息", "message", "msg"];
fn cell_str(v: &Value) -> String {
match v {
Value::Null => String::new(),
Value::String(s) => s.clone(),
Value::Bool(b) => b.to_string(),
Value::Number(n) => n.to_string(),
other => other.to_string(),
}
}
fn as_str_list(v: &Value) -> Vec<String> {
match v {
Value::Array(a) => a.iter().map(cell_str).collect(),
Value::Null => Vec::new(),
other => vec![cell_str(other)],
}
}
fn pick<'a>(m: &'a Map<String, Value>, keys: &[&str]) -> Option<&'a Value> {
keys.iter().find_map(|k| m.get(*k))
}
fn boolish(v: &Value) -> bool {
match v {
Value::Bool(b) => *b,
Value::String(s) => matches!(
s.trim(),
"1" | "true" | "True" | "yes" | "是" | "必填" | "required"
),
Value::Number(n) => n.as_i64().unwrap_or(0) != 0,
_ => false,
}
}
fn esc(s: &str) -> String {
s.replace('&', "&amp;")
.replace('<', "&lt;")
.replace('>', "&gt;")
.replace('"', "&quot;")
}
/// Normalize author field table → `[{name,label,type,required,default}, …]`
pub fn as_form_fields(table: &Value) -> Value {
match table {
Value::Array(rows) => {
let mut out = Vec::new();
for row in rows {
if let Some(m) = row.as_object() {
let name = normalize_ref(&pick(m, FIELD_KEYS).map(cell_str).unwrap_or_default());
if name.is_empty() {
continue;
}
let label = pick(m, LABEL_KEYS).map(cell_str).unwrap_or_else(|| name.clone());
let ty = pick(m, TYPE_KEYS)
.map(cell_str)
.unwrap_or_else(|| "text".into());
let required = pick(m, REQ_KEYS).map(boolish).unwrap_or(false);
let default = pick(m, DEF_KEYS).map(cell_str).unwrap_or_default();
out.push(json!({
"name": name,
"label": label,
"type": ty,
"required": required,
"default": default,
}));
}
}
Value::Array(out)
}
Value::Object(m) => {
let names = pick(m, FIELD_KEYS).map(as_str_list).unwrap_or_default();
let labels = pick(m, LABEL_KEYS).map(as_str_list).unwrap_or_default();
let types = pick(m, TYPE_KEYS).map(as_str_list).unwrap_or_default();
let reqs = pick(m, REQ_KEYS).map(as_str_list).unwrap_or_default();
let defs = pick(m, DEF_KEYS).map(as_str_list).unwrap_or_default();
let mut out = Vec::new();
for (i, name_raw) in names.iter().enumerate() {
let name = normalize_ref(name_raw);
if name.is_empty() {
continue;
}
let label = labels.get(i).cloned().unwrap_or_else(|| name.clone());
let ty = types.get(i).cloned().unwrap_or_else(|| "text".into());
let required = reqs
.get(i)
.map(|s| boolish(&Value::String(s.clone())))
.unwrap_or(false);
let default = defs.get(i).cloned().unwrap_or_default();
out.push(json!({
"name": name,
"label": label,
"type": ty,
"required": required,
"default": default,
}));
}
Value::Array(out)
}
_ => json!([]),
}
}
/// Normalize rules table → `[{field,rule,message}, …]`
pub fn as_form_rules(table: &Value) -> Value {
match table {
Value::Array(rows) => {
let mut out = Vec::new();
for row in rows {
if let Some(m) = row.as_object() {
let field =
normalize_ref(&pick(m, RULE_FIELD_KEYS).map(cell_str).unwrap_or_default());
let rule = pick(m, RULE_KEYS).map(cell_str).unwrap_or_default();
if field.is_empty() || rule.is_empty() {
continue;
}
let message = pick(m, MSG_KEYS).map(cell_str).unwrap_or_default();
out.push(json!({ "field": field, "rule": rule, "message": message }));
}
}
Value::Array(out)
}
Value::Object(m) => {
let fields = pick(m, RULE_FIELD_KEYS).map(as_str_list).unwrap_or_default();
let rules = pick(m, RULE_KEYS).map(as_str_list).unwrap_or_default();
let msgs = pick(m, MSG_KEYS).map(as_str_list).unwrap_or_default();
let n = fields.len().max(rules.len());
let mut out = Vec::new();
for i in 0..n {
let field = normalize_ref(&fields.get(i).cloned().unwrap_or_default());
let rule = rules.get(i).cloned().unwrap_or_default();
if field.is_empty() || rule.is_empty() {
continue;
}
let message = msgs.get(i).cloned().unwrap_or_default();
out.push(json!({ "field": field, "rule": rule, "message": message }));
}
Value::Array(out)
}
_ => json!([]),
}
}
pub fn form_new(table: Option<&str>, action: &str, id: Option<&str>) -> Value {
let action = match action {
"update" | "更新" => "update",
_ => "insert",
};
let mut m = Map::new();
if let Some(t) = table.filter(|s| !s.is_empty()) {
m.insert("table".into(), json!(t));
}
m.insert("action".into(), json!(action));
if let Some(i) = id.filter(|s| !s.is_empty()) {
m.insert("id".into(), json!(i));
}
m.insert("fields".into(), json!([]));
m.insert("rules".into(), json!([]));
m.insert("redirect".into(), json!("/"));
Value::Object(m)
}
fn sql_type_to_input(sql_type: &str, name: &str) -> &'static str {
let t = sql_type.to_ascii_uppercase();
if t.contains("INT") || t.contains("REAL") || t.contains("FLOAT") || t.contains("DOUBLE") {
return "number";
}
if name == "body" || name.ends_with("_body") || name.ends_with("_text") {
return "textarea";
}
if name.contains("email") {
return "email";
}
if name.contains("url") || name == "href" {
return "url";
}
"text"
}
/// Build a form handle from live SQLite schema (admin new/edit).
/// `admin_prefix` is the mount root (default `/admin`), without trailing slash.
pub fn from_schema(
db_url: &str,
table: &str,
action: &str,
id: Option<&str>,
admin_prefix: &str,
) -> Result<Value, String> {
let action = match action {
"update" | "更新" => "update",
_ => "insert",
};
let cols = db::table_info(db_url, table)?;
if cols.is_empty() {
return Err(format!("unknown table `{table}`"));
}
let mut fields = Vec::new();
for c in &cols {
if action == "insert" && c.pk && c.name == "id" {
continue; // SERIAL / AUTOINCREMENT
}
let input = sql_type_to_input(&c.sql_type, &c.name);
let required = c.notnull || (action == "update" && c.pk);
fields.push(json!({
"name": c.name,
"label": c.name,
"type": input,
"required": required,
"default": "",
"pk": c.pk,
}));
}
let base = admin_prefix.trim_end_matches('/');
let table_href = format!("{base}/{table}");
let mut form = form_new(Some(table), action, id);
if let Some(obj) = form.as_object_mut() {
obj.insert("fields".into(), Value::Array(fields));
obj.insert("redirect".into(), json!(&table_href));
if action == "insert" {
obj.insert("action_url".into(), json!(format!("{table_href}/new")));
obj.insert("title".into(), json!(format!("New {table}")));
} else if let Some(i) = id {
obj.insert(
"action_url".into(),
json!(format!("{table_href}/{i}/edit")),
);
obj.insert("title".into(), json!(format!("Edit {table} #{i}")));
obj.insert("id".into(), json!(i));
}
obj.insert("cancel_href".into(), json!(&table_href));
}
Ok(form)
}
pub fn set_fields(form: &Value, fields: &Value) -> Value {
let mut obj = form.as_object().cloned().unwrap_or_default();
obj.insert("fields".into(), as_form_fields(fields));
Value::Object(obj)
}
pub fn set_rules(form: &Value, rules: &Value) -> Value {
let mut obj = form.as_object().cloned().unwrap_or_default();
obj.insert("rules".into(), as_form_rules(rules));
Value::Object(obj)
}
fn data_map(data: &Value) -> Map<String, Value> {
let rows = crate::table::as_rows(data);
if let Some(Value::Object(m)) = rows.as_array().and_then(|a| a.first()) {
return m.clone();
}
match data {
Value::Object(m) => m.clone(),
_ => Map::new(),
}
}
fn field_text(data: &Map<String, Value>, name: &str) -> String {
data.get(name).map(cell_str).unwrap_or_default()
}
fn default_message(rule: &str, field: &str) -> String {
if let Some(rest) = rule.strip_prefix("min:") {
return format!("{field} must be at least {rest}");
}
if let Some(rest) = rule.strip_prefix("max:") {
return format!("{field} must be at most {rest}");
}
if let Some(other) = rule.strip_prefix("match:") {
return format!("{field} must match {other}");
}
if rule.starts_with("in:") {
return format!("{field} is not an allowed value");
}
match rule {
"required" => format!("{field} is required"),
"email" => format!("{field} must be an email"),
"url" => format!("{field} must be a URL"),
_ => format!("{field} failed `{rule}`"),
}
}
fn check_rule(rule: &str, field: &str, data: &Map<String, Value>) -> Option<String> {
let val = field_text(data, field);
let trimmed = val.trim();
if rule == "required" {
return if trimmed.is_empty() {
Some(default_message(rule, field))
} else {
None
};
}
if trimmed.is_empty() {
return None; // other rules skip empty (required covers emptiness)
}
if let Some(n) = rule.strip_prefix("min:") {
if let Ok(min) = n.parse::<i64>() {
if let Ok(num) = trimmed.parse::<f64>() {
if num < min as f64 {
return Some(default_message(rule, field));
}
} else if (trimmed.chars().count() as i64) < min {
return Some(default_message(rule, field));
}
}
return None;
}
if let Some(n) = rule.strip_prefix("max:") {
if let Ok(max) = n.parse::<i64>() {
if let Ok(num) = trimmed.parse::<f64>() {
if num > max as f64 {
return Some(default_message(rule, field));
}
} else if (trimmed.chars().count() as i64) > max {
return Some(default_message(rule, field));
}
}
return None;
}
if let Some(other) = rule.strip_prefix("match:") {
let other_val = field_text(data, other);
if val != other_val {
return Some(default_message(rule, field));
}
return None;
}
if let Some(list) = rule.strip_prefix("in:") {
let allowed: Vec<&str> = list.split(',').map(|s| s.trim()).collect();
if !allowed.contains(&trimmed) {
return Some(default_message(rule, field));
}
return None;
}
match rule {
"email" => {
if !(trimmed.contains('@') && trimmed.contains('.')) {
Some(default_message(rule, field))
} else {
None
}
}
"url" => {
if !(trimmed.starts_with("http://") || trimmed.starts_with("https://")) {
Some(default_message(rule, field))
} else {
None
}
}
_ => None,
}
}
/// Merge field-required + author rules; return `{ok, errors:[{field,message}]}`.
pub fn validate(form: &Value, extra_rules: Option<&Value>, data: &Value) -> Value {
let data = data_map(data);
let fields = form
.get("fields")
.cloned()
.unwrap_or(json!([]));
let mut rules = form
.get("rules")
.and_then(|v| v.as_array())
.cloned()
.unwrap_or_default();
if let Some(extra) = extra_rules {
if let Some(arr) = as_form_rules(extra).as_array() {
rules.extend(arr.clone());
}
}
// Auto required from field table (message overridable by later explicit rules)
if let Some(arr) = fields.as_array() {
for f in arr {
let name = f.get("name").and_then(|v| v.as_str()).unwrap_or("");
let required = f.get("required").and_then(|v| v.as_bool()).unwrap_or(false);
if required && !name.is_empty() {
let already = rules.iter().any(|r| {
r.get("field").and_then(|v| v.as_str()) == Some(name)
&& r.get("rule").and_then(|v| v.as_str()) == Some("required")
});
if !already {
rules.insert(
0,
json!({
"field": name,
"rule": "required",
"message": format!("{name} is required"),
}),
);
}
}
}
}
let mut errors = Vec::new();
for r in &rules {
let field = r.get("field").and_then(|v| v.as_str()).unwrap_or("");
let rule = r.get("rule").and_then(|v| v.as_str()).unwrap_or("");
let msg = r.get("message").and_then(|v| v.as_str()).unwrap_or("");
if field.is_empty() || rule.is_empty() {
continue;
}
if let Some(default_msg) = check_rule(rule, field, &data) {
errors.push(json!({
"field": field,
"message": if msg.is_empty() { default_msg } else { msg.to_string() },
}));
}
}
json!({
"ok": errors.is_empty(),
"errors": errors,
})
}
/// Form markup only (no document shell) — for page main-slot embed and `/_form`.
pub fn render_body(
form: &Value,
form_id: &str,
data: Option<&Value>,
errors: Option<&Value>,
csrf: Option<&str>,
) -> String {
let fields = form
.get("fields")
.and_then(|v| v.as_array())
.cloned()
.unwrap_or_default();
let data = data.map(data_map).unwrap_or_default();
let mut err_map = Map::new();
if let Some(Value::Array(errs)) = errors {
for e in errs {
if let (Some(f), Some(m)) = (
e.get("field").and_then(|v| v.as_str()),
e.get("message").and_then(|v| v.as_str()),
) {
err_map.insert(f.to_string(), json!(m));
}
}
} else if let Some(Value::Object(m)) = errors {
err_map = m.clone();
}
let action = form
.get("action")
.and_then(|v| v.as_str())
.unwrap_or("insert");
let table = form
.get("table")
.and_then(|v| v.as_str())
.unwrap_or("");
let heading = form
.get("title")
.and_then(|v| v.as_str())
.unwrap_or(form_id);
let post_to = form
.get("action_url")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.unwrap_or_else(|| format!("/_form/{form_id}"));
let cancel = form
.get("cancel_href")
.and_then(|v| v.as_str())
.unwrap_or("/");
let row_id = form.get("id").map(cell_str).unwrap_or_default();
let show_meta = form
.get("show_meta")
.and_then(|v| v.as_bool())
.unwrap_or(true);
let mut body = String::from("<div class=\"site-form\">");
if show_meta {
body.push_str(&format!(
"<p class=\"meta\">table=<code>{}</code> · action=<code>{}</code> · id=<code>{}</code></p>",
esc(table),
esc(action),
esc(heading)
));
}
body.push_str(&format!(
"<form method=\"post\" action=\"{}\"{}>",
esc(&post_to),
if fields.iter().any(|f| {
f.get("type")
.and_then(|v| v.as_str())
.is_some_and(|t| t.eq_ignore_ascii_case("file"))
}) {
" enctype=\"multipart/form-data\""
} else {
""
}
));
if let Some(token) = csrf.filter(|s| !s.is_empty()) {
body.push_str(&format!(
"<input type=\"hidden\" name=\"_csrf\" value=\"{}\"/>",
esc(token)
));
}
if action == "update" && !row_id.is_empty() {
body.push_str(&format!(
"<input type=\"hidden\" name=\"id\" value=\"{}\"/>",
esc(&row_id)
));
}
for f in &fields {
let name = f.get("name").and_then(|v| v.as_str()).unwrap_or("");
if name.is_empty() {
continue;
}
let label = f.get("label").and_then(|v| v.as_str()).unwrap_or(name);
let ty = f.get("type").and_then(|v| v.as_str()).unwrap_or("text");
let required = f.get("required").and_then(|v| v.as_bool()).unwrap_or(false);
let default = f.get("default").and_then(|v| v.as_str()).unwrap_or("");
let pk = f.get("pk").and_then(|v| v.as_bool()).unwrap_or(false);
let value = if data.contains_key(name) {
field_text(&data, name)
} else if name == "id" && !row_id.is_empty() {
row_id.clone()
} else {
default.to_string()
};
let readonly = action == "update" && pk && name == "id";
if readonly {
body.push_str(&format!(
"<label>{}<input type=\"text\" value=\"{}\" readonly/></label>",
esc(label),
esc(&value)
));
continue;
}
let req_attr = if required { " required" } else { "" };
body.push_str("<label>");
body.push_str(&esc(label));
if ty == "textarea" {
body.push_str(&format!(
"<textarea name=\"{}\" rows=\"5\"{}>{}</textarea>",
esc(name),
req_attr,
esc(&value)
));
} else {
let input_type = match ty {
"number" | "email" | "url" | "checkbox" | "file" => ty,
_ => "text",
};
if ty == "checkbox" {
let checked = matches!(value.as_str(), "1" | "true" | "on" | "yes");
body.push_str(&format!(
"<input type=\"checkbox\" name=\"{}\" value=\"1\"{}{}/>",
esc(name),
if checked { " checked" } else { "" },
req_attr
));
} else if ty == "file" {
body.push_str(&format!(
"<input type=\"file\" name=\"{}\"{}/>",
esc(name),
req_attr
));
} else {
body.push_str(&format!(
"<input type=\"{}\" name=\"{}\" value=\"{}\"{}/>",
esc(input_type),
esc(name),
esc(&value),
req_attr
));
}
}
if let Some(msg) = err_map.get(name).and_then(|v| v.as_str()) {
body.push_str(&format!("<span class=\"err\">{}</span>", esc(msg)));
}
body.push_str("</label>");
}
body.push_str(&format!(
"<div class=\"actions\"><button type=\"submit\">Submit</button><a href=\"{}\">cancel</a></div>",
esc(cancel)
));
body.push_str("</form></div>");
body
}
/// Standalone form document (`GET|POST /_form/{id}`).
pub fn render(
form: &Value,
form_id: &str,
data: Option<&Value>,
errors: Option<&Value>,
csrf: Option<&str>,
) -> String {
let heading = form
.get("title")
.and_then(|v| v.as_str())
.unwrap_or(form_id);
let mut body = String::from(
r#"<!DOCTYPE html><html lang="zh-CN"><head><meta charset="utf-8"/><meta name="viewport" content="width=device-width, initial-scale=1"/>"#,
);
body.push_str(&format!("<title>{}</title>", esc(heading)));
body.push_str(
r#"<style>
body{font-family:"IBM Plex Sans","Noto Sans SC",sans-serif;margin:1.5rem;background:#fafaf9;color:#1c1917}
.site-form form{max-width:28rem;display:grid;gap:.85rem}
.site-form label{display:grid;gap:.25rem;font-size:.9rem}
.site-form input,.site-form textarea{padding:.5rem .6rem;border:1px solid #e7e5e4;border-radius:4px;font:inherit}
.site-form input[readonly]{background:#f5f5f4;color:#57534e}
.site-form .err{color:#b91c1c;font-size:.85rem}
.site-form .actions{display:flex;gap:.75rem;align-items:center;flex-wrap:wrap}
.site-form button{background:#0f766e;color:#fff;border:0;padding:.55rem 1rem;border-radius:4px;cursor:pointer}
.site-form a{color:#0f766e}
.site-form .meta{color:#57534e;font-size:.9rem}
</style></head><body>"#,
);
body.push_str(&format!("<h1>{}</h1>", esc(heading)));
body.push_str(&render_body(form, form_id, data, errors, csrf));
body.push_str("</body></html>");
body
}
pub fn submit(form: &Value, data: &Value, db_url: &str) -> Result<Value, String> {
let action = form
.get("action")
.and_then(|v| v.as_str())
.unwrap_or("insert");
let mut row_map = data_map(data);
if action == "update" {
if let Some(fid) = form.get("id").map(cell_str).filter(|s| !s.is_empty()) {
if !row_map.contains_key("id") {
row_map.insert("id".to_string(), json!(fid));
}
}
}
let data = Value::Object(row_map.clone());
let v = validate(form, None, &data);
if !v.get("ok").and_then(|x| x.as_bool()).unwrap_or(false) {
return Ok(json!({
"ok": false,
"errors": v.get("errors").cloned().unwrap_or(json!([])),
}));
}
let table = form
.get("table")
.and_then(|v| v.as_str())
.ok_or_else(|| "form missing `table`".to_string())?;
let row = Value::Object(row_map);
let result = if action == "update" {
let id = form
.get("id")
.map(cell_str)
.filter(|s| !s.is_empty())
.or_else(|| data_map(&data).get("id").map(cell_str))
.ok_or_else(|| "update requires `id`".to_string())?;
db::update(db_url, table, &id, &row, None)?
} else {
db::insert(db_url, table, &json!([row]), None)?
};
let redirect = form
.get("redirect")
.and_then(|v| v.as_str())
.unwrap_or("/");
Ok(json!({
"ok": true,
"result": result,
"redirect": redirect,
}))
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn file_field_adds_multipart_enctype() {
let form = json!({
"table": "x",
"fields": [
{"name": "title", "label": "Title", "type": "text"},
{"name": "file", "label": "File", "type": "file"},
]
});
let html = render(&form, "up", None, None, None);
assert!(html.contains("enctype=\"multipart/form-data\""), "{html}");
assert!(html.contains("type=\"file\""), "{html}");
assert!(html.contains("name=\"file\""), "{html}");
}
}
File diff suppressed because it is too large. Load diff
-373
View File
@@ -1,373 +0,0 @@
//! HTTP → user `##` invoke routes for `app.invoke` / `configure invoke=`.
use std::sync::Arc;
use axum::extract::{Request, State};
use axum::http::{header, HeaderValue, StatusCode};
use axum::response::{IntoResponse, Response};
use axum::routing::{get, post};
use axum::Router;
use serde_json::{json, Map, Value};
use crate::middleware::{cell_text, JsonBody};
use crate::table;
use crate::call_lib_with_args;
/// One declared invoke mount.
#[derive(Clone, Debug)]
pub struct InvokeRoute {
pub method: String,
pub fn_path: String,
pub body_mode: String,
pub return_mode: String,
}
impl InvokeRoute {
pub fn from_json(v: &Value) -> Option<Self> {
let obj = v.as_object()?;
let fn_path = obj
.get("fn")
.or_else(|| obj.get("function"))
.and_then(|x| x.as_str())?
.trim()
.to_string();
if fn_path.is_empty() {
return None;
}
let method = obj
.get("method")
.and_then(|x| x.as_str())
.unwrap_or("POST")
.to_uppercase();
let body_mode = obj
.get("body")
.and_then(|x| x.as_str())
.unwrap_or("json")
.to_ascii_lowercase();
let return_mode = obj
.get("return")
.and_then(|x| x.as_str())
.unwrap_or("json")
.to_ascii_lowercase();
Some(Self {
method,
fn_path,
body_mode,
return_mode,
})
}
}
/// Normalize invoke-routes GFM table into `app.invoke_routes` map.
pub fn invoke_routes_from_table(table_v: &Value) -> Value {
let rows = table::as_rows(table_v);
let mut routes = Map::new();
if let Some(arr) = rows.as_array() {
for row in arr {
let col = |keys: &[&str]| -> String {
keys.iter()
.find_map(|k| row.get(*k).map(cell_text))
.unwrap_or_default()
};
let path = col(&["路径", "path", "Path"])
.trim()
.trim_start_matches('/')
.to_string();
if path.is_empty() {
continue;
}
let fn_path = col(&["函数", "fn", "function", "Function"]);
if fn_path.is_empty() {
continue;
}
let method = {
let m = col(&["方法", "method", "Method"]).to_uppercase();
if m.is_empty() {
"POST".into()
} else {
m
}
};
let body = {
let b = col(&["正文", "body", "Body"]).to_ascii_lowercase();
if b.is_empty() {
"json".into()
} else {
b
}
};
let ret = {
let r = col(&["返回", "return", "Return"]).to_ascii_lowercase();
if r.is_empty() {
"json".into()
} else {
r
}
};
routes.insert(
path,
json!({
"method": method,
"fn": fn_path,
"body": body,
"return": ret,
}),
);
}
}
Value::Object(routes)
}
pub fn mount_all(
mut app: Router<Arc<crate::http::AppState>>,
routes: &[(String, InvokeRoute)],
) -> Router<Arc<crate::http::AppState>> {
for (path, route) in routes {
app = mount_one(app, path, route.clone());
}
app
}
fn mount_one(
app: Router<Arc<crate::http::AppState>>,
path: &str,
route: InvokeRoute,
) -> Router<Arc<crate::http::AppState>> {
let r = route.clone();
match route.method.as_str() {
"GET" => app.route(
path,
get(move |State(st): State<Arc<crate::http::AppState>>, req: Request| {
let r = r.clone();
async move { invoke_handler(st, r, req).await }
}),
),
_ => app.route(
path,
post(move |State(st): State<Arc<crate::http::AppState>>, req: Request| {
let r = r.clone();
async move { invoke_handler(st, r, req).await }
}),
),
}
}
async fn invoke_handler(
_st: Arc<crate::http::AppState>,
route: InvokeRoute,
req: Request,
) -> Response {
let (parts, body) = req.into_parts();
let query_map = query_to_map(parts.uri.query());
let payload = match route.body_mode.as_str() {
"query" => Value::Object(query_map.clone()),
"raw" => {
let bytes = match axum::body::to_bytes(body, 8 * 1024 * 1024).await {
Ok(b) => b,
Err(e) => {
return err_json(StatusCode::BAD_REQUEST, &format!("body: {e}"));
}
};
json!({ "payload": String::from_utf8_lossy(&bytes).to_string() })
}
"form" => {
let bytes = match axum::body::to_bytes(body, 8 * 1024 * 1024).await {
Ok(b) => b,
Err(e) => {
return err_json(StatusCode::BAD_REQUEST, &format!("body: {e}"));
}
};
form_to_map(&bytes)
}
_ => {
// json (default)
let bytes = match axum::body::to_bytes(body, 8 * 1024 * 1024).await {
Ok(b) => b,
Err(e) => {
return err_json(StatusCode::BAD_REQUEST, &format!("body: {e}"));
}
};
if bytes.is_empty() {
Value::Object(query_map.clone())
} else {
match serde_json::from_slice::<Value>(&bytes) {
Ok(Value::Object(mut m)) => {
for (k, v) in query_map {
m.entry(k).or_insert(v);
}
Value::Object(m)
}
Ok(other) => json!({ "payload": other }),
Err(e) => {
return err_json(StatusCode::BAD_REQUEST, &format!("json: {e}"));
}
}
}
}
};
// Bind JSON object keys as named args; always offer `payload` too.
let mut args = Map::new();
if let Value::Object(ref m) = payload {
for (k, v) in m {
args.insert(k.clone(), v.clone());
}
}
if !args.contains_key("payload") {
args.insert("payload".to_string(), payload.clone());
}
let fn_path = route.fn_path.clone();
let result = tokio::task::spawn_blocking(move || call_lib_with_args(&fn_path, &Value::Object(args)))
.await;
let value = match result {
Ok(Ok(v)) => v,
Ok(Err(e)) => {
let status = if e.contains("unknown library")
|| e.contains("call_lib_path: need")
|| e.contains("not allowed")
|| e.contains("no site")
{
StatusCode::BAD_REQUEST
} else {
StatusCode::INTERNAL_SERVER_ERROR
};
return err_json(status, &e);
}
Err(e) => {
return err_json(StatusCode::INTERNAL_SERVER_ERROR, &format!("join: {e}"));
}
};
match route.return_mode.as_str() {
"text" => {
if matches!(value, Value::Null) {
return err_json(StatusCode::INTERNAL_SERVER_ERROR, "null result");
}
let text = match &value {
Value::String(s) => s.clone(),
other => other.to_string(),
};
(
StatusCode::OK,
[(
header::CONTENT_TYPE,
HeaderValue::from_static("text/plain; charset=utf-8"),
)],
text,
)
.into_response()
}
"status" => {
if matches!(value, Value::Null) {
return err_json(StatusCode::INTERNAL_SERVER_ERROR, "null result");
}
StatusCode::NO_CONTENT.into_response()
}
_ => {
let body = match json_wrap_invoke_value(value) {
Ok((status, body)) => {
let mut resp = JsonBody(body).into_response();
*resp.status_mut() = status;
resp
}
Err(resp) => resp,
};
body
}
}
}
/// Map host return value to invoke JSON. `Null` must not look like success (GAP-08).
pub(crate) fn json_wrap_invoke_value(
value: Value,
) -> Result<(StatusCode, Value), Response> {
match value {
Value::Null => Err(err_json(
StatusCode::INTERNAL_SERVER_ERROR,
"null result",
)),
Value::Object(_) => Ok((StatusCode::OK, value)),
other => Ok((StatusCode::OK, json!({ "ok": true, "result": other }))),
}
}
fn query_to_map(q: Option<&str>) -> Map<String, Value> {
let mut m = Map::new();
let Some(q) = q else {
return m;
};
for pair in q.split('&') {
if pair.is_empty() {
continue;
}
let (k, v) = pair.split_once('=').unwrap_or((pair, ""));
let k = urlencoding_decode(k);
let v = urlencoding_decode(v);
if !k.is_empty() {
m.insert(k, json!(v));
}
}
m
}
fn form_to_map(bytes: &[u8]) -> Value {
let s = String::from_utf8_lossy(bytes);
Value::Object(query_to_map(Some(&s)))
}
fn urlencoding_decode(s: &str) -> String {
let mut out = String::with_capacity(s.len());
let b = s.as_bytes();
let mut i = 0;
while i < b.len() {
match b[i] {
b'+' => {
out.push(' ');
i += 1;
}
b'%' if i + 2 < b.len() => {
let h = u8::from_str_radix(std::str::from_utf8(&b[i + 1..i + 3]).unwrap_or(""), 16);
if let Ok(c) = h {
out.push(c as char);
i += 3;
} else {
out.push('%');
i += 1;
}
}
c => {
out.push(c as char);
i += 1;
}
}
}
out
}
fn err_json(status: StatusCode, msg: &str) -> Response {
(
status,
JsonBody(json!({ "ok": false, "error": msg })),
)
.into_response()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn null_result_is_not_fake_ok() {
let err = json_wrap_invoke_value(Value::Null).expect_err("null must fail");
assert_eq!(err.status(), StatusCode::INTERNAL_SERVER_ERROR);
let ok = json_wrap_invoke_value(json!({ "hits": [] })).expect("object ok");
assert_eq!(ok.0, StatusCode::OK);
let wrap = json_wrap_invoke_value(json!(1)).expect("scalar wrap");
assert_eq!(wrap.0, StatusCode::OK);
assert_eq!(wrap.1["ok"], true);
assert_eq!(wrap.1["result"], 1);
}
}
File diff suppressed because it is too large. Load diff
-16
View File
@@ -1,16 +0,0 @@
//! Markdown → HTML for article bodies (GFM subset via pulldown-cmark).
use pulldown_cmark::{html, Options, Parser};
pub fn to_html(md: &str) -> String {
let mut opts = Options::empty();
opts.insert(
Options::ENABLE_STRIKETHROUGH
| Options::ENABLE_TABLES
| Options::ENABLE_TASKLISTS,
);
let parser = Parser::new_ext(md, opts);
let mut html_out = String::new();
html::push_html(&mut html_out, parser);
format!("<div class=\"article-body md\">{html_out}</div>")
}
-487
View File
@@ -1,487 +0,0 @@
//! Declarative middleware for the web app.
//!
//! All cross-cutting HTTP capabilities (CORS, security headers, compression,
//! body limits, JSON API routes) are configured as *data* on the `app` object
//! and applied to the axum router here. This keeps the authoring surface
//! declarative: a GFM table in Marqdo becomes a `middleware` map, and this
//! module turns it into axum layers — 配置即数据、装配即函数.
use std::sync::Arc;
use axum::body::Body;
use axum::extract::State;
use axum::http::{header, HeaderValue, Method, Request, StatusCode};
use axum::middleware::{self as axum_mw, Next};
use axum::response::{IntoResponse, Response};
use axum::routing::{get, post};
use axum::Router;
use serde_json::{json, Map, Value};
use tower_http::compression::CompressionLayer;
use tower_http::cors::{Any, CorsLayer};
use tower_http::limit::RequestBodyLimitLayer;
use tower_http::set_header::SetResponseHeaderLayer;
use crate::db;
/// Render a cell value as text (strings stay, numbers/bools stringify).
pub fn cell_text(v: &Value) -> String {
match v {
Value::Null => String::new(),
Value::String(s) => s.clone(),
Value::Bool(b) => b.to_string(),
Value::Number(n) => n.to_string(),
other => other.to_string(),
}
}
/// Parsed middleware configuration (from `app["middleware"]`).
#[derive(Clone, Default)]
pub struct Middleware {
pub cors: Option<CorsConfig>,
pub security: Vec<(String, String)>,
pub compress: bool,
pub body_limit: Option<u64>,
/// Log `METHOD path status duration_ms` to stderr for each request.
pub access_log: bool,
/// Global `Cache-Control` response header (e.g. `public, max-age=3600`).
pub cache_control: Option<String>,
/// JSON API routes: `path -> { method, table, where?, order?, limit? }`.
pub json_routes: Vec<(String, JsonRoute)>,
}
#[derive(Clone)]
pub struct CorsConfig {
pub allow_origins: Vec<String>,
pub methods: Vec<String>,
pub headers: Vec<String>,
pub expose_headers: Vec<String>,
pub credentials: bool,
}
#[derive(Clone)]
pub struct JsonRoute {
pub method: String,
pub table: String,
pub where_v: Option<Value>,
pub order: Option<String>,
pub limit: i64,
}
/// Parse `app["middleware"]` (a map) into a `Middleware`.
pub fn parse(app: &Value) -> Middleware {
let mut mw = Middleware::default();
let Some(obj) = app.get("middleware").and_then(|v| v.as_object()) else {
return mw;
};
if let Some(c) = obj.get("cors") {
mw.cors = parse_cors(c);
}
if let Some(sec) = obj.get("security").and_then(|v| v.as_object()) {
mw.security = sec
.iter()
.map(|(k, v)| (k.clone(), v.as_str().unwrap_or("").to_string()))
.filter(|(_, v)| !v.is_empty())
.collect();
}
mw.compress = obj
.get("compress")
.and_then(|v| v.as_bool())
.unwrap_or(false);
mw.access_log = obj
.get("access_log")
.and_then(|v| v.as_bool())
.unwrap_or(false);
mw.cache_control = obj
.get("cache_control")
.and_then(|v| v.as_str())
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty());
mw.body_limit = obj
.get("body_limit")
.and_then(|v| v.as_u64().or_else(|| v.as_i64().map(|i| i as u64)))
.filter(|n| *n > 0);
if let Some(routes) = obj.get("json_routes").and_then(|v| v.as_object()) {
for (path, spec) in routes {
if let Some(route) = parse_json_route(path, spec) {
mw.json_routes.push(route);
}
}
mw.json_routes.sort_by(|a, b| a.0.cmp(&b.0));
}
mw
}
fn parse_cors(v: &Value) -> Option<CorsConfig> {
let obj = v.as_object()?;
let str_list = |key: &str| -> Vec<String> {
obj.get(key)
.and_then(|v| v.as_array())
.map(|arr| {
arr.iter()
.filter_map(|x| x.as_str().map(|s| s.to_string()))
.collect()
})
.unwrap_or_default()
};
Some(CorsConfig {
allow_origins: str_list("allow_origins"),
methods: str_list("methods"),
headers: str_list("headers"),
expose_headers: str_list("expose_headers"),
credentials: obj
.get("credentials")
.and_then(|v| v.as_bool())
.unwrap_or(false),
})
}
fn parse_json_route(path: &str, spec: &Value) -> Option<(String, JsonRoute)> {
let spec = spec.as_object()?;
let method = spec
.get("method")
.and_then(|v| v.as_str())
.unwrap_or("GET")
.to_uppercase();
let table = spec.get("table")?.as_str()?.to_string();
let order = spec.get("order").and_then(|v| v.as_str()).map(|s| s.to_string());
let limit = spec
.get("limit")
.and_then(|v| v.as_i64())
.unwrap_or(200);
Some((
format!("/{}", path.trim_start_matches('/')),
JsonRoute {
method,
table,
where_v: spec.get("where").cloned(),
order,
limit,
},
))
}
/// Apply the middleware configuration to a router (state already provided).
pub fn apply(
router: Router<Arc<crate::http::AppState>>,
mw: &Middleware,
) -> Router<Arc<crate::http::AppState>> {
let mut app = router;
// JSON API routes first: axum layers only wrap routes registered before
// the `.layer()` call, so these must be mounted ahead of the pipeline.
for (path, route) in &mw.json_routes {
app = mount_json(app, path, route.clone());
}
// CORS
if let Some(c) = &mw.cors {
let mut layer = CorsLayer::new();
if c.allow_origins.is_empty() {
layer = layer.allow_origin(Any);
} else {
layer = layer.allow_origin(
c.allow_origins
.iter()
.filter_map(|o| o.parse::<HeaderValue>().ok())
.collect::<Vec<_>>(),
);
}
if c.methods.is_empty() {
layer = layer.allow_methods(Any);
} else {
layer = layer.allow_methods(
c.methods
.iter()
.filter_map(|m| m.parse::<Method>().ok())
.collect::<Vec<_>>(),
);
}
if c.headers.is_empty() {
layer = layer.allow_headers(Any);
} else {
layer = layer.allow_headers(
c.headers
.iter()
.filter_map(|h| h.parse::<axum::http::HeaderName>().ok())
.collect::<Vec<_>>(),
);
}
if !c.expose_headers.is_empty() {
layer = layer.expose_headers(
c.expose_headers
.iter()
.filter_map(|h| h.parse::<axum::http::HeaderName>().ok())
.collect::<Vec<_>>(),
);
}
layer = layer.allow_credentials(c.credentials);
app = app.layer(layer);
}
// Security response headers
for (name, value) in &mw.security {
let Some(header_name) = header_name_from(&name.to_ascii_lowercase()) else {
continue;
};
if let Ok(v) = HeaderValue::from_str(value) {
app = app.layer(SetResponseHeaderLayer::overriding(header_name, v));
}
}
// Compression
if mw.compress {
app = app.layer(CompressionLayer::new());
}
if let Some(ref cc) = mw.cache_control {
if let Ok(v) = HeaderValue::from_str(cc) {
app = app.layer(SetResponseHeaderLayer::overriding(
header::CACHE_CONTROL,
v,
));
}
}
// Request body limit
if let Some(bytes) = mw.body_limit {
app = app.layer(RequestBodyLimitLayer::new(bytes as usize));
}
// Access log (outermost so status/duration include other layers).
if mw.access_log {
app = app.layer(axum_mw::from_fn(access_log_mw));
}
app
}
async fn access_log_mw(req: Request<Body>, next: Next) -> Response {
let method = req.method().clone();
let path = req.uri().path().to_string();
let start = std::time::Instant::now();
let resp = next.run(req).await;
eprintln!(
"marqdo web {} {} {} {}ms",
method,
path,
resp.status().as_u16(),
start.elapsed().as_millis()
);
resp
}
fn header_name_from(name: &str) -> Option<axum::http::HeaderName> {
let h = match name {
"x_frame_options" | "x-frame-options" => header::X_FRAME_OPTIONS,
"content_security_policy" | "content-security-policy" => header::CONTENT_SECURITY_POLICY,
"x_content_type_options" | "x-content-type-options" => header::X_CONTENT_TYPE_OPTIONS,
"referrer_policy" | "referrer-policy" => header::REFERRER_POLICY,
"strict_transport_security" | "strict-transport-security" | "hsts" => {
header::STRICT_TRANSPORT_SECURITY
}
"x_xss_protection" | "x-xss-protection" => header::X_XSS_PROTECTION,
_ => return None,
};
Some(h)
}
/// Mount a JSON API route backed by a db query.
fn mount_json(
app: Router<Arc<crate::http::AppState>>,
path: &str,
route: JsonRoute,
) -> Router<Arc<crate::http::AppState>> {
let handler = match route.method.as_str() {
"POST" => post(move |State(st): State<Arc<crate::http::AppState>>| {
json_handler(st, route.clone())
}),
_ => get(move |State(st): State<Arc<crate::http::AppState>>| {
json_handler(st, route.clone())
}),
};
app.route(path, handler)
}
async fn json_handler(st: Arc<crate::http::AppState>, route: JsonRoute) -> Response {
let Some(url) = st.db_url.as_deref() else {
return (
StatusCode::INTERNAL_SERVER_ERROR,
JsonBody(json!({ "error": "no database" })),
)
.into_response();
};
let res = match &route.order {
Some(o) if !o.trim().is_empty() => {
db::select_order(url, &route.table, route.limit, route.where_v.as_ref(), Some(o), None, None)
}
_ => db::select(url, &route.table, route.limit, route.where_v.as_ref()),
};
match res {
Ok(data) => JsonBody(data).into_response(),
Err(e) => (
StatusCode::INTERNAL_SERVER_ERROR,
JsonBody(json!({ "error": e })),
)
.into_response(),
}
}
/// A JSON body response wrapper (sets `Content-Type: application/json`).
pub struct JsonBody(pub Value);
impl IntoResponse for JsonBody {
fn into_response(self) -> Response {
Response::builder()
.status(StatusCode::OK)
.header(header::CONTENT_TYPE, "application/json; charset=utf-8")
.body(Body::from(self.0.to_string()))
.unwrap()
}
}
/// Normalize a `security` table into the `middleware.security` map shape.
/// Accepts a GFM table `|头|值|` (list of rows) or an already-shaped map.
pub fn security_from_table(table: &Value) -> Value {
let rows = crate::table::as_rows(table);
let mut m = Map::new();
if let Some(arr) = rows.as_array() {
for row in arr {
let col = |keys: &[&str]| -> String {
keys.iter()
.find_map(|k| row.get(*k).map(cell_text))
.unwrap_or_default()
};
let k = col(&["头", "header", "Header", "name"]);
let v = col(&["值", "value", "Value"]);
if !k.is_empty() {
m.insert(k, json!(v));
}
}
}
Value::Object(m)
}
/// Normalize a CORS table `|允许来源|方法|头|暴露头|凭证|` (one row per origin)
/// into the `middleware.cors` map shape.
pub fn cors_from_table(table: &Value) -> Value {
let rows = crate::table::as_rows(table);
let mut allow_origins = Vec::new();
let mut methods = Vec::new();
let mut headers = Vec::new();
let mut expose_headers = Vec::new();
let mut credentials = false;
if let Some(arr) = rows.as_array() {
for row in arr {
let col = |keys: &[&str]| -> String {
keys.iter()
.find_map(|k| row.get(*k).map(cell_text))
.unwrap_or_default()
};
let origin = col(&["允许来源", "origin", "Origin"]);
if !origin.is_empty() && origin != "*" {
allow_origins.push(origin);
}
for m in col(&["方法", "methods", "Methods"]).split(',') {
let m = m.trim();
if !m.is_empty() {
methods.push(m.to_string());
}
}
for h in col(&["头", "headers", "Headers"]).split(',') {
let h = h.trim();
if !h.is_empty() {
headers.push(h.to_string());
}
}
for h in col(&["暴露头", "expose_headers", "Expose-Headers"]).split(',') {
let h = h.trim();
if !h.is_empty() {
expose_headers.push(h.to_string());
}
}
let cred = col(&["凭证", "credentials", "Credentials"]);
if cred.eq_ignore_ascii_case("true")
|| cred == "1"
|| cred.eq_ignore_ascii_case("yes")
{
credentials = true;
}
}
}
json!({
"allow_origins": allow_origins,
"methods": methods,
"headers": headers,
"expose_headers": expose_headers,
"credentials": credentials,
})
}
/// Normalize a JSON-routes table into `middleware.json_routes` map shape.
/// Accepts a GFM table `|路径|方法|表|条件|排序|上限|` (rows).
pub fn json_routes_from_table(table: &Value) -> Value {
let rows = crate::table::as_rows(table);
let mut routes = Map::new();
if let Some(arr) = rows.as_array() {
for row in arr {
let col = |keys: &[&str]| -> String {
keys.iter()
.find_map(|k| row.get(*k).map(cell_text))
.unwrap_or_default()
};
let path = col(&["路径", "path", "Path"]).trim_start_matches('/').to_string();
if path.is_empty() {
continue;
}
let method = col(&["方法", "method", "Method"]).to_uppercase();
let method = if method.is_empty() { "GET".into() } else { method };
let table = col(&["表", "table", "Table"]);
if table.is_empty() {
continue;
}
let order = col(&["排序", "order", "Order"]);
let limit = col(&["上限", "limit", "Limit"]).parse::<i64>().unwrap_or(200);
let mut spec = Map::new();
spec.insert("method".into(), json!(method));
spec.insert("table".into(), json!(table));
if !order.is_empty() {
spec.insert("order".into(), json!(order));
}
spec.insert("limit".into(), json!(limit));
routes.insert(path, Value::Object(spec));
}
}
Value::Object(routes)
}
/// Read the `middleware` map out of an app object (for logging/debug).
pub fn summary(app: &Value) -> String {
let mw = parse(app);
let mut parts = Vec::new();
if mw.cors.is_some() {
parts.push("cors".to_string());
}
if !mw.security.is_empty() {
parts.push("security".to_string());
}
if mw.compress {
parts.push("compress".to_string());
}
if mw.body_limit.is_some() {
parts.push("body_limit".to_string());
}
if mw.access_log {
parts.push("access_log".to_string());
}
if mw.cache_control.is_some() {
parts.push("cache_control".to_string());
}
if !mw.json_routes.is_empty() {
parts.push(format!("json:{}", mw.json_routes.len()));
}
if parts.is_empty() {
String::new()
} else {
parts.join(" + ")
}
}
-64
View File
@@ -1,64 +0,0 @@
//! Password hashing (argon2id) for admin users and auth tables.
use argon2::{
password_hash::{PasswordHash, PasswordHasher, PasswordVerifier, SaltString},
Argon2,
};
use getrandom::getrandom;
/// True when `stored` looks like an argon2 PHC string.
pub fn is_hashed(stored: &str) -> bool {
stored.starts_with("$argon2")
}
/// Hash a plaintext password (argon2id, random salt).
pub fn hash_password(plain: &str) -> Result<String, String> {
if plain.is_empty() {
return Err("password must not be empty".into());
}
let mut salt_bytes = [0u8; 16];
getrandom(&mut salt_bytes).map_err(|e| format!("getrandom: {e}"))?;
let salt = SaltString::encode_b64(&salt_bytes).map_err(|e| e.to_string())?;
let argon2 = Argon2::default();
argon2
.hash_password(plain.as_bytes(), &salt)
.map(|h| h.to_string())
.map_err(|e| e.to_string())
}
/// Verify plaintext against stored hash or legacy plaintext (dev / gold tests).
pub fn verify_password(plain: &str, stored: &str) -> bool {
if stored.is_empty() || plain.is_empty() {
return false;
}
if is_hashed(stored) {
let parsed = match PasswordHash::new(stored) {
Ok(h) => h,
Err(_) => return false,
};
Argon2::default()
.verify_password(plain.as_bytes(), &parsed)
.is_ok()
} else {
plain == stored
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn hash_and_verify_roundtrip() {
let hash = hash_password("secret").unwrap();
assert!(is_hashed(&hash));
assert!(verify_password("secret", &hash));
assert!(!verify_password("wrong", &hash));
}
#[test]
fn legacy_plaintext_still_works() {
assert!(verify_password("secret", "secret"));
assert!(!verify_password("other", "secret"));
}
}
-471
View File
@@ -1,471 +0,0 @@
//! Streaming reverse proxy routes for `app.proxy` / `configure proxy=`.
//!
//! Browser → Marqdo (same origin) → upstream, piping response bodies
//! (including `text/event-stream`) without buffering the full stream.
use std::sync::Arc;
use std::time::Duration;
use axum::body::Body;
use axum::extract::{Request, State};
use axum::http::{header, HeaderMap, HeaderName, HeaderValue, StatusCode};
use axum::response::{IntoResponse, Response};
use axum::routing::{MethodFilter, MethodRouter};
use axum::Router;
use futures_util::TryStreamExt;
use serde_json::{json, Map, Value};
use crate::middleware::cell_text;
use crate::table;
/// One declared proxy mount.
#[derive(Clone, Debug)]
pub struct ProxyRoute {
pub upstream: String,
pub stream: bool,
pub strip_prefix: String,
pub methods: Vec<String>,
/// `ENV_NAME=Header-Name` pairs injected on the upstream request.
pub headers_from_env: Vec<(String, String)>,
pub timeout_ms: u64,
}
impl ProxyRoute {
pub fn from_json(v: &Value) -> Option<Self> {
let obj = v.as_object()?;
let upstream = obj.get("upstream")?.as_str()?.trim().to_string();
if upstream.is_empty() {
return None;
}
let stream = obj
.get("stream")
.and_then(|x| x.as_bool())
.unwrap_or(true);
let strip_prefix = obj
.get("strip_prefix")
.and_then(|x| x.as_str())
.unwrap_or("")
.to_string();
let methods = obj
.get("methods")
.and_then(|x| x.as_array())
.map(|arr| {
arr.iter()
.filter_map(|m| m.as_str().map(|s| s.to_uppercase()))
.filter(|s| !s.is_empty())
.collect::<Vec<_>>()
})
.filter(|v| !v.is_empty())
.unwrap_or_else(|| vec!["POST".into()]);
let headers_from_env = parse_headers_from_env_value(obj.get("headers_from_env"));
let timeout_ms = obj
.get("timeout_ms")
.and_then(|x| x.as_u64().or_else(|| x.as_i64().map(|i| i as u64)))
.unwrap_or(120_000)
.max(1);
Some(Self {
upstream,
stream,
strip_prefix,
methods,
headers_from_env,
timeout_ms,
})
}
}
fn parse_headers_from_env_value(v: Option<&Value>) -> Vec<(String, String)> {
let Some(v) = v else {
return Vec::new();
};
match v {
Value::String(s) => parse_headers_from_env_str(s),
Value::Array(arr) => arr
.iter()
.filter_map(|x| x.as_str())
.flat_map(parse_headers_from_env_str)
.collect(),
_ => Vec::new(),
}
}
fn parse_headers_from_env_str(s: &str) -> Vec<(String, String)> {
s.split(|c| c == ',' || c == ';')
.filter_map(|part| {
let part = part.trim();
if part.is_empty() {
return None;
}
let (env, hdr) = part.split_once('=')?;
let env = env.trim();
let hdr = hdr.trim();
if env.is_empty() || hdr.is_empty() {
return None;
}
Some((env.to_string(), hdr.to_string()))
})
.collect()
}
/// Expand `$VAR` / `${VAR}` in upstream URL strings.
pub fn expand_env(s: &str) -> String {
let mut out = String::with_capacity(s.len());
let bytes = s.as_bytes();
let mut i = 0;
while i < bytes.len() {
if bytes[i] == b'$' {
if i + 1 < bytes.len() && bytes[i + 1] == b'{' {
if let Some(end) = s[i + 2..].find('}') {
let name = &s[i + 2..i + 2 + end];
out.push_str(&std::env::var(name).unwrap_or_default());
i += 3 + end;
continue;
}
} else {
let start = i + 1;
let mut end = start;
while end < bytes.len()
&& (bytes[end].is_ascii_alphanumeric() || bytes[end] == b'_')
{
end += 1;
}
if end > start {
let name = &s[start..end];
out.push_str(&std::env::var(name).unwrap_or_default());
i = end;
continue;
}
}
}
out.push(bytes[i] as char);
i += 1;
}
out
}
/// Normalize a proxy-routes GFM table into `app.proxy_routes` map shape.
pub fn proxy_routes_from_table(table_v: &Value) -> Value {
let rows = table::as_rows(table_v);
let mut routes = Map::new();
if let Some(arr) = rows.as_array() {
for row in arr {
let col = |keys: &[&str]| -> String {
keys.iter()
.find_map(|k| row.get(*k).map(cell_text))
.unwrap_or_default()
};
let path = normalize_path(&col(&["路径", "path", "Path"]));
if path.is_empty() || path == "/" {
continue;
}
let upstream = col(&["上游", "upstream", "Upstream"]);
if upstream.is_empty() {
continue;
}
let stream = {
let s = col(&["流式", "stream", "Stream"]);
if s.is_empty() {
true
} else {
matches!(
s.to_ascii_lowercase().as_str(),
"true" | "1" | "yes" | "on" | "真"
)
}
};
let strip_prefix = col(&["去前缀", "strip_prefix", "Strip-Prefix"]);
let methods_raw = col(&["方法", "methods", "Methods"]);
let methods: Vec<String> = if methods_raw.is_empty() {
vec!["POST".into()]
} else {
methods_raw
.split(',')
.map(|m| m.trim().to_uppercase())
.filter(|m| !m.is_empty())
.collect()
};
let headers_from_env = col(&[
"环境头",
"headers_from_env",
"Headers-From-Env",
]);
let timeout_ms = col(&["超时", "timeout_ms", "Timeout"])
.parse::<u64>()
.unwrap_or(120_000);
routes.insert(
path.trim_start_matches('/').to_string(),
json!({
"upstream": upstream,
"stream": stream,
"strip_prefix": strip_prefix,
"methods": methods,
"headers_from_env": headers_from_env,
"timeout_ms": timeout_ms,
}),
);
}
}
Value::Object(routes)
}
fn normalize_path(raw: &str) -> String {
let s = raw.trim();
if s.is_empty() {
return String::new();
}
if s.starts_with('/') {
s.to_string()
} else {
format!("/{s}")
}
}
/// Mount all proxy routes onto the router.
pub fn mount_all(
mut app: Router<Arc<crate::http::AppState>>,
routes: &[(String, ProxyRoute)],
) -> Router<Arc<crate::http::AppState>> {
for (path, route) in routes {
app = mount_one(app, path, route.clone());
}
app
}
fn mount_one(
app: Router<Arc<crate::http::AppState>>,
path: &str,
route: ProxyRoute,
) -> Router<Arc<crate::http::AppState>> {
let mut filter: Option<MethodFilter> = None;
for m in &route.methods {
let f = match m.as_str() {
"GET" => MethodFilter::GET,
"POST" => MethodFilter::POST,
"PUT" => MethodFilter::PUT,
"PATCH" => MethodFilter::PATCH,
"DELETE" => MethodFilter::DELETE,
"HEAD" => MethodFilter::HEAD,
"OPTIONS" => MethodFilter::OPTIONS,
_ => continue,
};
filter = Some(match filter {
None => f,
Some(acc) => acc.or(f),
});
}
let filter = filter.unwrap_or(MethodFilter::POST);
let r = route.clone();
let handler = MethodRouter::new().on(filter, move |State(_st): State<Arc<crate::http::AppState>>, req: Request| {
let r = r.clone();
async move { proxy_handler(r, req).await }
});
app.route(path, handler)
}
async fn proxy_handler(route: ProxyRoute, req: Request) -> Response {
let method = req.method().clone();
let uri = req.uri().clone();
let in_headers = req.headers().clone();
let body_bytes = match axum::body::to_bytes(req.into_body(), 64 * 1024 * 1024).await {
Ok(b) => b,
Err(e) => {
return (
StatusCode::BAD_REQUEST,
format!("read body: {e}"),
)
.into_response();
}
};
let target = match build_upstream_url(&route, uri.path(), uri.query()) {
Ok(u) => u,
Err(e) => return (StatusCode::BAD_GATEWAY, e).into_response(),
};
let client = match reqwest::Client::builder()
.timeout(Duration::from_millis(route.timeout_ms))
.build()
{
Ok(c) => c,
Err(e) => {
return (
StatusCode::INTERNAL_SERVER_ERROR,
format!("client: {e}"),
)
.into_response();
}
};
let mut builder = client.request(
reqwest::Method::from_bytes(method.as_str().as_bytes()).unwrap_or(reqwest::Method::POST),
&target,
);
// Forward selected inbound headers (skip hop-by-hop).
for (name, value) in in_headers.iter() {
let lname = name.as_str().to_ascii_lowercase();
if is_hop_by_hop(&lname) || lname == "host" {
continue;
}
if let Ok(v) = value.to_str() {
builder = builder.header(name.as_str(), v);
}
}
// Inject secrets from env (never from the browser).
for (env_name, header_name) in &route.headers_from_env {
if let Ok(val) = std::env::var(env_name) {
if !val.is_empty() {
// Common pattern: Authorization Bearer
let hdr_val = if header_name.eq_ignore_ascii_case("authorization")
&& !val.to_ascii_lowercase().starts_with("bearer ")
{
format!("Bearer {val}")
} else {
val
};
builder = builder.header(header_name.as_str(), hdr_val);
}
}
}
if !body_bytes.is_empty() {
builder = builder.body(body_bytes);
}
let upstream = match builder.send().await {
Ok(r) => r,
Err(e) => {
return (StatusCode::BAD_GATEWAY, format!("upstream: {e}")).into_response();
}
};
let status = StatusCode::from_u16(upstream.status().as_u16()).unwrap_or(StatusCode::BAD_GATEWAY);
let mut out_headers = HeaderMap::new();
for (name, value) in upstream.headers().iter() {
let lname = name.as_str().to_ascii_lowercase();
if is_hop_by_hop(&lname) {
continue;
}
if let (Ok(n), Ok(v)) = (
HeaderName::from_bytes(name.as_str().as_bytes()),
HeaderValue::from_bytes(value.as_bytes()),
) {
out_headers.append(n, v);
}
}
// Ensure SSE content-type is preserved if present.
if !out_headers.contains_key(header::CONTENT_TYPE) {
if let Some(ct) = upstream.headers().get(reqwest::header::CONTENT_TYPE) {
if let Ok(v) = HeaderValue::from_bytes(ct.as_bytes()) {
out_headers.insert(header::CONTENT_TYPE, v);
}
}
}
if route.stream {
// Industry SSE / reverse-proxy practice: disable buffering at CDN/Nginx.
out_headers.insert(
header::CACHE_CONTROL,
HeaderValue::from_static("no-cache, no-transform"),
);
out_headers.insert(
HeaderName::from_static("x-accel-buffering"),
HeaderValue::from_static("no"),
);
let stream = upstream.bytes_stream().map_err(|e| {
std::io::Error::new(std::io::ErrorKind::Other, e)
});
let body = Body::from_stream(stream);
let mut resp = Response::new(body);
*resp.status_mut() = status;
*resp.headers_mut() = out_headers;
resp
} else {
let bytes = match upstream.bytes().await {
Ok(b) => b,
Err(e) => {
return (StatusCode::BAD_GATEWAY, format!("upstream body: {e}")).into_response();
}
};
let mut resp = Response::new(Body::from(bytes));
*resp.status_mut() = status;
*resp.headers_mut() = out_headers;
resp
}
}
fn build_upstream_url(
route: &ProxyRoute,
req_path: &str,
query: Option<&str>,
) -> Result<String, String> {
let base = expand_env(&route.upstream);
if base.is_empty() {
return Err("proxy upstream empty after env expand".into());
}
// If upstream already looks like a full URL with path (contains scheme and
// more than origin), and strip_prefix empties the remainder, use base as-is.
let mut path = req_path.to_string();
let strip = route.strip_prefix.trim();
if !strip.is_empty() {
let strip_n = normalize_path(strip);
if path.starts_with(&strip_n) {
path = path[strip_n.len()..].to_string();
if path.is_empty() {
path = "/".into();
} else if !path.starts_with('/') {
path = format!("/{path}");
}
}
}
let url = if base.contains("://") {
let trimmed = base.trim_end_matches('/');
// If base has no path beyond origin, append path; if base includes a
// path suffix, append remaining path.
if let Some(idx) = trimmed.find("://") {
let after = &trimmed[idx + 3..];
if after.contains('/') {
// base includes path — append only if remaining path is not "/"
if path == "/" {
trimmed.to_string()
} else {
format!("{trimmed}{path}")
}
} else {
format!("{trimmed}{path}")
}
} else {
format!("{trimmed}{path}")
}
} else {
return Err(format!("proxy upstream must be absolute URL, got `{base}`"));
};
Ok(match query {
Some(q) if !q.is_empty() => {
if url.contains('?') {
format!("{url}&{q}")
} else {
format!("{url}?{q}")
}
}
_ => url,
})
}
fn is_hop_by_hop(name: &str) -> bool {
matches!(
name,
"connection"
| "keep-alive"
| "proxy-authenticate"
| "proxy-authorization"
| "te"
| "trailers"
| "transfer-encoding"
| "upgrade"
| "content-length"
)
}
@@ -1,95 +0,0 @@
//! Login rate limiting — max failures per IP+username within a window.
use std::collections::HashMap;
use std::sync::Mutex;
use std::time::{SystemTime, UNIX_EPOCH};
const MAX_FAILS: u32 = 5;
const WINDOW_SEC: u64 = 900; // 15 minutes
struct Window {
count: u32,
start: u64,
}
static ATTEMPTS: Mutex<Option<HashMap<String, Window>>> = Mutex::new(None);
fn now_secs() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0)
}
fn key(ip: &str, username: &str) -> String {
format!("{ip}|{username}")
}
fn with_map<T>(f: impl FnOnce(&mut HashMap<String, Window>) -> T) -> T {
let mut guard = ATTEMPTS.lock().unwrap_or_else(|e| e.into_inner());
let map = guard.get_or_insert_with(HashMap::new);
f(map)
}
/// Reset attempt counters (between listens / tests).
pub fn reset() {
let mut guard = ATTEMPTS.lock().unwrap_or_else(|e| e.into_inner());
*guard = Some(HashMap::new());
}
/// Returns `Err(message)` when the client is temporarily locked out.
pub fn check(ip: &str, username: &str) -> Result<(), String> {
let k = key(ip, username);
let now = now_secs();
with_map(|map| {
if let Some(w) = map.get(&k) {
if now.saturating_sub(w.start) < WINDOW_SEC && w.count >= MAX_FAILS {
return Err(
"Too many failed login attempts. Try again in about 15 minutes.".into(),
);
}
if now.saturating_sub(w.start) >= WINDOW_SEC {
map.remove(&k);
}
}
Ok(())
})
}
pub fn record_failure(ip: &str, username: &str) {
let k = key(ip, username);
let now = now_secs();
with_map(|map| {
let entry = map.entry(k).or_insert(Window { count: 0, start: now });
if now.saturating_sub(entry.start) >= WINDOW_SEC {
entry.count = 0;
entry.start = now;
}
entry.count = entry.count.saturating_add(1);
});
}
pub fn clear_success(ip: &str, username: &str) {
let k = key(ip, username);
with_map(|map| {
map.remove(&k);
});
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn locks_after_max_fails() {
reset();
let ip = "127.0.0.1";
let user = "admin";
for _ in 0..MAX_FAILS {
record_failure(ip, user);
}
assert!(check(ip, user).is_err());
clear_success(ip, user);
assert!(check(ip, user).is_ok());
}
}
File diff suppressed because it is too large. Load diff
-101
View File
@@ -1,101 +0,0 @@
//! RSS 2.0 feed assembly (content-site W4).
use serde_json::Value;
fn esc(s: &str) -> String {
s.replace('&', "&amp;")
.replace('<', "&lt;")
.replace('>', "&gt;")
.replace('"', "&quot;")
}
fn cell_str(v: &Value) -> String {
match v {
Value::Null => String::new(),
Value::String(s) => s.clone(),
Value::Bool(b) => b.to_string(),
Value::Number(n) => n.to_string(),
other => other.to_string(),
}
}
fn pick<'a>(m: &'a serde_json::Map<String, Value>, keys: &[&str]) -> Option<&'a Value> {
keys.iter().find_map(|k| m.get(*k))
}
/// Build RSS 2.0 XML from channel metadata and row maps.
pub fn build_rss(
title: &str,
link: &str,
description: &str,
items: &[Value],
) -> String {
let mut body = format!(
r#"<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"><channel>
<title>{t}</title>
<link>{l}</link>
<description>{d}</description>
"#,
t = esc(title),
l = esc(link),
d = esc(description),
);
for row in items {
let Some(m) = row.as_object() else {
continue;
};
let item_title = pick(m, &["title", "标题"])
.map(cell_str)
.unwrap_or_default();
let item_link = pick(m, &["link", "href", "链接", "slug"])
.map(cell_str)
.unwrap_or_default();
let item_desc = pick(m, &["description", "summary", "摘要", "body", "正文"])
.map(cell_str)
.unwrap_or_default();
let pub_date = pick(m, &["pubDate", "published", "created_at", "发布"])
.map(cell_str)
.unwrap_or_default();
let full_link = if item_link.starts_with("http") {
item_link.clone()
} else if item_link.starts_with('/') {
format!("{link}{item_link}")
} else if !item_link.is_empty() {
format!("{link}/{item_link}")
} else {
link.to_string()
};
body.push_str("<item>");
body.push_str(&format!("<title>{}</title>", esc(&item_title)));
body.push_str(&format!("<link>{}</link>", esc(&full_link)));
body.push_str(&format!(
"<description>{}</description>",
esc(&item_desc)
));
if !pub_date.is_empty() {
body.push_str(&format!("<pubDate>{}</pubDate>", esc(&pub_date)));
}
body.push_str("</item>");
}
body.push_str("</channel></rss>");
body
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn rss_contains_item() {
let xml = build_rss(
"Blog",
"http://example.com",
"Feed",
&[json!({"title": "Hi", "slug": "/post/a", "summary": "text"})],
);
assert!(xml.contains("<rss"));
assert!(xml.contains("<title>Hi</title>"));
}
}
-643
View File
@@ -1,643 +0,0 @@
//! Session store + cookie helpers + CSRF tokens (ext-web-net W3).
//!
//! - **CSPRNG** session ids (`getrandom`).
//! - **SQLite persistence** when `configure` receives a `db_url`; otherwise in-memory (offline gold).
//! - **Sliding expiry** on read/write.
//! - **CSRF** token per session (`_csrf` key).
use std::collections::HashMap;
use std::sync::Mutex;
use std::time::{SystemTime, UNIX_EPOCH};
use getrandom::getrandom;
use serde_json::{json, Map, Value};
use crate::db;
use crate::password;
type SessionMap = HashMap<String, Value>;
#[derive(Clone, Default)]
pub struct Config {
pub db_url: Option<String>,
pub ttl_sec: u64,
pub cookie_secure: bool,
}
static CONFIG: Mutex<Option<Config>> = Mutex::new(None);
struct MemStore {
ttl_sec: u64,
data: HashMap<String, (u64, SessionMap)>,
}
static MEM: Mutex<Option<MemStore>> = Mutex::new(None);
const SESSION_TABLE: &str = "_marqdo_sessions";
const CSRF_KEY: &str = "_csrf";
fn now_secs() -> u64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0)
}
fn cfg() -> Config {
CONFIG
.lock()
.unwrap_or_else(|e| e.into_inner())
.clone()
.unwrap_or(Config {
db_url: None,
ttl_sec: 3600,
cookie_secure: false,
})
}
/// Configure session backend before `listen` or offline ABI calls.
pub fn configure(config: Config) {
let ttl = if config.ttl_sec > 0 {
config.ttl_sec
} else {
3600
};
let mut c = config;
c.ttl_sec = ttl;
if let Some(ref url) = c.db_url {
let _ = ensure_session_table(url);
}
let mut guard = CONFIG.lock().unwrap_or_else(|e| e.into_inner());
*guard = Some(c);
}
/// Reset in-memory sessions (tests / each listen).
pub fn reset(ttl_sec: u64) {
let ttl = if ttl_sec > 0 { ttl_sec } else { 3600 };
{
let mut guard = CONFIG.lock().unwrap_or_else(|e| e.into_inner());
if let Some(ref mut c) = *guard {
c.ttl_sec = ttl;
} else {
*guard = Some(Config {
db_url: None,
ttl_sec: ttl,
cookie_secure: false,
});
}
}
let mut mem = MEM.lock().unwrap_or_else(|e| e.into_inner());
*mem = Some(MemStore {
ttl_sec: ttl,
data: HashMap::new(),
});
}
fn ensure_session_table(url: &str) -> Result<(), String> {
db::exec(
url,
&format!(
"CREATE TABLE IF NOT EXISTS \"{SESSION_TABLE}\" (
id TEXT PRIMARY KEY,
expires_at INTEGER NOT NULL,
data TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_marqdo_sessions_exp ON \"{SESSION_TABLE}\"(expires_at);"
),
None,
None,
)?;
Ok(())
}
fn prune_mem(store: &mut MemStore) {
let now = now_secs();
store.data.retain(|_, (exp, _)| *exp > now);
}
fn prune_sql(url: &str) {
let now = now_secs() as i64;
let _ = db::exec(
url,
&format!("DELETE FROM \"{SESSION_TABLE}\" WHERE expires_at <= ?1"),
Some(&json!([now])),
None,
);
}
fn secure_random_id() -> Result<String, String> {
let mut bytes = [0u8; 32];
getrandom(&mut bytes).map_err(|e| format!("getrandom: {e}"))?;
Ok(bytes.iter().map(|b| format!("{b:02x}")).collect())
}
fn random_csrf() -> Result<String, String> {
let mut bytes = [0u8; 24];
getrandom(&mut bytes).map_err(|e| format!("getrandom: {e}"))?;
Ok(bytes.iter().map(|b| format!("{b:02x}")).collect())
}
fn map_to_json(m: &SessionMap) -> Result<String, String> {
let obj: Map<String, Value> = m.iter().map(|(k, v)| (k.clone(), v.clone())).collect();
serde_json::to_string(&Value::Object(obj)).map_err(|e| e.to_string())
}
fn json_to_map(s: &str) -> SessionMap {
match serde_json::from_str::<Value>(s) {
Ok(Value::Object(m)) => m.into_iter().collect(),
_ => HashMap::new(),
}
}
fn load_sql(url: &str, id: &str) -> Option<(u64, SessionMap)> {
let q = db::query(
url,
&format!("SELECT expires_at, data FROM \"{SESSION_TABLE}\" WHERE id = ?1"),
Some(&json!([id])),
None,
)
.ok()?;
let rows = q.get("rows")?.as_array()?;
let row = rows.first()?;
let exp = row.get("expires_at")?.as_i64()? as u64;
let data_s = row.get("data")?.as_str()?;
Some((exp, json_to_map(data_s)))
}
fn save_sql(url: &str, id: &str, exp: u64, data: &SessionMap) -> Result<(), String> {
let data_s = map_to_json(data)?;
db::exec(
url,
&format!(
"INSERT INTO \"{SESSION_TABLE}\" (id, expires_at, data) VALUES (?1, ?2, ?3)
ON CONFLICT(id) DO UPDATE SET expires_at = excluded.expires_at, data = excluded.data"
),
Some(&json!([id, exp as i64, data_s])),
None,
)?;
Ok(())
}
fn delete_sql(url: &str, id: &str) {
let _ = db::exec(
url,
&format!("DELETE FROM \"{SESSION_TABLE}\" WHERE id = ?1"),
Some(&json!([id])),
None,
);
}
fn touch_expiry(ttl: u64) -> u64 {
now_secs() + ttl
}
fn ensure_csrf(data: &mut SessionMap) -> Result<String, String> {
if let Some(Value::String(t)) = data.get(CSRF_KEY) {
if !t.is_empty() {
return Ok(t.clone());
}
}
let t = random_csrf()?;
data.insert(CSRF_KEY.to_string(), json!(t));
Ok(t)
}
pub fn session_new(ttl_sec: Option<u64>) -> String {
let cfg = cfg();
let ttl = ttl_sec.unwrap_or(cfg.ttl_sec);
if ttl > 0 {
if let Ok(mut guard) = CONFIG.lock() {
if let Some(ref mut c) = *guard {
c.ttl_sec = ttl;
}
}
}
let id = secure_random_id().unwrap_or_else(|_| format!("fallback-{}", now_secs()));
let exp = touch_expiry(ttl);
let mut data = SessionMap::new();
let _ = ensure_csrf(&mut data);
if let Some(ref url) = cfg.db_url {
prune_sql(url);
let _ = save_sql(url, &id, exp, &data);
} else {
with_mem(|store| {
store.ttl_sec = ttl;
prune_mem(store);
store.data.insert(id.clone(), (exp, data));
});
}
id
}
fn with_mem<T>(f: impl FnOnce(&mut MemStore) -> T) -> T {
let mut guard = MEM.lock().unwrap_or_else(|e| e.into_inner());
let store = guard.get_or_insert_with(|| MemStore {
ttl_sec: cfg().ttl_sec,
data: HashMap::new(),
});
f(store)
}
fn get_record(id: &str) -> Option<(u64, SessionMap)> {
let cfg = cfg();
let now = now_secs();
if let Some(ref url) = cfg.db_url {
prune_sql(url);
let (exp, data) = load_sql(url, id)?;
if exp <= now {
delete_sql(url, id);
return None;
}
Some((exp, data))
} else {
with_mem(|store| {
prune_mem(store);
let (exp, data) = store.data.get(id)?.clone();
if exp <= now {
store.data.remove(id);
return None;
}
Some((exp, data))
})
}
}
fn put_record(id: &str, exp: u64, data: SessionMap) -> bool {
let cfg = cfg();
if let Some(ref url) = cfg.db_url {
save_sql(url, id, exp, &data).is_ok()
} else {
with_mem(|store| {
store.data.insert(id.to_string(), (exp, data));
true
})
}
}
pub fn session_set(id: &str, key: &str, value: Value) -> bool {
let cfg = cfg();
let Some((_, mut data)) = get_record(id) else {
return false;
};
data.insert(key.to_string(), value);
put_record(id, touch_expiry(cfg.ttl_sec), data)
}
pub fn session_get(id: &str, key: &str) -> Option<Value> {
let cfg = cfg();
let (_, data) = get_record(id)?;
let v = data.get(key).cloned()?;
put_record(id, touch_expiry(cfg.ttl_sec), data);
Some(v)
}
pub fn session_del(id: &str, key: &str) -> bool {
let cfg = cfg();
let Some((_, mut data)) = get_record(id) else {
return false;
};
let ok = data.remove(key).is_some();
if ok {
put_record(id, touch_expiry(cfg.ttl_sec), data);
}
ok
}
pub fn session_destroy(id: &str) -> bool {
let cfg = cfg();
if let Some(ref url) = cfg.db_url {
delete_sql(url, id);
true
} else {
with_mem(|store| store.data.remove(id).is_some())
}
}
/// CSRF token for an existing session (creates one if missing).
pub fn csrf_for(id: &str) -> Option<String> {
let cfg = cfg();
let (_, mut data) = get_record(id)?;
let token = ensure_csrf(&mut data).ok()?;
put_record(id, touch_expiry(cfg.ttl_sec), data);
Some(token)
}
pub fn validate_csrf(id: &str, token: &str) -> bool {
if token.is_empty() {
return false;
}
match session_get(id, CSRF_KEY) {
Some(Value::String(stored)) => stored == token,
_ => false,
}
}
/// Resolve session from cookie: reuse valid id or create a new one.
/// Returns `(session_id, set_cookie_header)`.
pub fn ensure_from_cookie(cookie_header: Option<&str>) -> (String, Option<String>) {
if let Some(id) = session_id_from_cookie(cookie_header) {
if get_record(&id).is_some() {
let _ = csrf_for(&id);
return (id, None);
}
}
let id = session_new(None);
let cfg = cfg();
(
id.clone(),
Some(session_cookie(&id, cfg.ttl_sec, cfg.cookie_secure)),
)
}
/// Parse the session id from a `Cookie` request header (looks for `marqdo_sid=…`).
pub fn session_id_from_cookie(cookie_header: Option<&str>) -> Option<String> {
let header = cookie_header?;
for part in header.split(';') {
let part = part.trim();
if let Some(v) = part.strip_prefix("marqdo_sid=") {
let v = v.trim_matches('"');
if !v.is_empty() {
return Some(v.to_string());
}
}
}
None
}
/// `Set-Cookie: marqdo_sid=<id>; HttpOnly; Path=/; SameSite=Lax; Max-Age=<ttl>[; Secure]`
pub fn session_cookie(id: &str, ttl_sec: u64, secure: bool) -> String {
let mut s = format!(
"marqdo_sid={id}; HttpOnly; Path=/; SameSite=Lax; Max-Age={ttl_sec}"
);
if secure {
s.push_str("; Secure");
}
s
}
/// Validate credentials; supports argon2 hashes or legacy plaintext in the password column.
/// Returns `(username, role)`. Missing role column defaults to `admin` (legacy admin tables).
pub fn check_credentials(users: &Value, username: &str, password: &str) -> Option<(String, String)> {
let rows = crate::table::as_rows(users);
let rows = match rows {
Value::Array(a) => a,
_ => return None,
};
for row in &rows {
let m = match row {
Value::Object(m) => m,
_ => continue,
};
let u = pick_cell(m, &["用户名", "用户", "username", "user", "账号"])
.map(cell_str)
.unwrap_or_default();
let p = pick_cell(m, &["密码", "password", "pass", "口令"])
.map(cell_str)
.unwrap_or_default();
if u == username && password::verify_password(password, &p) {
let role = pick_cell(m, &["角色", "role", "roles"])
.map(cell_str)
.filter(|s| !s.is_empty())
.unwrap_or_else(|| "admin".into());
return Some((u, role.to_ascii_lowercase()));
}
}
None
}
/// Session role (defaults to `visitor` when anonymous / missing).
pub fn session_role(cookie_header: Option<&str>) -> String {
let Some(sid) = session_id_from_cookie(cookie_header) else {
return "visitor".into();
};
match session_get(&sid, "role") {
Some(Value::String(r)) if !r.is_empty() => r.to_ascii_lowercase(),
_ => {
if session_get(&sid, "username").is_some() {
"admin".into()
} else {
"visitor".into()
}
}
}
}
pub fn role_allowed(role: &str, allowed: &[String]) -> bool {
if allowed.is_empty() {
return true;
}
let role = role.to_ascii_lowercase();
allowed.iter().any(|a| a.eq_ignore_ascii_case(&role))
}
pub fn parse_roles_csv(raw: &str) -> Vec<String> {
raw.split(',')
.map(|s| s.trim().to_ascii_lowercase())
.filter(|s| !s.is_empty())
.collect()
}
fn pick_cell<'a>(m: &'a Map<String, Value>, keys: &[&str]) -> Option<&'a Value> {
keys.iter().find_map(|k| m.get(*k))
}
fn cell_str(v: &Value) -> String {
match v {
Value::Null => String::new(),
Value::String(s) => s.clone(),
Value::Bool(b) => b.to_string(),
Value::Number(n) => n.to_string(),
other => other.to_string(),
}
}
pub fn abi_session_new(args: &Value) -> Result<Value, String> {
let ttl = args
.get("ttl_sec")
.and_then(|v| v.as_u64().or_else(|| v.as_i64().map(|i| i as u64)));
let id = session_new(ttl);
Ok(json!({ "id": id }))
}
pub fn abi_session_set(args: &Value) -> Result<Value, String> {
let id = arg_str(args, "id")?;
let key = arg_str(args, "key")?;
let value = args.get("value").cloned().unwrap_or(Value::Null);
Ok(json!({ "ok": session_set(id, key, value) }))
}
pub fn abi_session_get(args: &Value) -> Result<Value, String> {
let id = arg_str(args, "id")?;
let key = arg_str(args, "key")?;
match session_get(id, key) {
Some(v) => Ok(json!({ "ok": true, "value": v })),
None => Ok(json!({ "ok": false })),
}
}
pub fn abi_session_del(args: &Value) -> Result<Value, String> {
let id = arg_str(args, "id")?;
let key = arg_str(args, "key")?;
Ok(json!({ "ok": session_del(id, key) }))
}
pub fn abi_session_destroy(args: &Value) -> Result<Value, String> {
let id = arg_str(args, "id")?;
Ok(json!({ "ok": session_destroy(id) }))
}
pub fn abi_auth_login(args: &Value) -> Result<Value, String> {
let username = arg_str(args, "username")?;
let password = arg_str(args, "password")?;
let users = args.get("users").cloned().unwrap_or(Value::Null);
let ttl = args
.get("session_ttl")
.and_then(|v| v.as_u64().or_else(|| v.as_i64().map(|i| i as u64)));
match check_credentials(&users, username, password) {
Some((u, role)) => {
let id = session_new(ttl);
session_set(&id, "username", json!(u));
session_set(&id, "role", json!(role));
Ok(json!({ "ok": true, "session_id": id, "username": u, "role": role }))
}
None => Ok(json!({ "ok": false })),
}
}
pub fn abi_auth_check(args: &Value) -> Result<Value, String> {
let id = arg_str(args, "session_id")?;
match session_get(id, "username") {
Some(Value::String(u)) => {
let role = match session_get(id, "role") {
Some(Value::String(r)) => r,
_ => "admin".into(),
};
Ok(json!({ "ok": true, "username": u, "role": role }))
}
_ => Ok(json!({ "ok": false })),
}
}
pub fn abi_auth_logout(args: &Value) -> Result<Value, String> {
let id = arg_str(args, "session_id")?;
Ok(json!({ "ok": session_destroy(id) }))
}
pub fn abi_password_hash(args: &Value) -> Result<Value, String> {
let plain = arg_str(args, "password")?;
let hash = password::hash_password(plain)?;
Ok(json!({ "hash": hash }))
}
fn arg_str<'a>(v: &'a Value, key: &str) -> Result<&'a str, String> {
v.get(key)
.and_then(|x| x.as_str())
.ok_or_else(|| format!("missing `{key}`"))
}
#[cfg(test)]
mod tests {
use super::*;
fn users() -> Value {
json!([
{ "username": "admin", "password": "secret" },
{ "用户": "站长", "密码": "pw123" }
])
}
#[test]
fn session_set_get_del() {
reset(3600);
let id = session_new(None);
assert!(!id.is_empty());
assert!(session_set(&id, "username", json!("admin")));
assert_eq!(session_get(&id, "username"), Some(json!("admin")));
assert!(session_del(&id, "username"));
assert_eq!(session_get(&id, "username"), None);
assert!(session_destroy(&id));
}
#[test]
fn auth_login_check_logout() {
reset(3600);
let r = abi_auth_login(&json!({
"username": "admin",
"password": "secret",
"users": users(),
}))
.unwrap();
assert_eq!(r["ok"].as_bool(), Some(true));
let sid = r["session_id"].as_str().unwrap().to_string();
let check = abi_auth_check(&json!({ "session_id": sid })).unwrap();
assert_eq!(check["ok"].as_bool(), Some(true));
let bad = abi_auth_login(&json!({
"username": "admin",
"password": "wrong",
"users": users(),
}))
.unwrap();
assert_eq!(bad["ok"].as_bool(), Some(false));
let out = abi_auth_logout(&json!({ "session_id": sid })).unwrap();
assert_eq!(out["ok"].as_bool(), Some(true));
}
#[test]
fn csrf_roundtrip() {
reset(3600);
let id = session_new(None);
let t = csrf_for(&id).unwrap();
assert!(validate_csrf(&id, &t));
assert!(!validate_csrf(&id, "bad"));
}
#[test]
fn cookie_roundtrip() {
reset(3600);
let id = session_new(None);
let cookie = session_cookie(&id, 3600, false);
assert!(cookie.starts_with("marqdo_sid="));
let got = session_id_from_cookie(Some(&format!("theme=light; {cookie}")));
assert_eq!(got.as_deref(), Some(id.as_str()));
}
#[test]
fn hashed_password_login() {
reset(3600);
let hash = password::hash_password("secret").unwrap();
let users = json!([{ "username": "admin", "password": hash }]);
let r = abi_auth_login(&json!({
"username": "admin",
"password": "secret",
"users": users,
}))
.unwrap();
assert_eq!(r["ok"].as_bool(), Some(true));
}
#[test]
fn session_persists_in_sqlite() {
let dir = std::env::temp_dir().join(format!("marqdo_web_sess_{}", std::process::id()));
let _ = std::fs::create_dir_all(&dir);
let path = dir.join("persist.db");
let _ = std::fs::remove_file(&path);
let url = format!("sqlite:{}", path.display());
configure(Config {
db_url: Some(url.clone()),
ttl_sec: 3600,
cookie_secure: false,
});
let id = session_new(None);
assert!(session_set(&id, "theme", json!("dark")));
// Re-configure (simulates listen restart with same db).
configure(Config {
db_url: Some(url),
ttl_sec: 3600,
cookie_secure: false,
});
assert_eq!(session_get(&id, "theme"), Some(json!("dark")));
let _ = std::fs::remove_dir_all(dir);
}
}
-152
View File
@@ -1,152 +0,0 @@
//! sitemap.xml / robots.txt helpers (W7).
use serde_json::{json, Value};
fn cell_str(v: &Value) -> String {
match v {
Value::Null => String::new(),
Value::String(s) => s.clone(),
Value::Bool(b) => b.to_string(),
Value::Number(n) => n.to_string(),
other => other.to_string(),
}
}
fn xml_escape(s: &str) -> String {
s.replace('&', "&amp;")
.replace('<', "&lt;")
.replace('>', "&gt;")
.replace('"', "&quot;")
.replace('\'', "&apos;")
}
/// Build a sitemap.xml document from rows.
///
/// Each row may use keys: `loc`/`url`/`路径`, optional `lastmod`/`更新`,
/// `changefreq`/`频率`, `priority`/`优先级`.
pub fn build_sitemap(base: &str, items: &Value) -> String {
let base = base.trim_end_matches('/');
let rows: Vec<Value> = match items {
Value::Array(a) => a
.iter()
.map(|row| match row {
Value::String(s) => json!({ "loc": s }),
Value::Object(_) => row.clone(),
other => json!({ "loc": cell_str(other) }),
})
.collect(),
Value::Object(m) => {
// Columnar GFM: { loc: [...], ... }
let locs = m
.get("loc")
.or_else(|| m.get("路径"))
.or_else(|| m.get("path"))
.or_else(|| m.get("url"));
match locs {
Some(Value::Array(a)) => a
.iter()
.enumerate()
.map(|(i, loc)| {
let mut obj = serde_json::Map::new();
obj.insert("loc".into(), loc.clone());
for (k, v) in m {
if k == "loc" || k == "路径" || k == "path" || k == "url" {
continue;
}
if let Some(Value::Array(col)) = Some(v) {
if let Some(cell) = col.get(i) {
obj.insert(k.clone(), cell.clone());
}
}
}
Value::Object(obj)
})
.collect(),
_ => Vec::new(),
}
}
_ => Vec::new(),
};
let mut out = String::from(
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n\
<urlset xmlns=\"http://www.sitemaps.org/schemas/sitemap/0.9\">\n",
);
for row in &rows {
let Some(obj) = row.as_object() else {
continue;
};
let loc_raw = obj
.get("loc")
.or_else(|| obj.get("url"))
.or_else(|| obj.get("路径"))
.or_else(|| obj.get("path"))
.map(cell_str)
.unwrap_or_default();
if loc_raw.is_empty() {
continue;
}
let loc = if loc_raw.starts_with("http://") || loc_raw.starts_with("https://") {
loc_raw
} else if loc_raw.starts_with('/') {
format!("{base}{loc_raw}")
} else {
format!("{base}/{loc_raw}")
};
out.push_str(" <url>\n");
out.push_str(&format!(" <loc>{}</loc>\n", xml_escape(&loc)));
if let Some(lm) = obj
.get("lastmod")
.or_else(|| obj.get("更新"))
.map(cell_str)
.filter(|s| !s.is_empty())
{
out.push_str(&format!(" <lastmod>{}</lastmod>\n", xml_escape(&lm)));
}
if let Some(cf) = obj
.get("changefreq")
.or_else(|| obj.get("频率"))
.map(cell_str)
.filter(|s| !s.is_empty())
{
out.push_str(&format!(
" <changefreq>{}</changefreq>\n",
xml_escape(&cf)
));
}
if let Some(pr) = obj
.get("priority")
.or_else(|| obj.get("优先级"))
.map(cell_str)
.filter(|s| !s.is_empty())
{
out.push_str(&format!(" <priority>{}</priority>\n", xml_escape(&pr)));
}
out.push_str(" </url>\n");
}
out.push_str("</urlset>\n");
out
}
/// Default robots.txt body allowing all and pointing at a sitemap URL.
pub fn build_robots(sitemap_url: Option<&str>) -> String {
let mut out = String::from("User-agent: *\nAllow: /\n");
if let Some(u) = sitemap_url.filter(|s| !s.is_empty()) {
out.push_str(&format!("Sitemap: {u}\n"));
}
out
}
/// ABI helper returning `{ xml }`.
pub fn sitemap_json(base: &str, items: &Value) -> Value {
json!({ "xml": build_sitemap(base, items) })
}
/// ABI helper returning `{ body }`.
#[allow(dead_code)]
pub fn robots_json(sitemap_url: Option<&str>, body: Option<&str>) -> Value {
let text = match body.filter(|s| !s.is_empty()) {
Some(b) => b.to_string(),
None => build_robots(sitemap_url),
};
json!({ "body": text })
}
-661
View File
@@ -1,661 +0,0 @@
//! `# storage` — local `file:` blobs; `s3://` reserved for live S3/MinIO (SigV4 later).
use serde_json::{json, Value};
use std::fs;
use std::path::{Path, PathBuf};
fn is_file(url: &str) -> bool {
let u = url.trim().to_ascii_lowercase();
u.starts_with("file:") || u.starts_with("file://")
}
fn is_s3(url: &str) -> bool {
url.trim().to_ascii_lowercase().starts_with("s3://")
}
fn file_root(url: &str) -> Result<PathBuf, String> {
let raw = url
.trim()
.strip_prefix("file://")
.or_else(|| url.trim().strip_prefix("file:"))
.unwrap_or(url.trim());
let p = PathBuf::from(raw);
if !p.as_os_str().is_empty() {
fs::create_dir_all(&p).map_err(|e| format!("storage mkdir: {e}"))?;
}
Ok(p)
}
fn safe_key(key: &str) -> Result<String, String> {
let k = key.trim().trim_start_matches('/');
if k.is_empty() {
return Err("storage key is empty".into());
}
if k.contains("..") {
return Err("storage key must not contain `..`".into());
}
Ok(k.replace('\\', "/"))
}
fn path_for(root: &Path, key: &str) -> Result<PathBuf, String> {
let k = safe_key(key)?;
Ok(root.join(k))
}
/// Open storage. For `file:` ensures the directory exists.
pub fn open(url: &str) -> Result<Value, String> {
let url = url.trim();
if is_file(url) {
let _ = file_root(url)?;
Ok(json!({ "_type": "storage", "url": url, "backend": "file" }))
} else if is_s3(url) {
// Validate shape early so author typos fail at open, not at put.
let rest = url
.strip_prefix("s3://")
.ok_or_else(|| "bad s3 url".to_string())?;
let bucket = rest.split('?').next().unwrap_or("");
if bucket.is_empty() {
return Err("s3 url missing bucket (s3://bucket?endpoint=…)".into());
}
Ok(json!({ "_type": "storage", "url": url, "backend": "s3" }))
} else {
Err(format!(
"storage url must be `file:…` or `s3://bucket?…`, got `{url}`"
))
}
}
fn load_bytes(body: Option<&str>, path: Option<&str>) -> Result<Vec<u8>, String> {
if let Some(p) = path.filter(|s| !s.is_empty()) {
return fs::read(p).map_err(|e| format!("read path `{p}`: {e}"));
}
if let Some(b) = body {
return Ok(b.as_bytes().to_vec());
}
Err("storage.put needs `body` or `path`".into())
}
pub fn put(
url: &str,
key: &str,
body: Option<&str>,
path: Option<&str>,
content_type: Option<&str>,
) -> Result<Value, String> {
let bytes = load_bytes(body, path)?;
let ct = content_type.unwrap_or("application/octet-stream");
put_bytes(url, key, &bytes, ct)
}
/// Persist raw bytes (multipart upload / binary save).
pub fn put_bytes(
url: &str,
key: &str,
bytes: &[u8],
content_type: &str,
) -> Result<Value, String> {
let ct = if content_type.trim().is_empty() {
"application/octet-stream"
} else {
content_type.trim()
};
if is_file(url) {
let root = file_root(url)?;
let dest = path_for(&root, key)?;
if let Some(parent) = dest.parent() {
fs::create_dir_all(parent).map_err(|e| e.to_string())?;
}
fs::write(&dest, bytes).map_err(|e| e.to_string())?;
let _ = fs::write(ctype_path(&dest), ct);
Ok(json!({ "ok": true, "key": key, "size": bytes.len(), "content_type": ct }))
} else if is_s3(url) {
s3_put_http(url, key, bytes, ct)
} else {
Err("unknown storage backend".into())
}
}
/// Read object bytes for HTTP download. Returns `(bytes, content_type, filename)`.
pub fn read_bytes(url: &str, key: &str) -> Result<Option<(Vec<u8>, String, String)>, String> {
let filename = key
.rsplit('/')
.next()
.filter(|s| !s.is_empty())
.unwrap_or("download")
.to_string();
if is_file(url) {
let root = file_root(url)?;
let dest = path_for(&root, key)?;
if !dest.is_file() {
return Ok(None);
}
let bytes = fs::read(&dest).map_err(|e| e.to_string())?;
let ct = fs::read_to_string(ctype_path(&dest))
.unwrap_or_else(|_| "application/octet-stream".into());
Ok(Some((bytes, ct, filename)))
} else if is_s3(url) {
let cfg = S3Conf::parse(url)?;
let target = cfg.object_url(key)?;
let (status, ct, body) = cfg.request_bytes("GET", &target)?;
if status == 404 {
return Ok(None);
}
if !(200..300).contains(&status) {
return Err(format!("s3 get HTTP {status}"));
}
let content_type = if ct.trim().is_empty() {
"application/octet-stream".into()
} else {
ct
};
Ok(Some((body, content_type, filename)))
} else {
Err("unknown storage backend".into())
}
}
fn ctype_path(dest: &Path) -> PathBuf {
PathBuf::from(format!("{}.ctype", dest.display()))
}
pub fn get(url: &str, key: &str) -> Result<Value, String> {
if is_file(url) {
let root = file_root(url)?;
let dest = path_for(&root, key)?;
if !dest.is_file() {
return Ok(json!({ "ok": false }));
}
let bytes = fs::read(&dest).map_err(|e| e.to_string())?;
let ct = fs::read_to_string(ctype_path(&dest))
.unwrap_or_else(|_| "application/octet-stream".into());
match String::from_utf8(bytes.clone()) {
Ok(s) => Ok(json!({
"ok": true,
"key": key,
"body": s,
"content_type": ct,
"size": s.len(),
})),
Err(_) => Ok(json!({
"ok": true,
"key": key,
"path": dest.to_string_lossy(),
"content_type": ct,
"size": bytes.len(),
"binary": true,
})),
}
} else if is_s3(url) {
s3_get_http(url, key)
} else {
Err("unknown storage backend".into())
}
}
pub fn delete(url: &str, key: &str) -> Result<Value, String> {
if is_file(url) {
let root = file_root(url)?;
let dest = path_for(&root, key)?;
let ok = if dest.is_file() {
fs::remove_file(&dest).map_err(|e| e.to_string())?;
let _ = fs::remove_file(ctype_path(&dest));
true
} else {
false
};
Ok(json!({ "ok": ok }))
} else if is_s3(url) {
s3_delete_http(url, key)
} else {
Err("unknown storage backend".into())
}
}
pub fn list(url: &str, prefix: Option<&str>) -> Result<Value, String> {
let prefix = prefix.unwrap_or("").trim_start_matches('/');
if is_file(url) {
let root = file_root(url)?;
let mut keys = Vec::new();
walk_keys(&root, &root, prefix, &mut keys)?;
keys.sort();
Ok(json!({ "ok": true, "keys": keys, "count": keys.len() }))
} else if is_s3(url) {
s3_list_http(url, prefix)
} else {
Err("unknown storage backend".into())
}
}
fn walk_keys(root: &Path, dir: &Path, prefix: &str, out: &mut Vec<String>) -> Result<(), String> {
let entries = match fs::read_dir(dir) {
Ok(e) => e,
Err(_) => return Ok(()),
};
for ent in entries {
let ent = ent.map_err(|e| e.to_string())?;
let path = ent.path();
if path
.file_name()
.and_then(|n| n.to_str())
.is_some_and(|n| n.ends_with(".ctype"))
{
continue;
}
if path.is_dir() {
walk_keys(root, &path, prefix, out)?;
} else if path.is_file() {
let rel = path
.strip_prefix(root)
.map_err(|e| e.to_string())?
.to_string_lossy()
.replace('\\', "/");
if prefix.is_empty() || rel.starts_with(prefix) {
out.push(rel);
}
}
}
Ok(())
}
/// Live S3/MinIO via AWS SigV4. Requires `endpoint` query for path-style (MinIO).
fn s3_put_http(url: &str, key: &str, body: &[u8], content_type: &str) -> Result<Value, String> {
let cfg = S3Conf::parse(url)?;
let target = cfg.object_url(key)?;
let status = cfg.request("PUT", &target, body, content_type)?;
if (200..300).contains(&status) {
Ok(json!({ "ok": true, "key": key, "size": body.len() }))
} else {
Err(format!("s3 put HTTP {status}"))
}
}
fn s3_get_http(url: &str, key: &str) -> Result<Value, String> {
let cfg = S3Conf::parse(url)?;
let target = cfg.object_url(key)?;
let (status, ct, bytes) = cfg.request_bytes("GET", &target)?;
if status == 404 {
return Ok(json!({ "ok": false }));
}
if !(200..300).contains(&status) {
return Err(format!("s3 get HTTP {status}"));
}
match String::from_utf8(bytes.clone()) {
Ok(s) => Ok(json!({
"ok": true, "key": key, "body": s, "content_type": ct, "size": s.len()
})),
Err(_) => Ok(json!({
"ok": true, "key": key, "content_type": ct, "size": bytes.len(), "binary": true
})),
}
}
fn s3_delete_http(url: &str, key: &str) -> Result<Value, String> {
let cfg = S3Conf::parse(url)?;
let target = cfg.object_url(key)?;
let status = cfg.request("DELETE", &target, &[], "")?;
if (200..300).contains(&status) || status == 204 || status == 404 {
Ok(json!({ "ok": status != 404 }))
} else {
Err(format!("s3 delete HTTP {status}"))
}
}
fn s3_list_http(url: &str, prefix: &str) -> Result<Value, String> {
let cfg = S3Conf::parse(url)?;
let list_url = cfg.list_url(prefix);
let (status, _ct, bytes) = cfg.request_bytes("GET", &list_url)?;
let text = String::from_utf8_lossy(&bytes).into_owned();
if !(200..300).contains(&status) {
return Err(format!("s3 list HTTP {status}: {text}"));
}
let mut keys = Vec::new();
for part in text.split("<Key>").skip(1) {
if let Some(end) = part.find("</Key>") {
keys.push(part[..end].to_string());
}
}
Ok(json!({ "ok": true, "keys": keys, "count": keys.len() }))
}
struct S3Conf {
bucket: String,
endpoint: String,
region: String,
access_key: String,
secret_key: String,
path_style: bool,
}
impl S3Conf {
fn parse(url: &str) -> Result<Self, String> {
let rest = url
.trim()
.strip_prefix("s3://")
.ok_or_else(|| "not an s3 url".to_string())?;
let (bucket, query) = match rest.split_once('?') {
Some((b, q)) => (b.to_string(), q),
None => (rest.to_string(), ""),
};
if bucket.is_empty() {
return Err("s3 url missing bucket".into());
}
let mut endpoint = String::new();
let mut region = "us-east-1".to_string();
let mut access_key = std::env::var("AWS_ACCESS_KEY_ID")
.or_else(|_| std::env::var("S3_ACCESS_KEY"))
.unwrap_or_default();
let mut secret_key = std::env::var("AWS_SECRET_ACCESS_KEY")
.or_else(|_| std::env::var("S3_SECRET_KEY"))
.unwrap_or_default();
let mut path_style = true;
for pair in query.split('&').filter(|p| !p.is_empty()) {
let (k, v) = pair.split_once('=').unwrap_or((pair, ""));
let v = pct_decode(v);
match k {
"endpoint" => endpoint = v,
"region" => region = v,
"access_key" | "accessKey" => access_key = v,
"secret_key" | "secretKey" => secret_key = v,
"path_style" => {
path_style = matches!(v.as_str(), "1" | "true" | "True" | "yes")
}
_ => {}
}
}
if endpoint.is_empty() {
return Err(
"s3 url needs `endpoint=` for MinIO/path-style (e.g. s3://bucket?endpoint=http://127.0.0.1:9000)"
.into(),
);
}
if access_key.is_empty() || secret_key.is_empty() {
return Err(
"s3 credentials missing (access_key/secret_key query or AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY)"
.into(),
);
}
Ok(Self {
bucket,
endpoint: endpoint.trim_end_matches('/').to_string(),
region,
access_key,
secret_key,
path_style,
})
}
fn object_url(&self, key: &str) -> Result<String, String> {
let k = safe_key(key)?;
if self.path_style {
Ok(format!("{}/{}/{}", self.endpoint, self.bucket, k))
} else {
Err("virtual-hosted s3 not implemented in this wave — set path_style=true".into())
}
}
fn list_url(&self, prefix: &str) -> String {
format!(
"{}/{}?list-type=2&prefix={}",
self.endpoint,
self.bucket,
uri_encode(prefix)
)
}
fn request(
&self,
method: &str,
url: &str,
body: &[u8],
content_type: &str,
) -> Result<u16, String> {
let (status, _, _) = self.request_inner(method, url, body, content_type)?;
Ok(status)
}
fn request_bytes(&self, method: &str, url: &str) -> Result<(u16, String, Vec<u8>), String> {
self.request_inner(method, url, &[], "")
}
fn request_inner(
&self,
method: &str,
url: &str,
body: &[u8],
content_type: &str,
) -> Result<(u16, String, Vec<u8>), String> {
use hmac::{Hmac, Mac};
use sha2::{Digest, Sha256};
type HmacSha256 = Hmac<Sha256>;
let (scheme_host, path_query) = split_url(url)?;
let (path, query) = match path_query.split_once('?') {
Some((p, q)) => (p, q),
None => (path_query.as_str(), ""),
};
let host = scheme_host
.split("://")
.nth(1)
.ok_or_else(|| "bad url host".to_string())?;
let amz_date = amz_timestamp();
let date_stamp = &amz_date[..8];
let payload_hash = hex::encode(Sha256::digest(body));
let mut hdrs: Vec<(String, String)> = vec![
("host".into(), host.into()),
("x-amz-content-sha256".into(), payload_hash.clone()),
("x-amz-date".into(), amz_date.clone()),
];
if method == "PUT" && !content_type.is_empty() {
hdrs.push(("content-type".into(), content_type.into()));
}
hdrs.sort_by(|a, b| a.0.cmp(&b.0));
let signed_headers = hdrs.iter().map(|(k, _)| k.as_str()).collect::<Vec<_>>().join(";");
let canonical_headers = hdrs
.iter()
.map(|(k, v)| format!("{k}:{}\n", v.trim()))
.collect::<String>();
let canonical_query = canonical_query_string(query);
let canonical_request = format!(
"{method}\n{path}\n{canonical_query}\n{canonical_headers}\n{signed_headers}\n{payload_hash}"
);
let scope = format!("{date_stamp}/{}/s3/aws4_request", self.region);
let string_to_sign = format!(
"AWS4-HMAC-SHA256\n{amz_date}\n{scope}\n{}",
hex::encode(Sha256::digest(canonical_request.as_bytes()))
);
let mut mac =
HmacSha256::new_from_slice(format!("AWS4{}", self.secret_key).as_bytes()).unwrap();
mac.update(date_stamp.as_bytes());
let k_date = mac.finalize().into_bytes();
let mut mac = HmacSha256::new_from_slice(&k_date).unwrap();
mac.update(self.region.as_bytes());
let k_region = mac.finalize().into_bytes();
let mut mac = HmacSha256::new_from_slice(&k_region).unwrap();
mac.update(b"s3");
let k_service = mac.finalize().into_bytes();
let mut mac = HmacSha256::new_from_slice(&k_service).unwrap();
mac.update(b"aws4_request");
let k_signing = mac.finalize().into_bytes();
let mut mac = HmacSha256::new_from_slice(&k_signing).unwrap();
mac.update(string_to_sign.as_bytes());
let signature = hex::encode(mac.finalize().into_bytes());
let auth = format!(
"AWS4-HMAC-SHA256 Credential={}/{scope}, SignedHeaders={signed_headers}, Signature={signature}",
self.access_key
);
let agent = ureq::AgentBuilder::new()
.timeout(std::time::Duration::from_secs(30))
.build();
let mut req = agent.request(method, url);
req = req
.set("Authorization", &auth)
.set("x-amz-content-sha256", &payload_hash)
.set("x-amz-date", &amz_date);
if method == "PUT" && !content_type.is_empty() {
req = req.set("Content-Type", content_type);
}
let result = if method == "PUT" {
req.send_bytes(body)
} else {
req.call()
};
match result {
Ok(r) => {
let status = r.status();
let ct = r
.header("content-type")
.unwrap_or("application/octet-stream")
.to_string();
let mut buf = Vec::new();
let _ = std::io::Read::read_to_end(&mut r.into_reader(), &mut buf);
Ok((status, ct, buf))
}
Err(ureq::Error::Status(code, r)) => {
let ct = r
.header("content-type")
.unwrap_or("application/octet-stream")
.to_string();
let mut buf = Vec::new();
let _ = std::io::Read::read_to_end(&mut r.into_reader(), &mut buf);
Ok((code, ct, buf))
}
Err(e) => Err(e.to_string()),
}
}
}
fn split_url(url: &str) -> Result<(String, String), String> {
let without = url
.strip_prefix("https://")
.or_else(|| url.strip_prefix("http://"))
.ok_or_else(|| "url must be http(s)".to_string())?;
let scheme = if url.starts_with("https://") {
"https"
} else {
"http"
};
let (hostport, rest) = match without.find('/') {
Some(i) => (&without[..i], &without[i..]),
None => (without, "/"),
};
Ok((format!("{scheme}://{hostport}"), rest.to_string()))
}
fn amz_timestamp() -> String {
let secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
// UTC YYYYMMDDTHHMMSSZ via a tiny formatter (no chrono dep).
let days = secs / 86400;
let tod = secs % 86400;
let hour = tod / 3600;
let min = (tod % 3600) / 60;
let sec = tod % 60;
let (y, m, d) = civil_from_days(days as i64);
format!("{y:04}{m:02}{d:02}T{hour:02}{min:02}{sec:02}Z")
}
/// Algorithm from Howard Hinnant's date algorithms (days since 1970-01-01 → Y-M-D).
fn civil_from_days(z: i64) -> (i32, u32, u32) {
let z = z + 719468;
let era = if z >= 0 { z } else { z - 146096 } / 146097;
let doe = (z - era * 146097) as u64;
let yoe = (doe - doe / 1460 + doe / 36524 - doe / 146096) / 365;
let y = (yoe as i64) + era * 400;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let d = doy - (153 * mp + 2) / 5 + 1;
let m = if mp < 10 { mp + 3 } else { mp - 9 };
let y = if m <= 2 { y + 1 } else { y };
(y as i32, m as u32, d as u32)
}
fn uri_encode(s: &str) -> String {
let mut out = String::new();
for b in s.bytes() {
match b {
b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'_' | b'.' | b'~' => {
out.push(b as char)
}
_ => out.push_str(&format!("%{b:02X}")),
}
}
out
}
fn canonical_query_string(query: &str) -> String {
if query.is_empty() {
return String::new();
}
let mut pairs: Vec<(String, String)> = query
.split('&')
.filter(|p| !p.is_empty())
.map(|p| {
let (k, v) = p.split_once('=').unwrap_or((p, ""));
(pct_decode(k), pct_decode(v))
})
.collect();
pairs.sort_by(|a, b| a.0.cmp(&b.0).then(a.1.cmp(&b.1)));
pairs
.iter()
.map(|(k, v)| format!("{}={}", uri_encode(k), uri_encode(v)))
.collect::<Vec<_>>()
.join("&")
}
fn pct_decode(s: &str) -> String {
let mut out = String::with_capacity(s.len());
let b = s.as_bytes();
let mut i = 0;
while i < b.len() {
if b[i] == b'%' && i + 2 < b.len() {
if let Ok(v) = u8::from_str_radix(std::str::from_utf8(&b[i + 1..i + 3]).unwrap_or(""), 16)
{
out.push(v as char);
i += 3;
continue;
}
}
out.push(b[i] as char);
i += 1;
}
out
}
#[cfg(test)]
mod tests {
use super::*;
use std::time::{SystemTime, UNIX_EPOCH};
#[test]
fn file_put_get_list_delete() {
let nanos = SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap()
.as_nanos();
let dir = std::env::temp_dir().join(format!("marqdo-storage-{nanos}"));
let url = format!("file:{}", dir.display());
open(&url).unwrap();
put(&url, "a/hello.txt", Some("hi"), None, Some("text/plain")).unwrap();
let g = get(&url, "a/hello.txt").unwrap();
assert_eq!(g["ok"], true);
assert_eq!(g["body"], "hi");
let ls = list(&url, Some("a/")).unwrap();
assert_eq!(ls["count"], 1);
delete(&url, "a/hello.txt").unwrap();
assert_eq!(get(&url, "a/hello.txt").unwrap()["ok"], false);
let _ = fs::remove_dir_all(dir);
}
#[test]
fn s3_open_validates_bucket() {
let r = open("s3://mybucket?endpoint=http://127.0.0.1:9000");
assert!(r.is_ok());
assert!(open("s3://").is_err());
}
}
-898
View File
@@ -1,898 +0,0 @@
//! GFM table helpers + site path parsing (cells stay string literals).
use serde_json::{json, Map, Value};
const FIELD_KEYS: &[&str] = &["字段", "name", "列", "column"];
const TYPE_KEYS: &[&str] = &["类型", "type"];
const NULL_KEYS: &[&str] = &["可空", "null", "nullable"];
const FRONT_KEYS: &[&str] = &["属性", "前端变量", "front", "field"];
const BACK_KEYS: &[&str] = &["值", "后端数据库", "back", "db"];
const CSS_KEYS: &[&str] = &["样式", "绑定css样式", "css", "class", "style"];
const SRC_KEYS: &[&str] = &["组件", "导入的页面", "src", "page"];
const STYLE_KEYS: &[&str] = &["样式", "style", "class"];
const PROP_KEYS: &[&str] = &["属性", "property", "prop", "名", "name"];
const VAL_KEYS: &[&str] = &["值", "value"];
const SEL_KEYS: &[&str] = &["选择器", "selector", "sel"];
const MEDIA_KEYS: &[&str] = &["媒体", "media", "mq", "@media"];
const WHEN_KEYS: &[&str] = &["当", "when", "if", "条件", "visible"];
const NAV_LABEL_KEYS: &[&str] = &["标签", "label", "title", "text", "名"];
const NAV_HREF_KEYS: &[&str] = &["链接", "href", "url", "path", "地址"];
pub fn normalize_ref(s: &str) -> String {
let s = s.trim();
let mut out = String::with_capacity(s.len());
let mut rest = s;
while let Some(start) = rest.find('`') {
out.push_str(&rest[..start]);
rest = &rest[start + 1..];
if let Some(end) = rest.find('`') {
out.push_str(&rest[..end]);
rest = &rest[end + 1..];
} else {
out.push('`');
out.push_str(rest);
return out;
}
}
out.push_str(rest);
out
}
fn pick<'a>(m: &'a Map<String, Value>, keys: &[&str]) -> Option<&'a Value> {
keys.iter().find_map(|k| m.get(*k))
}
fn cell_str(v: &Value) -> String {
match v {
Value::Null => String::new(),
Value::String(s) => s.clone(),
Value::Bool(b) => b.to_string(),
Value::Number(n) => n.to_string(),
other => other.to_string(),
}
}
fn as_str_list(v: &Value) -> Vec<String> {
match v {
Value::Array(a) => a.iter().map(cell_str).collect(),
Value::Null => Vec::new(),
other => vec![cell_str(other)],
}
}
fn boolish(v: &Value) -> bool {
match v {
Value::Bool(b) => *b,
Value::String(s) => matches!(s.trim(), "1" | "true" | "True" | "yes" | "是" | "可"),
Value::Number(n) => n.as_i64().unwrap_or(0) != 0,
_ => false,
}
}
const UNIQUE_KEYS: &[&str] = &["唯一", "unique", "uniq"];
const INDEX_KEYS: &[&str] = &["索引", "index", "idx"];
const FK_KEYS: &[&str] = &["外键", "fk", "references", "ref", "引用"];
/// Schema table → `[{name,type,nullable,unique,index,fk}, …]`
pub fn as_fields(table: &Value) -> Value {
match table {
Value::Array(rows) => {
let mut out = Vec::new();
for row in rows {
if let Some(m) = row.as_object() {
let name = pick(m, FIELD_KEYS).map(cell_str).unwrap_or_default();
if name.is_empty() {
continue;
}
let ty = pick(m, TYPE_KEYS)
.map(cell_str)
.unwrap_or_else(|| "text".into());
let nullable = pick(m, NULL_KEYS).map(boolish).unwrap_or(true);
let unique = pick(m, UNIQUE_KEYS).map(boolish).unwrap_or(false);
let index = pick(m, INDEX_KEYS).map(boolish).unwrap_or(false);
let fk = pick(m, FK_KEYS)
.map(cell_str)
.filter(|s| !s.is_empty())
.unwrap_or_default();
out.push(json!({
"name": name,
"type": ty,
"nullable": nullable,
"unique": unique,
"index": index,
"fk": fk,
}));
}
}
Value::Array(out)
}
Value::Object(m) => {
let names = pick(m, FIELD_KEYS).map(as_str_list).unwrap_or_default();
let types = pick(m, TYPE_KEYS).map(as_str_list).unwrap_or_default();
let nulls = pick(m, NULL_KEYS).map(as_str_list).unwrap_or_default();
let uniques = pick(m, UNIQUE_KEYS).map(as_str_list).unwrap_or_default();
let indexes = pick(m, INDEX_KEYS).map(as_str_list).unwrap_or_default();
let fks = pick(m, FK_KEYS).map(as_str_list).unwrap_or_default();
let mut out = Vec::new();
for (i, name) in names.iter().enumerate() {
if name.is_empty() {
continue;
}
let ty = types.get(i).cloned().unwrap_or_else(|| "text".into());
let nullable = nulls
.get(i)
.map(|s| boolish(&Value::String(s.clone())))
.unwrap_or(true);
let unique = uniques
.get(i)
.map(|s| boolish(&Value::String(s.clone())))
.unwrap_or(false);
let index = indexes
.get(i)
.map(|s| boolish(&Value::String(s.clone())))
.unwrap_or(false);
let fk = fks.get(i).cloned().unwrap_or_default();
out.push(json!({
"name": name,
"type": ty,
"nullable": nullable,
"unique": unique,
"index": index,
"fk": fk,
}));
}
Value::Array(out)
}
_ => json!([]),
}
}
/// Seed / row table → list of maps.
pub fn as_rows(table: &Value) -> Value {
match table {
Value::Array(a) => Value::Array(a.clone()),
Value::Object(m) => {
// @ / 行 / row column → list of maps
let keys: Vec<String> = m.keys().cloned().collect();
let row_key = ["@", "行", "row"]
.iter()
.find(|k| m.contains_key(**k))
.map(|s| s.to_string());
if let Some(rk) = row_key {
let n = as_str_list(m.get(&rk).unwrap()).len();
let mut out = Vec::new();
for i in 0..n {
let mut row = Map::new();
for k in &keys {
if k == &rk {
continue;
}
let list = as_str_list(m.get(k).unwrap_or(&Value::Null));
row.insert(k.clone(), json!(list.get(i).cloned().unwrap_or_default()));
}
if let Some(id) = as_str_list(m.get(&rk).unwrap()).get(i) {
if !id.is_empty() {
row.insert("id".into(), json!(id));
}
}
out.push(Value::Object(row));
}
return Value::Array(out);
}
// Columnar GFM table: each value is a same-length array → zip into rows.
let mut col_lens: Vec<usize> = Vec::new();
let mut all_arrays = !m.is_empty();
for v in m.values() {
match v {
Value::Array(a) => col_lens.push(a.len()),
_ => {
all_arrays = false;
break;
}
}
}
if all_arrays {
let n = col_lens.iter().copied().max().unwrap_or(0);
if n > 0 && col_lens.iter().all(|&l| l == n || l == 0) {
let mut out = Vec::new();
for i in 0..n {
let mut row = Map::new();
for (k, v) in m {
if let Value::Array(a) = v {
row.insert(
k.clone(),
a.get(i).cloned().unwrap_or(Value::Null),
);
}
}
out.push(Value::Object(row));
}
return Value::Array(out);
}
}
// single map
Value::Array(vec![Value::Object(m.clone())])
}
_ => json!([]),
}
}
/// Bind table → `[{front,back,css,media,when}, …]`
pub fn as_bind(table: &Value) -> Value {
match table {
Value::Array(rows) => {
let mut out = Vec::new();
for row in rows {
if let Some(m) = row.as_object() {
let front = normalize_ref(&pick(m, FRONT_KEYS).map(cell_str).unwrap_or_default());
let back = normalize_ref(&pick(m, BACK_KEYS).map(cell_str).unwrap_or_default());
let css = normalize_ref(&pick(m, CSS_KEYS).map(cell_str).unwrap_or_default());
let media =
normalize_ref(&pick(m, MEDIA_KEYS).map(cell_str).unwrap_or_default());
let when = normalize_ref(&pick(m, WHEN_KEYS).map(cell_str).unwrap_or_default());
if !front.is_empty() || !back.is_empty() {
out.push(json!({
"front": front,
"back": back,
"css": css,
"media": media,
"when": when,
}));
}
}
}
Value::Array(out)
}
Value::Object(m) => {
let fronts = pick(m, FRONT_KEYS).map(as_str_list).unwrap_or_default();
let backs = pick(m, BACK_KEYS).map(as_str_list).unwrap_or_default();
let csses = pick(m, CSS_KEYS).map(as_str_list).unwrap_or_default();
let medias = pick(m, MEDIA_KEYS).map(as_str_list).unwrap_or_default();
let whens = pick(m, WHEN_KEYS).map(as_str_list).unwrap_or_default();
let n = fronts.len().max(backs.len());
let mut out = Vec::new();
for i in 0..n {
let front = normalize_ref(&fronts.get(i).cloned().unwrap_or_default());
let back = normalize_ref(&backs.get(i).cloned().unwrap_or_default());
let css = normalize_ref(&csses.get(i).cloned().unwrap_or_default());
let media = normalize_ref(&medias.get(i).cloned().unwrap_or_default());
let when = normalize_ref(&whens.get(i).cloned().unwrap_or_default());
if !front.is_empty() || !back.is_empty() {
out.push(json!({
"front": front,
"back": back,
"css": css,
"media": media,
"when": when,
}));
}
}
Value::Array(out)
}
_ => json!([]),
}
}
/// Nav / link table → row maps with label/href (and optional media/when/class).
/// Accepts bind rows (`front`/`back`) or author tables (`标签`/`链接`).
pub fn as_nav_rows(table: &Value) -> Vec<Map<String, Value>> {
let binds = as_bind(table);
let bind_arr = binds.as_array().cloned().unwrap_or_default();
if !bind_arr.is_empty() {
return bind_arr
.into_iter()
.filter_map(|v| v.as_object().cloned())
.collect();
}
as_rows(table)
.as_array()
.cloned()
.unwrap_or_default()
.into_iter()
.filter_map(|v| v.as_object().cloned())
.collect()
}
/// Pick label/href from a nav row (bind or 标签/链接 shape).
pub fn nav_label_href(m: &Map<String, Value>) -> (String, String) {
let label = pick(m, NAV_LABEL_KEYS)
.or_else(|| m.get("front"))
.map(cell_str)
.map(|s| normalize_ref(&s))
.unwrap_or_default();
let href = pick(m, NAV_HREF_KEYS)
.or_else(|| m.get("back"))
.map(cell_str)
.map(|s| normalize_ref(&s))
.filter(|s| !s.is_empty())
.unwrap_or_else(|| "#".into());
(label, href)
}
pub fn nav_media_when_class(m: &Map<String, Value>) -> (String, String, String) {
let media = pick(m, MEDIA_KEYS)
.or_else(|| m.get("media"))
.map(cell_str)
.map(|s| normalize_ref(&s))
.unwrap_or_default();
let when = pick(m, WHEN_KEYS)
.or_else(|| m.get("when"))
.map(cell_str)
.map(|s| normalize_ref(&s))
.unwrap_or_default();
let class = pick(m, CSS_KEYS)
.or_else(|| m.get("css"))
.map(cell_str)
.map(|s| normalize_ref(&s))
.unwrap_or_default();
(media, when, class)
}
/// Page table → `[{src,style,slot}, …]`
pub fn as_compose(table: &Value) -> Value {
match table {
Value::Object(m) => {
let srcs = pick(m, SRC_KEYS).map(as_str_list).unwrap_or_default();
let styles = pick(m, STYLE_KEYS).map(as_str_list).unwrap_or_default();
let mut out = Vec::new();
for (i, src_raw) in srcs.iter().enumerate() {
let src = normalize_ref(src_raw);
if src.is_empty() {
continue;
}
let style = normalize_ref(&styles.get(i).cloned().unwrap_or_default());
let slot = infer_slot(&src);
out.push(json!({ "src": src, "style": style, "slot": slot }));
}
Value::Array(out)
}
Value::Array(rows) => {
let mut out = Vec::new();
for row in rows {
if let Some(m) = row.as_object() {
let src = normalize_ref(&pick(m, SRC_KEYS).map(cell_str).unwrap_or_default());
if src.is_empty() {
continue;
}
let style = normalize_ref(&pick(m, STYLE_KEYS).map(cell_str).unwrap_or_default());
let slot = infer_slot(&src);
out.push(json!({ "src": src, "style": style, "slot": slot }));
}
}
Value::Array(out)
}
_ => json!([]),
}
}
fn infer_slot(src: &str) -> String {
let base = src.rsplit('.').next().unwrap_or(src);
match base {
"nav" | "links" | "导航" => "nav".into(),
"side" | "sidebar" | "侧栏" => "sidebar".into(),
"foot" | "footer" | "页脚" => "footer".into(),
other => other.to_string(),
}
}
pub fn normalize_slot(name: &str) -> String {
match name {
"nav" | "links" | "导航" => "nav".into(),
"side" | "sidebar" | "侧栏" => "sidebar".into(),
"foot" | "footer" | "页脚" => "footer".into(),
"main" | "index" | "" | "主体" => "main".into(),
other => other.to_string(),
}
}
/// Style table → CSS text.
///
/// Table shapes:
/// 1. Rule rows `|媒体|选择器|属性|值|` — rows grouped by `媒体` emit
/// `@media { … }` blocks; rows sharing a selector are merged into one rule.
/// 2. Rule rows `|选择器|属性|值|` — each row emits `selector { prop: value; }`,
/// rows sharing a selector are merged into one rule. `selector` may be any
/// CSS selector (`aside.side`, `ul.side-nav a:hover`).
/// 3. Plain property rows `|属性|值|` — emitted as `.name { prop: value; }`.
/// 4. **`@keyframes`** — selector `@keyframes name` (optional stop in the
/// selector: `@keyframes name from`) or stop in the `属性` column:
/// `|@keyframes fade|from|opacity: 0|` → nested keyframe rules.
///
/// Examples (rule shape):
/// ```text
/// | 媒体 | 选择器 | 属性 | 值 |
/// |------|--------|------|-----|
/// | (max-width: 860px) | aside.side | padding | 1rem 2rem |
///
/// | 选择器 | 属性 | 值 |
/// |--------|------|-----|
/// | aside.side | padding | 2rem 1.25rem |
/// | @keyframes pulse | 0% | opacity: 0 |
/// | @keyframes pulse | 100% | opacity: 1 |
/// | @keyframes pulse from | opacity | 0 |
/// ```
pub fn as_css_named(name: &str, table: &Value) -> String {
as_css_named_checked(name, table, false).unwrap_or_default()
}
/// Like [`as_css_named`], but when `strict` is true, reject cells that look like
/// T5 evaluated a bare `/` (numeric/bool CSS values). Default mode warns on stderr.
pub fn as_css_named_checked(name: &str, table: &Value, strict: bool) -> Result<String, String> {
let name = normalize_ref(name);
if name.is_empty() {
return Ok(String::new());
}
let mut out = String::new();
match table {
Value::Object(m) => {
let sels = pick(m, SEL_KEYS).map(as_str_list).unwrap_or_default();
let props = pick(m, PROP_KEYS).map(as_str_list).unwrap_or_default();
let vals_raw = pick(m, VAL_KEYS);
let vals = vals_raw.map(as_str_list).unwrap_or_default();
let medias = pick(m, MEDIA_KEYS).map(as_str_list).unwrap_or_default();
let n = props.len().max(sels.len()).max(vals.len());
if n == 0 {
return Ok(String::new());
}
if let Some(raw) = vals_raw {
check_css_value_cells(raw, &props, strict)?;
}
let rows: Vec<(String, String, String, String)> = (0..n)
.map(|i| {
(
medias.get(i).cloned().unwrap_or_default(),
sels.get(i).cloned().unwrap_or_default(),
props.get(i).cloned().unwrap_or_default(),
vals.get(i).cloned().unwrap_or_default(),
)
})
.collect();
emit_css_rows(&rows, &name, &mut out);
}
Value::Array(rows) => {
let mut collected = Vec::new();
for row in rows {
let Some(m) = row.as_object() else {
continue;
};
let prop = pick(m, PROP_KEYS).map(cell_str).unwrap_or_default();
if let Some(v) = pick(m, VAL_KEYS) {
note_css_value_cell(&prop, v, strict)?;
}
collected.push((
pick(m, MEDIA_KEYS).map(cell_str).unwrap_or_default(),
pick(m, SEL_KEYS).map(cell_str).unwrap_or_default(),
prop,
pick(m, VAL_KEYS).map(cell_str).unwrap_or_default(),
));
}
emit_css_rows(&collected, &name, &mut out);
}
_ => {}
}
Ok(out)
}
fn css_value_suspicious(v: &Value) -> Option<&'static str> {
// Numbers are normal CSS (z-index, opacity, …). Do not treat Markup-evaluated
// ints/floats as bare `/` division false positives.
match v {
Value::Bool(_) => Some("boolean CSS value (quote the cell if intentional)"),
_ => None,
}
}
fn note_css_value_cell(prop: &str, val: &Value, strict: bool) -> Result<(), String> {
let Some(why) = css_value_suspicious(val) else {
return Ok(());
};
let prop = if prop.trim().is_empty() {
"(value)"
} else {
prop.trim()
};
let msg = format!("ext/web style: suspicious cell for `{prop}`: {why}");
if strict {
Err(msg)
} else {
eprintln!("warning: {msg}");
Ok(())
}
}
fn check_css_value_cells(vals: &Value, props: &[String], strict: bool) -> Result<(), String> {
match vals {
Value::Array(a) => {
for (i, v) in a.iter().enumerate() {
let prop = props.get(i).map(|s| s.as_str()).unwrap_or("");
note_css_value_cell(prop, v, strict)?;
}
}
other => note_css_value_cell(props.first().map(|s| s.as_str()).unwrap_or(""), other, strict)?,
}
Ok(())
}
/// Parse `@keyframes name` or `@keyframes name stop` from a selector cell.
fn parse_keyframes_sel(sel: &str) -> Option<(String, Option<String>)> {
let t = sel.trim();
if t.len() < 10 || !t[..10].eq_ignore_ascii_case("@keyframes") {
return None;
}
let rest = t[10..].trim();
if rest.is_empty() {
return None;
}
// First token = animation name; optional second = keyframe stop.
let mut parts = rest.split_whitespace();
let anim = parts.next()?.to_string();
let stop = parts.next().map(|s| s.to_string());
Some((anim, stop))
}
fn is_keyframe_stop(s: &str) -> bool {
let t = s.trim();
if t.is_empty() {
return false;
}
if t.eq_ignore_ascii_case("from") || t.eq_ignore_ascii_case("to") {
return true;
}
// 0%, 50%, 100.5%, etc.
let stripped = t.trim_end_matches('%');
!stripped.is_empty()
&& stripped
.chars()
.all(|c| c.is_ascii_digit() || c == '.')
&& t.ends_with('%')
}
fn keyframe_decl_lines(css_prop: &str, value: &str) -> Vec<String> {
// Value may be a full declaration (`opacity: 0`) or bare value when the
// CSS property lived in the selector form `@keyframes name from` + 属性.
let v = value.trim().trim_end_matches(';').trim();
if v.is_empty() {
return Vec::new();
}
if v.contains(':') {
v.split(';')
.map(|p| p.trim())
.filter(|p| !p.is_empty())
.map(|p| format!(" {p};"))
.collect()
} else if !css_prop.is_empty() && !is_keyframe_stop(css_prop) {
// `@keyframes name from` + 属性=opacity + 值=0
vec![format!(" {css_prop}: {v};")]
} else {
// Bare value — prefer `opacity: 0` in the 值 column.
vec![format!(" {v};")]
}
}
fn push_keyframe(
groups: &mut Vec<(String, Vec<(String, Vec<String>)>)>,
anim: &str,
stop: &str,
decls: Vec<String>,
) {
if anim.is_empty() || stop.is_empty() || decls.is_empty() {
return;
}
let anim_group = match groups.iter_mut().find(|(n, _)| n == anim) {
Some((_, g)) => g,
None => {
groups.push((anim.to_string(), Vec::new()));
&mut groups.last_mut().unwrap().1
}
};
match anim_group.iter_mut().find(|(s, _)| s == stop) {
Some((_, d)) => d.extend(decls),
None => anim_group.push((stop.to_string(), decls)),
}
}
fn emit_css_rows(rows: &[(String, String, String, String)], class_name: &str, out: &mut String) {
let mut by_media: Vec<(String, Vec<(String, Vec<String>)>)> = Vec::new();
let mut by_keyframes: Vec<(String, Vec<(String, Vec<String>)>)> = Vec::new();
let mut plain: Vec<String> = Vec::new();
for (media, sel, p, v) in rows {
let media = media.trim();
let sel = sel.trim();
let p = p.trim();
let v = v.trim();
if let Some((anim, stop_in_sel)) = parse_keyframes_sel(sel) {
let stop_from_sel = stop_in_sel.is_some();
let (stop, prop_for_decl) = match stop_in_sel {
Some(s) => (s, p.to_string()),
None => {
// 属性 = keyframe stop (from / to / N%); 值 = declaration(s)
if !is_keyframe_stop(p) {
continue;
}
(p.to_string(), String::new())
}
};
let decls = if stop_from_sel {
keyframe_decl_lines(&prop_for_decl, v)
} else {
keyframe_decl_lines("", v)
};
let decls = if decls.is_empty() && !v.is_empty() && !prop_for_decl.is_empty() {
keyframe_decl_lines(&prop_for_decl, v)
} else {
decls
};
push_keyframe(&mut by_keyframes, &anim, &stop, decls);
continue;
}
if p.is_empty() {
continue;
}
let decl = format!(" {p}: {v};");
if !media.is_empty() {
let group = match by_media.iter_mut().find(|(mq, _)| *mq == media) {
Some((_, g)) => g,
None => {
by_media.push((media.to_string(), Vec::new()));
let (_, g) = by_media.last_mut().unwrap();
g
}
};
push_rule(group, sel, decl);
} else if !sel.is_empty() {
let group = match by_media.iter_mut().find(|(mq, _)| mq.is_empty()) {
Some((_, g)) => g,
None => {
by_media.push((String::new(), Vec::new()));
let (_, g) = by_media.last_mut().unwrap();
g
}
};
push_rule(group, sel, decl);
} else {
plain.push(decl);
}
}
for (media, rules) in by_media {
if media.is_empty() {
for (sel, decls) in rules {
out.push_str(&format!("{} {{\n{}\n}}\n", sel, decls.join("\n")));
}
} else {
let inner = rules
.iter()
.map(|(sel, decls)| format!("{} {{\n{}\n}}", sel, decls.join("\n")))
.collect::<Vec<_>>()
.join("\n");
out.push_str(&format!("@media {} {{\n{}\n}}\n", media, inner));
}
}
for (anim, stops) in by_keyframes {
out.push_str(&format!("@keyframes {} {{\n", anim));
for (stop, decls) in stops {
out.push_str(&format!(" {} {{\n", stop));
out.push_str(&decls.join("\n"));
out.push('\n');
out.push_str(" }\n");
}
out.push_str("}\n");
}
if !plain.is_empty() {
out.push_str(&format!(".{} {{\n{}\n}}\n", class_name, plain.join("\n")));
}
}
/// Push `decl` under `sel` into a `(selector, declarations)` group list,
/// merging declarations when the selector already exists.
fn push_rule(groups: &mut Vec<(String, Vec<String>)>, sel: &str, decl: String) {
if sel.is_empty() {
return;
}
match groups.iter_mut().find(|(s, _)| s == sel) {
Some((_, decls)) => decls.push(decl),
None => groups.push((sel.to_string(), vec![decl])),
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum SitePath {
LibMember { lib: String, member: String },
DbField { table: String, field: String },
Plain(String),
}
pub fn parse_site_path(raw: &str) -> SitePath {
let s = normalize_ref(raw);
if s.is_empty() {
return SitePath::Plain(String::new());
}
let parts: Vec<&str> = s.split('.').filter(|p| !p.is_empty()).collect();
match parts.as_slice() {
[lib, member] => SitePath::LibMember {
lib: (*lib).into(),
member: (*member).into(),
},
[_, table, field] => SitePath::DbField {
table: (*table).into(),
field: (*field).into(),
},
_ => SitePath::Plain(s),
}
}
pub fn bind_table_name(binds: &[Value]) -> Option<String> {
let mut found: Option<String> = None;
for b in binds {
let back = b.get("back").and_then(|v| v.as_str()).unwrap_or("");
if let SitePath::DbField { table, .. } = parse_site_path(back) {
match &found {
None => found = Some(table),
Some(prev) if prev != &table => return None,
_ => {}
}
} else if back.contains('.') {
let mut p = back.splitn(2, '.');
if let (Some(t), Some(_)) = (p.next(), p.next()) {
match &found {
None => found = Some(t.into()),
Some(prev) if prev != t => return None,
_ => {}
}
}
}
}
found
}
pub fn project_rows(binds: &[Value], rows: &[Value]) -> Value {
let mut out = Vec::new();
for row in rows {
let obj = row.as_object().cloned().unwrap_or_default();
let mut m = Map::new();
let mut css = Map::new();
for b in binds {
let front = b.get("front").and_then(|v| v.as_str()).unwrap_or("");
let back = b.get("back").and_then(|v| v.as_str()).unwrap_or("");
let class = b.get("css").and_then(|v| v.as_str()).unwrap_or("");
let col = match parse_site_path(back) {
SitePath::DbField { field, .. } => field,
SitePath::LibMember { member, .. } => member,
SitePath::Plain(s) if s.contains('.') => {
s.split('.').next_back().unwrap_or(&s).to_string()
}
SitePath::Plain(s) => s,
};
if let Some(v) = obj.get(&col).or_else(|| obj.get(front)) {
m.insert(front.to_string(), v.clone());
}
if !class.is_empty() {
css.insert(front.to_string(), json!(class));
}
}
if let Some(id) = obj.get("id") {
m.insert("id".into(), id.clone());
}
if !css.is_empty() {
m.insert("_css".into(), Value::Object(css));
}
out.push(Value::Object(m));
}
Value::Array(out)
}
const META_KEY_KEYS: &[&str] = &["键", "key", "name"];
const META_VAL_KEYS: &[&str] = &["值", "value"];
/// SEO / OpenGraph meta table → `{title, description, og:…, canonical, …}`.
pub fn as_meta_map(table: &Value) -> Map<String, Value> {
let mut out = Map::new();
match table {
Value::Array(rows) => {
for row in rows {
let Some(obj) = row.as_object() else {
continue;
};
let key = META_KEY_KEYS
.iter()
.find_map(|k| obj.get(*k))
.map(|v| normalize_ref(&meta_text(v)))
.unwrap_or_default();
let val = META_VAL_KEYS
.iter()
.find_map(|k| obj.get(*k))
.map(|v| meta_text(v))
.unwrap_or_default();
if !key.is_empty() {
out.insert(key, json!(val));
}
}
}
Value::Object(m) => {
for (k, v) in m {
out.insert(k.clone(), json!(meta_text(v)));
}
}
_ => {}
}
out
}
fn meta_text(v: &Value) -> String {
match v {
Value::Null => String::new(),
Value::String(s) => s.clone(),
Value::Bool(b) => b.to_string(),
Value::Number(n) => n.to_string(),
other => other.to_string(),
}
}
#[cfg(test)]
mod css_tests {
use super::*;
use serde_json::json;
#[test]
fn keyframes_stop_in_prop() {
let t = json!([
{"选择器": "@keyframes pulse", "属性": "0%", "值": "opacity: 0"},
{"选择器": "@keyframes pulse", "属性": "100%", "值": "opacity: 1"},
]);
let css = as_css_named("anim", &t);
assert!(css.contains("@keyframes pulse {"), "{css}");
assert!(css.contains("0% {"), "{css}");
assert!(css.contains("opacity: 0;"), "{css}");
assert!(css.contains("100% {"), "{css}");
assert!(!css.contains("0%: opacity"), "{css}");
}
#[test]
fn keyframes_stop_in_selector() {
let t = json!([
{"选择器": "@keyframes fade from", "属性": "opacity", "值": "0"},
{"选择器": "@keyframes fade to", "属性": "opacity", "值": "1"},
]);
let css = as_css_named("anim", &t);
assert!(css.contains("@keyframes fade {"), "{css}");
assert!(css.contains("from {"), "{css}");
assert!(css.contains("opacity: 0;"), "{css}");
assert!(css.contains("to {"), "{css}");
}
#[test]
fn media_still_works() {
let t = json!([
{"媒体": "(max-width: 800px)", "选择器": ".x", "属性": "color", "值": "red"},
]);
let css = as_css_named("x", &t);
assert!(css.contains("@media (max-width: 800px)"), "{css}");
assert!(css.contains(".x {"), "{css}");
assert!(css.contains("color: red;"), "{css}");
}
#[test]
fn strict_rejects_numeric_css_value() {
let t = json!([
{"选择器": ".x", "属性": "grid-column", "值": 0},
]);
let err = as_css_named_checked("x", &t, true).unwrap_err();
assert!(err.contains("suspicious"), "{err}");
// Non-strict still emits (with warning on stderr).
let css = as_css_named_checked("x", &t, false).unwrap();
assert!(css.contains("grid-column: 0;"), "{css}");
}
#[test]
fn quoted_string_ok_in_strict() {
let t = json!([
{"选择器": ".x", "属性": "grid-column", "值": "1 / 5"},
]);
let css = as_css_named_checked("x", &t, true).unwrap();
assert!(css.contains("grid-column: 1 / 5;"), "{css}");
}
}
-264
View File
@@ -1,264 +0,0 @@
//! HTTP upload validation and safe object keys (W5).
use serde_json::{json, Value};
use std::fs;
use std::time::{SystemTime, UNIX_EPOCH};
/// Validate size / MIME / extension against an optional allowlist table.
///
/// `types` may be:
/// - `null` / missing → allow any
/// - string CSV of MIME types (`image/png,image/jpeg`)
/// - GFM-style rows: `|类型|扩展名|` / `|type|ext|` (ext may be comma-separated)
pub fn validate(
filename: &str,
content_type: &str,
size: u64,
max_bytes: u64,
types: Option<&Value>,
) -> Result<Value, String> {
if size > max_bytes {
return Ok(json!({
"ok": false,
"error": format!("file too large: {size} > {max_bytes}"),
}));
}
let name = filename.trim();
if name.is_empty() {
return Ok(json!({ "ok": false, "error": "empty filename" }));
}
if name.contains("..") || name.contains('/') || name.contains('\\') {
// Basename only is expected from clients; reject path fragments.
return Ok(json!({ "ok": false, "error": "unsafe filename" }));
}
let ct = content_type.trim().to_ascii_lowercase();
let ct = ct.split(';').next().unwrap_or(&ct).trim();
let ext = name
.rsplit('.')
.next()
.filter(|e| !e.is_empty() && *e != name)
.unwrap_or("")
.to_ascii_lowercase();
if let Some(t) = types {
if !types_allow(t, ct, &ext) {
return Ok(json!({
"ok": false,
"error": format!("type not allowed: content_type={ct} ext={ext}"),
}));
}
}
Ok(json!({
"ok": true,
"filename": name,
"content_type": if ct.is_empty() { "application/octet-stream" } else { ct },
"size": size,
"ext": ext,
}))
}
fn types_allow(types: &Value, content_type: &str, ext: &str) -> bool {
match types {
Value::Null => true,
Value::String(s) => {
let s = s.trim();
if s.is_empty() {
return true;
}
s.split(',')
.map(|p| p.trim().to_ascii_lowercase())
.filter(|p| !p.is_empty())
.any(|p| p == content_type)
}
Value::Array(rows) => {
if rows.is_empty() {
return true;
}
for row in rows {
if row_allows(row, content_type, ext) {
return true;
}
}
false
}
Value::Object(obj) => {
// Shapes:
// 1) Columnar GFM: `{类型: [mime…], 扩展名: [ext…]}` (Marqdo default)
// 2) Single allowlist row: `{类型|type, 扩展名|ext}` with string values
// 3) MIME→ext map: `{ "image/png": "png", … }`
let col_mime = obj
.get("类型")
.or_else(|| obj.get("type"))
.or_else(|| obj.get("mime"));
let col_ext = obj
.get("扩展名")
.or_else(|| obj.get("ext"))
.or_else(|| obj.get("extension"));
if let (Some(Value::Array(mimes)), Some(exts_v)) = (col_mime, col_ext) {
let exts: Vec<String> = match exts_v {
Value::Array(a) => a
.iter()
.map(|x| x.as_str().unwrap_or("").to_ascii_lowercase())
.collect(),
Value::String(s) => vec![s.to_ascii_lowercase()],
_ => Vec::new(),
};
if mimes.is_empty() {
return true;
}
for (i, mime_v) in mimes.iter().enumerate() {
let mime = mime_v
.as_str()
.unwrap_or("")
.trim()
.to_ascii_lowercase();
let row_ext = exts.get(i).map(|s| s.as_str()).unwrap_or("");
let mime_ok = mime.is_empty() || mime == content_type;
let ext_ok = if row_ext.is_empty() {
true
} else {
row_ext
.split(',')
.map(|e| e.trim().trim_start_matches('.'))
.any(|e| e == ext)
};
if mime_ok && ext_ok {
return true;
}
}
return false;
}
if col_mime.is_some() || col_ext.is_some() {
return row_allows(&Value::Object(obj.clone()), content_type, ext);
}
if obj.is_empty() {
return true;
}
for (mime, exts_v) in obj {
let mime = mime.trim().to_ascii_lowercase();
let exts = match exts_v {
Value::String(s) => s.clone(),
Value::Array(a) => a
.iter()
.filter_map(|x| x.as_str())
.collect::<Vec<_>>()
.join(","),
_ => String::new(),
};
let exts = exts.trim().to_ascii_lowercase();
let mime_ok = mime.is_empty() || mime == content_type;
let ext_ok = if exts.is_empty() {
true
} else {
exts.split(',')
.map(|e| e.trim().trim_start_matches('.'))
.any(|e| e == ext)
};
if mime_ok && ext_ok {
return true;
}
}
false
}
_ => true,
}
}
fn row_allows(row: &Value, content_type: &str, ext: &str) -> bool {
let Some(obj) = row.as_object() else {
if let Some(s) = row.as_str() {
return s.trim().eq_ignore_ascii_case(content_type);
}
return false;
};
let mime = obj
.get("类型")
.or_else(|| obj.get("type"))
.or_else(|| obj.get("mime"))
.and_then(|v| v.as_str())
.unwrap_or("")
.trim()
.to_ascii_lowercase();
let exts = obj
.get("扩展名")
.or_else(|| obj.get("ext"))
.or_else(|| obj.get("extension"))
.and_then(|v| v.as_str())
.unwrap_or("")
.trim()
.to_ascii_lowercase();
let mime_ok = mime.is_empty() || mime == content_type;
let ext_ok = if exts.is_empty() {
true
} else {
exts.split(',')
.map(|e| e.trim().trim_start_matches('.'))
.any(|e| e == ext)
};
mime_ok && ext_ok
}
/// Build `prefix` + timestamp-random + sanitized basename.
pub fn make_key(prefix: &str, filename: &str) -> Result<String, String> {
let base = filename
.rsplit(['/', '\\'])
.next()
.unwrap_or(filename)
.trim();
if base.is_empty() {
return Err("empty filename".into());
}
let safe: String = base
.chars()
.map(|c| {
if c.is_ascii_alphanumeric() || c == '.' || c == '-' || c == '_' {
c
} else {
'_'
}
})
.collect();
if safe.contains("..") {
return Err("unsafe filename".into());
}
let mut pref = prefix.trim().trim_start_matches('/').replace('\\', "/");
if !pref.is_empty() && !pref.ends_with('/') {
pref.push('/');
}
if pref.contains("..") {
return Err("prefix must not contain `..`".into());
}
let secs = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
let mut rnd = [0u8; 4];
getrandom::getrandom(&mut rnd).map_err(|e| e.to_string())?;
let hex = hex::encode(rnd);
Ok(format!("{pref}{secs}-{hex}-{safe}"))
}
/// Offline save: read local `path` bytes into storage under `key` (or auto key).
pub fn save(
storage_url: &str,
key: Option<&str>,
path: &str,
content_type: Option<&str>,
prefix: Option<&str>,
) -> Result<Value, String> {
let bytes = fs::read(path).map_err(|e| format!("read `{path}`: {e}"))?;
let filename = path
.rsplit(['/', '\\'])
.next()
.unwrap_or("upload.bin");
let ct = content_type
.filter(|s| !s.is_empty())
.unwrap_or("application/octet-stream");
let object_key = match key.filter(|k| !k.is_empty()) {
Some(k) => k.to_string(),
None => make_key(prefix.unwrap_or("uploads/"), filename)?,
};
crate::storage::put_bytes(storage_url, &object_key, &bytes, ct)
}
-132
View File
@@ -1,132 +0,0 @@
//! WebSocket client primitive (design ext-web-net §3.4).
//!
//! Single request–response style: connect, send one text `message`, collect all
//! server text replies until close, then return `{ok, messages}`.
use futures_util::{SinkExt, StreamExt};
use serde_json::{json, Value};
use tokio_tungstenite::tungstenite::http::header::HeaderName;
/// Convert `headers` (a JSON map) into tungstenite header pairs.
fn headers_from_value(headers: Option<&Value>) -> Vec<(HeaderName, String)> {
let Some(h) = headers else {
return Vec::new();
};
match h {
Value::Object(m) => m
.iter()
.filter_map(|(k, v)| {
HeaderName::from_bytes(k.as_bytes())
.ok()
.map(|hn| (hn, v.as_str().unwrap_or("").to_string()))
})
.collect(),
_ => Vec::new(),
}
}
/// Connect to a WebSocket URL, send `message`, drain text replies until close.
pub fn connect(
url: &str,
message: &str,
headers: Option<&Value>,
timeout_sec: u64,
) -> Result<Value, String> {
if !(url.starts_with("ws://") || url.starts_with("wss://")) {
return Err("ws url must start with ws:// or wss://".into());
}
let hdrs = headers_from_value(headers);
let mut builder = tokio_tungstenite::tungstenite::client::IntoClientRequest::into_client_request(
url.to_string(),
)
.map_err(|e| format!("bad ws url: {e}"))?;
for (hn, v) in &hdrs {
builder
.headers_mut()
.insert(hn.clone(), v.parse().map_err(|e| format!("bad header value: {e}"))?);
}
let rt = tokio::runtime::Runtime::new().map_err(|e| e.to_string())?;
rt.block_on(async move {
let timeout = tokio::time::Duration::from_secs(timeout_sec.max(1));
let fut = tokio_tungstenite::connect_async(builder);
let (mut socket, _resp) = tokio::time::timeout(timeout, fut)
.await
.map_err(|_| "ws connect timeout".to_string())?
.map_err(|e| format!("ws connect: {e}"))?;
let send = tokio_tungstenite::tungstenite::Message::Text(message.to_string());
socket
.send(send)
.await
.map_err(|e| format!("ws send: {e}"))?;
let mut messages = Vec::new();
loop {
match tokio::time::timeout(timeout, socket.next()).await {
Ok(Some(Ok(tokio_tungstenite::tungstenite::Message::Text(t)))) => {
messages.push(t.to_string());
}
Ok(Some(Ok(tokio_tungstenite::tungstenite::Message::Close(_))))
| Ok(Some(Err(_))) => break,
Ok(None) => break,
Ok(Some(Ok(_))) => {}
Err(_) => break,
}
}
Ok(json!({ "ok": true, "messages": messages }))
})
}
#[cfg(test)]
mod tests {
use super::*;
use tokio::net::TcpListener;
use tokio_tungstenite::tungstenite::Message;
#[test]
fn echo_roundtrip() {
let rt = tokio::runtime::Runtime::new().expect("rt");
let listener = rt.block_on(async { TcpListener::bind("127.0.0.1:0").await.expect("bind") });
let addr = listener.local_addr().expect("addr");
let server = std::thread::spawn(move || {
let rt = tokio::runtime::Runtime::new().expect("server rt");
rt.block_on(async move {
let (stream, _) = listener.accept().await.expect("accept");
let ws = tokio_tungstenite::accept_async(stream).await.expect("ws");
let (mut sink, mut source) = ws.split();
if let Some(Ok(Message::Text(t))) = source.next().await {
let _ = sink.send(Message::Text(t)).await;
}
});
});
let url = format!("ws://{addr}/");
let got = connect(&url, "ping", None, 5).expect("connect");
assert_eq!(got["ok"], true);
assert_eq!(got["messages"], json!(["ping"]));
server.join().expect("server join");
}
#[test]
fn refuses_bad_url() {
let err = connect("http://127.0.0.1:1", "hi", None, 1).unwrap_err();
assert!(err.contains("ws://") || err.contains("ws url"));
}
#[test]
fn refused_connection() {
// Connect to a port that is closed; connect_async fails fast.
let rt = tokio::runtime::Runtime::new().expect("rt");
let listener = rt.block_on(async { TcpListener::bind("127.0.0.1:0").await.expect("bind") });
let addr = listener.local_addr().expect("addr");
drop(listener); // port now closed
let url = format!("ws://{addr}/");
let err = connect(&url, "hi", None, 2).unwrap_err();
assert!(
err.starts_with("ws connect") || err.contains("timeout"),
"unexpected error: {err}"
);
}
}
-92
View File
@@ -1,92 +0,0 @@
//! Per-path WebSocket broadcast hubs (W6 remainder).
use std::collections::HashMap;
use std::sync::Mutex;
use tokio::sync::broadcast;
const CAP: usize = 256;
static HUBS: Mutex<Option<HashMap<String, broadcast::Sender<String>>>> = Mutex::new(None);
fn with_hubs<T>(f: impl FnOnce(&mut HashMap<String, broadcast::Sender<String>>) -> T) -> T {
let mut g = HUBS.lock().unwrap_or_else(|e| e.into_inner());
let m = g.get_or_insert_with(HashMap::new);
f(m)
}
/// Subscribe to (or create) the broadcast channel for `path`.
pub fn subscribe(path: &str) -> broadcast::Receiver<String> {
with_hubs(|m| {
if let Some(tx) = m.get(path) {
return tx.subscribe();
}
let (tx, rx) = broadcast::channel(CAP);
m.insert(path.to_string(), tx);
rx
})
}
/// Publish a text frame to all subscribers on `path`.
pub fn publish(path: &str, text: String) {
with_hubs(|m| {
let tx = m.entry(path.to_string()).or_insert_with(|| {
let (tx, _) = broadcast::channel(CAP);
tx
});
let _ = tx.send(text);
});
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum WsMode {
Echo,
Broadcast,
Drain,
}
impl WsMode {
pub fn parse(v: &serde_json::Value) -> Self {
match v {
serde_json::Value::Bool(true) => WsMode::Echo,
serde_json::Value::Bool(false) => WsMode::Drain,
serde_json::Value::String(s) => match s.trim().to_ascii_lowercase().as_str() {
"broadcast" | "广播" => WsMode::Broadcast,
"drain" | "false" | "0" | "no" | "drain_only" => WsMode::Drain,
_ => WsMode::Echo,
},
serde_json::Value::Object(m) => {
if let Some(mode) = m.get("mode").or_else(|| m.get("模式")) {
return Self::parse(mode);
}
if let Some(echo) = m.get("echo").or_else(|| m.get("回显")) {
return Self::parse(echo);
}
WsMode::Echo
}
_ => WsMode::Echo,
}
}
pub fn as_str(self) -> &'static str {
match self {
WsMode::Echo => "echo",
WsMode::Broadcast => "broadcast",
WsMode::Drain => "drain",
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn broadcast_fanout() {
let mut a = subscribe("/t-fanout");
let mut b = subscribe("/t-fanout");
publish("/t-fanout", "hi".into());
assert_eq!(a.try_recv().unwrap(), "hi");
assert_eq!(b.try_recv().unwrap(), "hi");
}
}
+3 -1
View File
@@ -24,4 +24,6 @@ Override path: `export MARQDO_WEB_PLUGIN=$PWD/plugins/web/build/libweb.so`
| `abi/marqdo_abi_cgo.h` | C contract for cgo |
| `internal/*` | page / db / http / ws / … |
Rust reference (not built by default): `plugins/web-rust-archive/`.
The legacy Rust implementation was archived during v1.6 and then removed from the
working tree (it remains in git history before commit `<this commit>`); see
`CHANGELOG.md` “默认 libweb 为 Go”。