Compare commits
26
Commits
v1.1.0
..
ext-v1.3.2
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f6374d1593 | ||
|
|
9ede4bc63d | ||
|
|
3ae8f2c843 | ||
|
|
e9c1a8a730 | ||
|
|
5554b1642c | ||
|
|
fe15603a91 | ||
|
|
1418742655 | ||
|
|
ed31a2b31d | ||
|
|
56ab52356f | ||
|
|
2e425c3010 | ||
|
|
46a0263d7f | ||
|
|
ff252fd469 | ||
|
|
67bac1bf02 | ||
|
|
33f8b94bb0 | ||
|
|
0ed57b6d54 | ||
|
|
c29d48f38c | ||
|
|
f1e82ce4e5 | ||
|
|
2d412e3bba | ||
|
|
46d78c6e15 | ||
|
|
81a14f9b38 | ||
|
|
88dc6d5a54 | ||
|
|
4a2d89406b | ||
|
|
d8d435fead | ||
|
|
1f8e61f3bf | ||
|
|
a9ee858f53 | ||
|
|
8489d90926 |
No files matched your search
@@ -0,0 +1,26 @@
|
||||
---
|
||||
description: Marqdo Web Artifact — no degradation; Document/Endpoint/Resource only
|
||||
globs: ext/web/**,ext/net/**,ext/data/**,ext/security/**,examples/**/*,tests/ext/web*,doc/design/ext-web*,doc/adr/0007*
|
||||
alwaysApply: true
|
||||
---
|
||||
|
||||
# Web Artifact — 不允许降级
|
||||
|
||||
Canonical: `doc/adr/0007-web-document-endpoint.md` · `doc/design/ext-web-artifact.md` · `doc/next/007.md` (superseded).
|
||||
|
||||
## Hard bans
|
||||
|
||||
1. **Do not** restore or reintroduce author-facing `compose_main` / `compose_intro` / `compose_*` fragmented table DSLs, god `# page` mutation piles, or `app.configure(...)` junk-drawer APIs.
|
||||
2. **Do not** use `json.set` / `json.append` bag glue for pages, routes, or middleware. Tables = data. View schema is only `|type|slot|value|attrs|style|` (ZH aliases allowed).
|
||||
3. **Do not** call `ensure_plugin` or `host_*` from author programs or from `ext/**` (plugin load is facade/capability only).
|
||||
4. **Do not** register implicit user routes. Declared routes come from Artifact metadata or `web.route`. System routes (`/_part`, `/_form`, …) must appear in `web.inspect`.
|
||||
5. **Do not** invent DSL, nested frontmatter under `##`, or new table dialects when the design is silent — **stop and ask**.
|
||||
6. **Do not** mix EN/ZH API names in one `.mq.md`.
|
||||
7. **Do not** silently delete capabilities. Removals require design text + tests updated in the same change.
|
||||
8. **Do not** put DB/auth/cache/storage/websocket author APIs back into `ext/web/web.mq.md` — they live in `ext/data`, `ext/security`, `ext/net`.
|
||||
|
||||
## Required direction
|
||||
|
||||
- Document (`type: web` / `类型: 网页`), Endpoint (`type: endpoint` / `类型: 端点`), Resource modules.
|
||||
- Facade only: `web.page` · `web.route` · `web.serve` · `web.render` · `web.inspect` (ZH mirrors).
|
||||
- Code is documentation: the `.mq.md` is the page/API docs and the implementation.
|
||||
@@ -92,7 +92,7 @@ Better still — call a named helper:
|
||||
7. **Do not use `##` inside `# main` for document sectioning** — those become nested functions. Use paragraphs / `---` breaks instead.
|
||||
8. **Index:** prefer `[键](集合)` / `` [`名`](集合) `` over footnote `` m[^k] ``.
|
||||
|
||||
Browser client programs: prefer `import browser:lib/browser.mq.md` (no native plugin). Server sites: `ext/web` + GFM page/style tables.
|
||||
Browser client programs: prefer `import browser:lib/browser.mq.md` (no native plugin). Server sites: Artifact Metadata (`type: web|endpoint`) + `ext/web` facade + `ext/data` / `ext/security` / `ext/net` — see [ext-web-artifact.md](../../doc/design/ext-web-artifact.md). **Never** restore `compose_*` / `app.configure`.
|
||||
|
||||
## Developing libraries (`lib/*`, `ext/*`)
|
||||
|
||||
@@ -121,7 +121,9 @@ Canonical thin-template + waves: [lib-code-as-docs.md](../../doc/design/lib-code
|
||||
| Need | Place |
|
||||
|------|--------|
|
||||
| Browser effect maps (WASM client) | `lib/browser.mq.md` |
|
||||
| Site HTML / HTTP / DB | `ext/web` classes + compose from tables |
|
||||
| Site HTML / HTTP | `ext/web` Document/Endpoint + View tables |
|
||||
| DB / cache / storage | `ext/data` |
|
||||
| Auth / OIDC / RBAC | `ext/security` |
|
||||
| List/map primitives | `lib/table.mq.md` |
|
||||
| Parse JSON text | `lib/json.mq.md` only |
|
||||
|
||||
|
||||
@@ -201,7 +201,7 @@ import clock:lib/time.mq.md
|
||||
- Instance methods stay `` > `obj`.method `` (backticks on the receiver only).
|
||||
- `lib/…` resolves via `MARQDO_LIB`, cwd `lib/`, or `lib/` next to the `marqdo` binary.
|
||||
- Design: [module-namespace.md](../../doc/design/module-namespace.md).
|
||||
- Official optional extensions (`ext/`, not stdlib): install with `marqdo ext add llm|agent|web|quantum|linalg` (see `doc/design/ext-cli.md`). `ext/llm` — chat; **`ext/agent`** — document-driven agents (see **ext/agent** below); **`ext/web`** — dynamic sites (see below); **`ext/quantum`** — circuits + Q7/Q8 (see below); **`ext/linalg`** — formula-first linear algebra (see below).
|
||||
- Official optional extensions (`ext/`, not stdlib): install with `marqdo ext add llm|agent|web|quantum|linalg` (see `doc/design/ext-cli.md`). **`ext/llm`** — intelligence primitive (`ask` / `stream` / `collect`; `complete` is compat); **`ext/agent`** — document-driven agents (see **ext/agent** below); **`ext/web`** — dynamic sites (see below); **`ext/quantum`** — circuits + Q7/Q8 (see below); **`ext/linalg`** — formula-first linear algebra (see below).
|
||||
- Native plugins: `lib/plugin` — `## load` / `unload` / `list`. C ABI: `include/marqdo_abi.h`.
|
||||
- Prefer `table.put` / `表.改` for list/map element updates; keep `json` for parse/stringify/quote (see `doc/design/stdlib-table.md`).
|
||||
- Builtins (no import): `print`/`打印`, `input`/`输入`, `len`/`长度`, `str`/`文本`, `int`/`整数`; literals `True`/`真`, `False`/`假`, `None`/`空`; logic `and`/`且`, `or`/`或`, `not`/`非`.
|
||||
@@ -216,50 +216,61 @@ Install: `marqdo ext add agent` (ZH: `智能体`). Native: `cargo build --releas
|
||||
|
||||
1. **Code is documentation** — the runbook `.mq.md` is the model’s ground truth, not a hidden system prompt bag.
|
||||
2. **Call site is first-class** — `build_step_context` injects path/function/line so the model knows where it is in the program.
|
||||
3. **Document is the knowledge base (OKF)** — writeback → promote/solidify → reuse / `llm_free`; MCP/corpus are evidence only.
|
||||
3. **Document is the knowledge base (OKF)** — writeback → promote/solidify → reuse / `llm_free`; MCP/corpus are evidence only. **Engineering Knowledge(EKC)**:`marqdo knowledge|find|reuse|…` + `agent.preflight`(REUSE→ADAPT→CREATE)——见 [engineering-knowledge.md](../../doc/design/engineering-knowledge.md)。
|
||||
4. **Skill Compilation (v2)** — successful runs become `.mq.md` episodes; after enough evidence, `maybe_learn` compiles an executable skill (`llm_free`). Prefer `## run` over bare `## plan` when you want learning.
|
||||
5. **Adaptive Routing (P4)** — router is abstract (`marqdo` / `small-llm` / `llm` / `auto`); `jev` is a compat alias of `small-llm`. Decision via `router_model` (any llm handle). Policy Compilation writes `policies/router.mq.md`.
|
||||
|
||||
APIs: `# agent` / `# 智能体` → `## step` / `## plan`. Context: standing + task + **call site** + tools + budgeted source/skill + act protocol (`CALL:` / `READ:`). Inspect without LLM: `dump_step_context`. Inside `writeback=` / `写回=` branches use `host_writeback_record` (param name shadows the writeback import).
|
||||
APIs: `# agent` / `# 智能体` → `## run` / `## route` / `## step` / `## plan` (+ `resolve` / `record_episode` / `maybe_learn` / `compile` / `compile_policy` / `metrics`). Context: standing + task + **call site** + tools + budgeted source/skill + act protocol (`CALL:` / `READ:`). Inspect without LLM: `dump_step_context`. Inside `writeback=` / `写回=` branches use `host_writeback_record` (param name shadows the writeback import).
|
||||
|
||||
Examples: [agent-pong](../../examples/agent-pong/) · [agent-okf-flywheel](../../examples/agent-okf-flywheel/). Harness: `scripts/agent-harness.sh`. Design: [ext-agent.md](../../doc/design/ext-agent.md) · Wave B: [agent-framework-2026-09.md](../../doc/research/agent-framework-2026-09.md). Dev LLM: copy [.env.example](../../.env.example) → `.env` (gitignored).
|
||||
Examples: [agent-pong](../../examples/agent-pong/) · [agent-okf-flywheel](../../examples/agent-okf-flywheel/). Harness: `scripts/agent-harness.sh`. Design: [ext-agent.md](../../doc/design/ext-agent.md) · Wave B: [agent-framework-2026-09.md](../../doc/research/agent-framework-2026-09.md) · v2: [doc/next/003.md](../../doc/next/003.md) · P4: [doc/next/004.md](../../doc/next/004.md). Dev LLM: copy [.env.example](../../.env.example) → `.env` (gitignored).
|
||||
|
||||
## Official extension: `ext/web` (dynamic sites)
|
||||
## Official extension: `ext/web` (Document / Endpoint / Resource)
|
||||
|
||||
Install: `marqdo ext add web` (ZH id: `网页`). Build native plugin first: `cargo build --release -p marqdo_plugin_web`.
|
||||
Install: `marqdo ext add web` (ZH id: `网页`). Build native plugin: `scripts/build-web-plugin.sh` / `marqdo ext add web`.
|
||||
|
||||
Import **one language file** — never mix EN/ZH API names in the same `.mq.md`:
|
||||
**Constitution (ADR 0007 — do not degrade):**
|
||||
|
||||
- EN: `import web:ext/web/web.mq.md`
|
||||
1. **Document / Endpoint / Resource** — `.mq.md` with `type: web|endpoint` (ZH `类型: 网页|端点`) is the site surface; not a Flask-style class pile.
|
||||
2. **Code is documentation** — the file is docs + route + handler.
|
||||
3. **Table = data** — View schema only `|type|slot|value|attrs|style|`. No `compose_*` dialects, no `json.set` glue.
|
||||
4. **Facade only** — `web.page` · `web.route` · `web.serve` · `web.render` · `web.inspect`. DB → `ext/data`; auth → `ext/security`; ws/http → `ext/net`.
|
||||
5. **No author `ensure_plugin` / `host_*`**; import web = capability (plugin auto-load).
|
||||
6. **Declared vs system routes**; `web.inspect` lists both. No silent user routes.
|
||||
|
||||
Import **one language** per file:
|
||||
|
||||
- EN: `import web:ext/web/web.mq.md` (+ `import data:ext/data/db.mq.md` as needed)
|
||||
- ZH: `导入 网页:ext/web/网页.mq.md`
|
||||
|
||||
**Hard rules (web):**
|
||||
```markdown
|
||||
---
|
||||
type: web
|
||||
route: /
|
||||
title: Hello
|
||||
import web:ext/web/web.mq.md
|
||||
---
|
||||
|
||||
- Authors use **GFM tables + `#` classes** — no `json.parse` / `json.set` glue, no hand-built part JSON.
|
||||
- Table cells stay **literal strings**; path text like `` `posts`.`title` `` is resolved by web classes.
|
||||
- **Quote path/MIME cells** that contain `/` (T5 cell expressions treat bare `/` as division): `"/static/logo.svg"`, `"image/png"`.
|
||||
- **`ext/**` never calls `host_*`** — hot path is native `plugins/web` ABI (default **Go `libweb`**, build via `scripts/build-web-plugin.sh`).
|
||||
- HTTPS: terminate at reverse proxy; set `cookie_secure=True` on auth (no in-process TLS).
|
||||
# Hello
|
||||
|
||||
**Typical layout:**
|
||||
|
||||
```text
|
||||
index.mq.md # entry + home page table + listen
|
||||
pages/ # sub-pages
|
||||
components/ # reusable |属性|值|样式| tables
|
||||
styles/ # CSS modules
|
||||
public/ # static + favicon.ico|png|svg
|
||||
db/ # schema + open/init
|
||||
data/ # sqlite runtime (gitignore)
|
||||
Executable document as a page.
|
||||
```
|
||||
|
||||
**Core objects (EN / ZH):** `# page`/`页面`, `# style`/`样式`, `# db`/`数据库`, `# form`/`表单`, `# app`/`应用`, `# auth`/`鉴权`, `# cache`/`缓存`, `# storage`/`存储`.
|
||||
```markdown
|
||||
---
|
||||
type: endpoint
|
||||
method: POST
|
||||
path: /api/ping
|
||||
response: json
|
||||
---
|
||||
|
||||
**Typical flow:** `db.init` → `page.compose_*` → `app` + `static` / `listen`. Client: `marqdo wasm build -o static` + `web.client_embed source="/static/client.mq.md"` — **no author JS**; bridge may implement rich host effects ([browser-wasm-e.md](../../doc/roadmap/browser-wasm-e.md) · [browser-app](../../examples/browser-app/)).
|
||||
*{"ok": true}*
|
||||
```
|
||||
|
||||
**Shipped surface (W0–W7 + P3 + W8 + route D mount):** middleware + JSON API (`app.configure`); transactions, pagination, FTS search (`db.migrate`, `db.fts`, `db.search`); security (argon2, CSRF, SQLite sessions, login rate limit); SEO / RSS / Markdown (`page.meta`, `route_rss`, `lib/net.markdown_parse`); upload / download / gallery; sitemap / robots / error pages / redirects; RBAC (`app.gate`, user `role`); audit timestamps + FK in `db.init`; ETag on downloads; **W8** site icons (`app.icons` → `/favicon.ico`), head resource tables (`page.head`), image assembly (`make_images` / `page.images`); **表驱动引言** `page.compose_intro` / `页面.引言装配`(属性/值/样式表 → `.main-intro`,避免手写 HTML 字符串);**D** `client_embed` auto-mount + DOM effect helpers (`text_patch` / `dom_patch`).
|
||||
Site entry may `# main` → `web.serve root="."`. Client WASM: `ext/web/client.mq.md` / `lib/browser` — no author JS.
|
||||
|
||||
Design: [ext-web.md](../../doc/design/ext-web.md) · assets: [web-assets-and-images.md](../../doc/design/web-assets-and-images.md) · capability matrix: [web-net-capabilities.md](../../doc/design/web-net-capabilities.md) · example: [examples/marqdo-blog/](../../examples/marqdo-blog/) · client: [examples/web-client-site/](../../examples/web-client-site/).
|
||||
Design: [ext-web-artifact.md](../../doc/design/ext-web-artifact.md) · ADR [0007](../../doc/adr/0007-web-document-endpoint.md) · rule: `.cursor/rules/web-artifact-no-degrade.mdc` · example: [examples/marqdo-blog/](../../examples/marqdo-blog/).
|
||||
|
||||
Web patterns: [examples.md](examples.md) §13 · API index: [reference.md](reference.md).
|
||||
Web patterns: [examples.md](examples.md) · API index: [reference.md](reference.md).
|
||||
|
||||
## Official extension: `ext/quantum` (circuits + Q7/Q8)
|
||||
|
||||
@@ -323,7 +334,7 @@ Examples: [linalg-svd](../../examples/linalg-svd/) · [linalg-least-squares](../
|
||||
| Import `lib/text` then call bare `split` | `> text.split …` (qualified) |
|
||||
| Import `lib/text` then call `拆分` | Match file language (`text.split`, not 文本) |
|
||||
| `json.set` / `json.append` to build maps or lists | GFM tables; sparse `table.put`; named helpers (`browser.*`, `web.*`) |
|
||||
| `json.set` to build page parts | GFM tables + `page.compose_*` / `web.page` methods |
|
||||
| `json.set` / `compose_*` page parts | `type: web` Document + View table / `web.render` / `web.serve` |
|
||||
| Import `json` for every browser handler | `import browser:lib/browser.mq.md` + tables ([marqdo-dev](../marqdo-dev/SKILL.md)) |
|
||||
| Mix `web.page` and `网页.页面` in one file | One import language per `.mq.md` |
|
||||
| Call `host_web_*` from `ext/web` | Use `# app` / `# db` methods; plugin ABI only |
|
||||
|
||||
@@ -38,7 +38,7 @@ jobs:
|
||||
run: |
|
||||
export PATH="/c/ProgramData/mingw64/mingw64/bin:/c/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/bin:$PATH"
|
||||
cargo build --release
|
||||
cargo build --release -p marqdo_plugin_agent -p marqdo_plugin_quantum -p marqdo_plugin_linalg
|
||||
cargo build --release -p marqdo_plugin_agent -p marqdo_plugin_quantum -p marqdo_plugin_linalg -p marqdo_plugin_llm
|
||||
bash ./scripts/build-web-plugin.sh
|
||||
|
||||
- name: Checkout extension sources (branch vscode-extension only)
|
||||
@@ -87,6 +87,7 @@ jobs:
|
||||
Copy-Item target/release/agent.dll "$stageNative/native/agent.dll"
|
||||
Copy-Item target/release/quantum.dll "$stageNative/native/quantum.dll"
|
||||
Copy-Item target/release/linalg.dll "$stageNative/native/linalg.dll"
|
||||
Copy-Item target/release/llm.dll "$stageNative/native/llm.dll"
|
||||
if (Test-Path "dist/$nativeZip") { Remove-Item "dist/$nativeZip" -Force }
|
||||
Compress-Archive -Path "$stageNative/*" -DestinationPath "dist/$nativeZip" -Force
|
||||
$stage = "dist/stage-bundle"
|
||||
@@ -100,6 +101,7 @@ jobs:
|
||||
Copy-Item target/release/agent.dll "$stage/ext/native/agent.dll"
|
||||
Copy-Item target/release/quantum.dll "$stage/ext/native/quantum.dll"
|
||||
Copy-Item target/release/linalg.dll "$stage/ext/native/linalg.dll"
|
||||
Copy-Item target/release/llm.dll "$stage/ext/native/llm.dll"
|
||||
if (Test-Path "dist/$zip") { Remove-Item "dist/$zip" -Force }
|
||||
Compress-Archive -Path "$stage/*" -DestinationPath "dist/$zip" -Force
|
||||
$stageLib = "dist/stage-stdlib"
|
||||
@@ -124,12 +126,14 @@ jobs:
|
||||
## Marqdo ${{ github.ref_name }}
|
||||
|
||||
### 亮点 / Highlights
|
||||
- **v1.1.0**:AI 原生闭环三件套——核心表面守卫(问题一)· 渐进式契约 + `marqdo check`(问题二)· **MLSP for AI**(`marqdo mlsp`:locate/syntax/validate/repair_targets/repair_apply/schema,AI 按需查语法而非背诵)(问题三)· `run --json` 结构化诊断
|
||||
- **v1.1.0**: AI-native loop — core-surface guard, progressive contracts + `marqdo check`, **MLSP for AI** (`marqdo mlsp`: query syntax on demand instead of memorizing), `run --json` structured diagnostics
|
||||
- **v1.0.7**:WS/网络边界加固(同源握手、require_auth/需要登录、消息上限、HTTP 8 MiB 截断);`url_encode` `%20`
|
||||
- **v1.0.3**:Ubuntu PPA + 扩展 CDN(ext.marqdo.com);CLI/扩展可分发;noble 捆绑 Rust 工具链
|
||||
- **v1.0.3**: Ubuntu PPA + extension CDN; independent CLI/ext packs; bundled Rust for noble
|
||||
- **v1.0.1**:修复 Go `libweb` 动态路由 `{param}` 查询代入、详情 Markdown、CSS 数字误报
|
||||
- **v1.3.0**:Engineering Knowledge Compiler——L0–L4 · `reuse`/`find`/`duplicate` · Agent preflight(REUSE→ADAPT→CREATE)
|
||||
- **v1.3.0**: Engineering Knowledge Compiler — L0–L4 catalog, reuse/find/duplicate, Agent preflight (REUSE→ADAPT→CREATE)
|
||||
- **v1.2.0**:Web Artifact(ADR 0007)——Document/Endpoint · `web.serve`/`render`/`inspect` · `ext/data|security|net` · 移除 `compose_*`/`configure`
|
||||
- **v1.2.0**: Web Artifact (ADR 0007) — Document/Endpoint, web.serve/render/inspect, ext/data|security|net, compose_*/configure removed
|
||||
- **v1.1.1**:AI 栈收口——llm `ask`/`stream`/`collect` · Metadata Binding · Skill Compilation + Adaptive Routing · MCP 客户端 + resume · 可选 `plugins/llm`
|
||||
- **v1.1.1**: AI stack — llm ask/stream/collect, Metadata Binding, Skill Compilation + Adaptive Routing, MCP client + resume, optional plugins/llm
|
||||
- **v1.0.7**:WS/网络边界加固;`url_encode` `%20`
|
||||
- **v1.0.3**:Ubuntu PPA + 扩展 CDN(ext.marqdo.com)
|
||||
- **Linux 包必出**:`*-native-x86_64-unknown-linux-gnu.zip` + Linux CLI bundle
|
||||
- Details: [CHANGELOG](https://github.com/cflmy/marqdo/blob/main/CHANGELOG.md) · [mapping v0.3](https://github.com/cflmy/marqdo/blob/main/doc/design/markdown-mapping-v0.3.md)
|
||||
|
||||
@@ -139,7 +143,7 @@ jobs:
|
||||
|-------|----------|
|
||||
| `marqdo-*-windows-msvc.exe` | CLI with **embedded** official stdlib |
|
||||
| `marqdo-*-windows-msvc.zip` | `marqdo.exe` + `lib/` + `ext/`(含 native DLL) |
|
||||
| `marqdo-*-native-*.zip` | 预编译 `web`/`agent`/`quantum`/`linalg`(Windows DLL + **Linux .so 必出**) |
|
||||
| `marqdo-*-native-*.zip` | 预编译 `web`/`agent`/`quantum`/`linalg`/`llm`(Windows DLL + **Linux .so 必出**) |
|
||||
| `marqdo-*-unknown-linux-gnu.zip` | Linux CLI + `lib/` + `ext/` + native `.so` |
|
||||
| `marqdo-*-stdlib.zip` | `lib/` only |
|
||||
| `marqdo-*-ext.zip` | Official `ext/` L1 sources |
|
||||
@@ -184,7 +188,7 @@ jobs:
|
||||
run: |
|
||||
set -euo pipefail
|
||||
cargo build --release
|
||||
cargo build --release -p marqdo_plugin_agent -p marqdo_plugin_quantum -p marqdo_plugin_linalg
|
||||
cargo build --release -p marqdo_plugin_agent -p marqdo_plugin_quantum -p marqdo_plugin_linalg -p marqdo_plugin_llm
|
||||
bash ./scripts/build-web-plugin.sh
|
||||
|
||||
- name: Package Linux native zip + CLI bundle
|
||||
@@ -193,7 +197,7 @@ jobs:
|
||||
VER="${GITHUB_REF_NAME#v}"
|
||||
TARGET="x86_64-unknown-linux-gnu"
|
||||
mkdir -p dist/stage-native/native
|
||||
for lib in libweb.so libagent.so libquantum.so liblinalg.so; do
|
||||
for lib in libweb.so libagent.so libquantum.so liblinalg.so libllm.so; do
|
||||
src="target/release/${lib}"
|
||||
if [[ ! -f "$src" && "$lib" == libweb.so ]]; then
|
||||
src="plugins/web/build/libweb.so"
|
||||
|
||||
+147
@@ -1,5 +1,152 @@
|
||||
# Changelog
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Added
|
||||
|
||||
### Fixed
|
||||
|
||||
### Changed
|
||||
|
||||
## Ext pack v1.3.2 — 2026-10-13
|
||||
|
||||
### Highlights
|
||||
|
||||
**Trailing slash 路由修复(CLI 仍为 v1.3.0)**:Go `ServeMux` 精确匹配导致 `/post/foo/`(带尾部斜杠)落入 404。新增 `withStripTrailingSlash` 中间件作为最外层 handler wrapper,所有下游中间件(RBAC、Tenant、路由)均看到规范化路径。覆盖静态路由、动态路由、根路径、查询参数保留。
|
||||
|
||||
```bash
|
||||
marqdo ext add web # CDN latest → 1.3.2
|
||||
# 或:MARQDO_EXT_VERSION=1.3.2 marqdo ext add web
|
||||
```
|
||||
|
||||
### Fixed
|
||||
|
||||
- **web**: `/post/foo/` trailing slash now routes the same as `/post/foo` (dynamic and static routes). Root `/` unchanged. Query strings preserved after normalization.
|
||||
|
||||
## Ext pack v1.3.1 — 2026-10-08
|
||||
|
||||
### Highlights
|
||||
|
||||
**OIDC 路径型 `redirect_uri`(CLI 仍为 v1.3.0)**:`redirect_uri=/oidc/callback` 按请求 Host / `X-Forwarded-*` 拼绝对回调,写入会话供 token 交换;可选 `redirect_origins` allowlist。apex / www 多域名无需写死单一绝对 URI。
|
||||
|
||||
```bash
|
||||
marqdo ext add web # CDN latest → 1.3.1
|
||||
# 或:MARQDO_EXT_VERSION=1.3.1 marqdo ext add web
|
||||
```
|
||||
|
||||
### Added
|
||||
- **路径型 redirect_uri**:绝对 URL 行为不变;路径模式解析 public origin,session 键 `oidc_redirect_uri`。
|
||||
- **`redirect_origins` / `回调来源`**:逗号分隔 origin allowlist(路径模式)。
|
||||
- **ABI**:`web_app_oidc` 透传 `redirect_origins`;`ext/security/oidc.attach` 文档同步。
|
||||
|
||||
### Changed
|
||||
- 扩展包 SemVer **1.3.1**(`ext/VERSION`);与 CLI **v1.3.0** 可独立安装。
|
||||
|
||||
## v1.3.0 — 2026-09-28
|
||||
|
||||
### Highlights
|
||||
|
||||
**Engineering Knowledge Compiler(EKC)**:把仓库投影为 AI 可查询的工程知识系统——L0–L4 Catalog、Capability / Decision / Constraint / Failure、知识图、Reuse Preflight(REUSE→ADAPT→CREATE)与 Context Pack。本版 CLI 与扩展包 **同步为 1.3.0**。
|
||||
|
||||
- **杀手命令**:`marqdo reuse "task"` / `find` / `duplicate` / `impact` / `conflicts` / `stale` / `verify` / `knowledge`
|
||||
- **Agent**:`agent.preflight` · `agent_eng_*`;`plan … eng_preflight=True` 在造轮子前强制 EFI
|
||||
- **投影**:`.marqdo/catalog|knowledge|graph|agent/` + `engineering.yaml`;`catalog` 默认挂钩 EKC
|
||||
- **示例 / 金样**:`examples/engineering-knowledge/` · `tests/engineering/` · `tests/ext/eng-preflight.mq.md`
|
||||
|
||||
```bash
|
||||
sudo add-apt-repository ppa:cflmy/marqdo && sudo apt update && sudo apt install marqdo
|
||||
git checkout v1.3.0 && cargo build --release
|
||||
marqdo knowledge . -o .marqdo
|
||||
marqdo reuse "resolve configuration"
|
||||
marqdo ext add agent
|
||||
```
|
||||
|
||||
**核心表面 Δ(Core surface Δ)**:**无增删**——19 个核心构造不变。本版为 **Runtime / Tooling**(知识编译器 + CLI + agent 插件),不触碰语言标记。
|
||||
|
||||
### Added
|
||||
- **`src/knowledge/` EKC**:L0–L4 IR、指纹、能力推导、图边、compile、find/reuse/preflight、impact/lifecycle/duplicate、metrics、policy、learn。
|
||||
- **CLI**:`knowledge` · `find` · `reuse` · `impact` · `conflicts` · `stale` · `duplicate` · `verify`;`catalog`/`sync` 默认编译 EKC(`--no-knowledge` 可关)。
|
||||
- **Agent**:`agent_eng_preflight` / `agent_eng_reuse` / `agent_eng_record`;`ext/ai` `## preflight` + `plan eng_preflight`。
|
||||
- **规范**:`doc/design/engineering-knowledge.md`;提案叙事 `doc/next/008.md`(已实现标注)。
|
||||
|
||||
### Fixed
|
||||
|
||||
### Changed
|
||||
- **OKF type 注册表**扩展 Capability/Decision/Constraint/Failure/Context Pack/Policy 等([okf.md](doc/design/okf.md))。
|
||||
- **核心表面 Δ**:**无增删**。
|
||||
|
||||
## v1.2.0 — 2026-09-28
|
||||
|
||||
### Highlights
|
||||
|
||||
**Web Artifact 重写(ADR 0007)**:Document / Endpoint / Resource 一等公民;`type: web|endpoint`(ZH `类型: 网页|端点`);facade `web.page|route|serve|render|inspect|use`;DB/auth/ws 拆到 `ext/data` · `ext/security` · `ext/net`。本版 CLI 与扩展包 **同步为 1.2.0**。
|
||||
|
||||
- **Document**:文件级 Artifact + `data_source` / `data_order` / `data_where`;正文 Markdown → HTML;列表卡片 / 详情文章由 serve 时加载。
|
||||
- **Endpoint**:`# main` 处理器;请求经 `run_artifact` Binding 注入;`response` 序列化 JSON/text/html。
|
||||
- **破坏性**:删除作者面 `compose_*` / `app.configure` / 作者 `ensure_plugin`;旧 web gold 移除,新金样 `ext_web_*`。
|
||||
- **示例**:`examples/marqdo-blog/` Artifact 站(pages/api/db + smoke);`web-client-site` serve 扫描。
|
||||
|
||||
```bash
|
||||
sudo add-apt-repository ppa:cflmy/marqdo && sudo apt update && sudo apt install marqdo
|
||||
git checkout v1.2.0 && cargo build --release
|
||||
bash scripts/build-web-plugin.sh && marqdo ext add web
|
||||
marqdo run examples/marqdo-blog/serve.mq.md
|
||||
```
|
||||
|
||||
**核心表面 Δ(Core surface Δ)**:**无增删**——复用 Metadata `type`;装配语义在 ext + plugin。语言标记不变。
|
||||
|
||||
### Added
|
||||
- **Web Artifact 模型(ADR 0007)**:`type: web|endpoint`(ZH `类型: 网页|端点`);核心 `src/artifact/`;`sys.artifact_kind`;无 `# main` 时 Result 摘要。
|
||||
- **`ext/web` facade**:`page` · `route` · `serve` · `render` · `inspect` · `use`;子模块 page/route/dom/client/component。
|
||||
- **`ext/data` · `ext/security` · `ext/net`**:从 monolithic web 拆出 Resource。
|
||||
- **统一 View** `|type|slot|value|attrs|style|` + ABI `web_render_nodes` / `web_serve_root` / `web_inspect` / `web_app_use`。
|
||||
- **规则**:`.cursor/rules/web-artifact-no-degrade.mdc`;设计 `doc/design/ext-web-artifact.md`。
|
||||
|
||||
### Fixed
|
||||
- Document `data_source` / `data_where` now drive list cards and detail articles under `web.serve` (ADR 0007 flat metadata).
|
||||
|
||||
### Changed
|
||||
- **破坏性**:删除作者面 `compose_*` / `app.configure` / 作者 `ensure_plugin`。旧 web gold 已移除;新金样 `web-artifact-*` / `ext_web_*`。
|
||||
- **核心表面 Δ**:**无增删**(复用 Metadata `type`)。
|
||||
- 修订 `ext-web.md` C1;`doc/next/007.md` superseded。
|
||||
|
||||
## v1.1.1 — 2026-09-24
|
||||
|
||||
### Highlights
|
||||
|
||||
**AI 栈收口(agent + llm)**:语义 `ask`/`stream`/`collect`、Artifact Metadata Binding、Skill Compilation + Adaptive Routing、MCP 客户端与 resume 检查点、可选 `plugins/llm` Transport ABI。
|
||||
本版 CLI 与扩展包 **同步为 1.1.1**。
|
||||
|
||||
- **LLM 原语**:`ext/ai/llm` 从 OpenAI HTTP 客户端提升为 `ask`/`stream`/`collect` 智能原语(`complete` 保留兼容);`LLMResult.tool_calls`;可选原生 `llm_chat_completions`(无 `.so` 时回退 `lib/net`)。
|
||||
- **Metadata Binding**:frontmatter `${env|arg|sys|secret}` 加载期解析;Phase 2 将绑定键注入入口 env(正文禁止 `${…}`);`sys.meta` / `--bind`;llm ctor 优先 meta。
|
||||
- **Agent**:Skill Compilation(episode → llm_free skills)+ Adaptive Routing(`marqdo` / `small-llm` / `llm` / `auto`;`jev` 为 `small-llm` 兼容别名);MCP **stdio 客户端**;resume 检查点;零 LLM 金样路径。
|
||||
- **度量与后端**:`model_turn` usage、named handles(fast/reasoning)、`type:prompt` 加载、openai/ollama 分流。
|
||||
|
||||
```bash
|
||||
# Ubuntu PPA / 源码
|
||||
sudo add-apt-repository ppa:cflmy/marqdo && sudo apt update && sudo apt install marqdo
|
||||
git checkout v1.1.1 && cargo build --release
|
||||
marqdo ext add llm && marqdo ext add agent
|
||||
```
|
||||
|
||||
**核心表面 Δ(Core surface Δ)**:**无增删**——19 个核心构造不变。本版为 **Runtime**(Metadata Binding、LLM/agent 插件与扩展)变更,不触碰语言标记。
|
||||
|
||||
### Added
|
||||
- **`ext/ai/llm` 语义表面**:`ask` / `stream` / `collect`(`complete` 兼容);`LLMResult.tool_calls`(OpenAI-shaped `choices[0].message.tool_calls`);`sys.module_source` + 入口文档 `prompt_body` 离线路径。
|
||||
- **`plugins/llm` Transport ABI**(`llm_chat_completions`):`ext/ai/llm/openai` 优先原生插件,否则 `lib/net` HTTP/SSE。
|
||||
- **Artifact Metadata Binding**:frontmatter `${env|arg|sys|secret}` → `Module.metadata`;Secret 掩码;`sys.meta` / `--bind`;Phase 2 入口 env 注入(正文拒绝 `${…}`;字节码 prologue 同源)。
|
||||
- **Agent Skill Compilation + Adaptive Routing**:episode → `maybe_learn` → llm_free skills;路由后端 `marqdo` / `small-llm` / `llm` / `auto`(`jev` = `small-llm` 兼容别名,不再探测 `MARQDO_JEV`)。
|
||||
- **Agent MCP stdio 客户端**(`agent_mcp_client` / `mcp_connect` / `mcp_client_list` / `mcp_client_call`)+ Python mock 离线金样。
|
||||
- **Agent resume 检查点**(`.marqdo/agent-resume/<id>.json` — save/load/clear/list;`run` 可设 `resume_id=`)。
|
||||
- **AI 栈度量与后端**:`model_turn` usage、named handles(fast/reasoning)、`type:prompt` 加载、openai/ollama 分流、agent-zero-llm 金样。
|
||||
|
||||
### Fixed
|
||||
- **plan force 不再冲掉已有 `resources` workbook**(保留 llm_free 资源;金样 `agent-plan-preserve`)。
|
||||
- **compiled-skill Marqdo 语法**:llm_free await 不再误落到 plan+HTTP。
|
||||
|
||||
### Changed
|
||||
- Adaptive Routing:决策面为 `router_model`(任意 llm handle);`jev` 降为 `small-llm` 文档/兼容别名。
|
||||
|
||||
## v1.1.0 — 2026-09-23
|
||||
|
||||
### Highlights
|
||||
|
||||
Generated
+12
-2
@@ -572,7 +572,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
|
||||
|
||||
[[package]]
|
||||
name = "marqdo"
|
||||
version = "1.1.0"
|
||||
version = "1.3.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"assert_cmd",
|
||||
@@ -603,7 +603,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "marqdo-wasm"
|
||||
version = "1.1.0"
|
||||
version = "1.3.0"
|
||||
dependencies = [
|
||||
"marqdo",
|
||||
"serde_json",
|
||||
@@ -632,6 +632,16 @@ dependencies = [
|
||||
"serde_json",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "marqdo_plugin_llm"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"idna_adapter",
|
||||
"serde_json",
|
||||
"ureq",
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "marqdo_plugin_quantum"
|
||||
version = "0.1.0"
|
||||
|
||||
+2
-1
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "marqdo"
|
||||
version = "1.1.0"
|
||||
version = "1.3.0"
|
||||
edition = "2021"
|
||||
license = "Apache-2.0"
|
||||
description = "Marqdo — Markdown markers as an interpreted programming language"
|
||||
@@ -58,6 +58,7 @@ getrandom = { version = "=0.2.17", default-features = false, features = ["custom
|
||||
members = [
|
||||
"plugins/demo",
|
||||
"plugins/agent",
|
||||
"plugins/llm",
|
||||
"plugins/quantum",
|
||||
"plugins/linalg",
|
||||
"crates/marqdo-wasm",
|
||||
|
||||
@@ -6,7 +6,7 @@ Marqdo 把 Markdown **标记当作编程语法**:同一个 `.mq.md` 文件,
|
||||
|
||||
欢迎访问 [marqdo 官方网站](https://www.marqdo.com/) 阅读更多特性与可执行文档。
|
||||
|
||||
语法宪法:[doc/design/markdown-mapping.md](doc/design/markdown-mapping.md) · 对象:[objects.md](doc/design/objects.md) · 调用实参:[call-arguments.md](doc/design/call-arguments.md) · 用户站:[user-site.md](doc/design/user-site.md) · 浏览:[view.md](doc/design/view.md) · 调试:[view-debug.md](doc/design/view-debug.md) · OKF 清单:[catalog-cli.md](doc/design/catalog-cli.md) · VS Code 扩展:[vscode-extension.md](doc/design/vscode-extension.md)(分支 **`vscode-extension`**;提交约定 [vscode-extension-commit.md](doc/design/vscode-extension-commit.md)) · **AI Skill**:[skills/marqdo/](skills/marqdo/)(说明 [ai-skill.md](doc/design/ai-skill.md)) · 官方扩展:[ext-llm.md](doc/design/ext-llm.md)(`ext/llm`)· [ext-agent.md](doc/design/ext-agent.md)(智能体开发框架)· [ext-cli.md](doc/design/ext-cli.md)(`marqdo ext list/add/remove`) · 原生插件 ABI:[ext-abi.md](doc/design/ext-abi.md)(`lib/plugin`) · 金样例:[tests/](tests/) · 用户文档:[public/](public/) · 变更:[CHANGELOG.md](CHANGELOG.md)
|
||||
语法宪法:[doc/design/markdown-mapping.md](doc/design/markdown-mapping.md) · 对象:[objects.md](doc/design/objects.md) · 调用实参:[call-arguments.md](doc/design/call-arguments.md) · 用户站:[user-site.md](doc/design/user-site.md) · 浏览:[view.md](doc/design/view.md) · 调试:[view-debug.md](doc/design/view-debug.md) · OKF 清单:[catalog-cli.md](doc/design/catalog-cli.md) · Engineering Knowledge:[engineering-knowledge.md](doc/design/engineering-knowledge.md) · VS Code 扩展:[vscode-extension.md](doc/design/vscode-extension.md)(分支 **`vscode-extension`**;提交约定 [vscode-extension-commit.md](doc/design/vscode-extension-commit.md)) · **AI Skill**:[skills/marqdo/](skills/marqdo/)(说明 [ai-skill.md](doc/design/ai-skill.md)) · 官方扩展:[ext-llm.md](doc/design/ext-llm.md)(`ext/llm`)· [ext-agent.md](doc/design/ext-agent.md)(智能体开发框架)· [ext-cli.md](doc/design/ext-cli.md)(`marqdo ext list/add/remove`) · 原生插件 ABI:[ext-abi.md](doc/design/ext-abi.md)(`lib/plugin`) · 金样例:[tests/](tests/) · 用户文档:[public/](public/) · 变更:[CHANGELOG.md](CHANGELOG.md)
|
||||
|
||||
---
|
||||
|
||||
@@ -123,6 +123,13 @@ Marqdo 把 Markdown **标记当作编程语法**:同一个 `.mq.md` 文件,
|
||||
```bash
|
||||
marqdo catalog [PATH] -o OUT_DIR
|
||||
marqdo sync [PATH] -o OUT_DIR # catalog 的别名
|
||||
|
||||
# Engineering Knowledge Compiler(L0–L4 能力目录 / 复用门闩)
|
||||
marqdo knowledge [PATH] -o OUT_DIR
|
||||
marqdo find "authentication"
|
||||
marqdo reuse "implement oauth login"
|
||||
marqdo duplicate [--fail]
|
||||
marqdo verify
|
||||
```
|
||||
|
||||
| 参数 | 默认 | 说明 |
|
||||
@@ -130,7 +137,7 @@ marqdo sync [PATH] -o OUT_DIR # catalog 的别名
|
||||
| `PATH` | `.` | 工程根或含 `.mq.md` 的目录 |
|
||||
| `-o` / `--out` | `.marqdo` | 输出目录 |
|
||||
|
||||
产物示例:`catalog.yaml`、`index.md`、`modules/*.md`(`type: Marqdo Module` 等)。设计见 [generated-yaml-manifest.md](doc/design/generated-yaml-manifest.md) · 调研 [okf-and-marqdo.md](doc/research/okf-and-marqdo.md)。
|
||||
`catalog` 成功后默认再跑 EKC(`--no-knowledge` 可关)。产物示例:`catalog.yaml`、`engineering.yaml`、`catalog/capabilities/`、`graph/graph.json`。设计见 [engineering-knowledge.md](doc/design/engineering-knowledge.md) · [catalog-cli.md](doc/design/catalog-cli.md) · 调研 [okf-and-marqdo.md](doc/research/okf-and-marqdo.md)。
|
||||
|
||||
智能体框架横向调研(LangGraph / CrewAI / 厂商 SDK 等 vs Marqdo 文档驱动优势):[agent-frameworks-and-marqdo.md](doc/research/agent-frameworks-and-marqdo.md) · 优化路线 [ext-agent-optimize.md](doc/roadmap/ext-agent-optimize.md)。
|
||||
|
||||
@@ -142,30 +149,34 @@ marqdo catalog public -o .marqdo
|
||||
|
||||
---
|
||||
|
||||
## 现状(v1.1.0)
|
||||
## 现状(v1.3.0)
|
||||
|
||||
- **AI 原生闭环(v1.1.0 新增)**:核心表面守卫(19 构造,单边改动 CI 必红)· **渐进式契约**(文档内嵌 `参数`/`返回`/`字段` 表 + 四边界校验 + `marqdo check` 防漂移)· **MLSP for AI**(`marqdo mlsp`:`locate`/`syntax`/`validate`/`repair_targets`/`repair_apply`/`schema`,AI 按需查语法而非背诵)· **结构化诊断**(`run --json`,错误即数据);验证见 [perf-validation-report-2026-09-23.md](doc/roadmap/perf-validation-report-2026-09-23.md)
|
||||
- **Engineering Knowledge(v1.3.0)**:EKC L0–L4 · `marqdo knowledge|find|reuse|duplicate|verify|…` · Agent `preflight` / REUSE→ADAPT→CREATE · [engineering-knowledge.md](doc/design/engineering-knowledge.md)
|
||||
- **Web Artifact(v1.2.0 · ADR 0007)**:Document/Endpoint 一等公民(`type: web|endpoint`)· facade `page`/`route`/`serve`/`render`/`inspect`/`use` · Resource 拆到 `ext/data`/`security`/`net` · 统一 View 表 · 破坏性移除 `compose_*`/`configure`
|
||||
- **AI 栈(v1.1.1)**:`ext/ai/llm` 语义 `ask`/`stream`/`collect` · Artifact Metadata Binding(`${env|arg|sys|secret}`)· Agent Skill Compilation + Adaptive Routing · MCP stdio 客户端 + resume 检查点 · 可选 `plugins/llm` Transport ABI
|
||||
- **AI 原生闭环(v1.1.0)**:核心表面守卫(19 构造,单边改动 CI 必红)· **渐进式契约**(文档内嵌 `参数`/`返回`/`字段` 表 + 四边界校验 + `marqdo check` 防漂移)· **MLSP for AI**(`marqdo mlsp`:`locate`/`syntax`/`validate`/`repair_targets`/`repair_apply`/`schema`,AI 按需查语法而非背诵)· **结构化诊断**(`run --json`,错误即数据);验证见 [perf-validation-report-2026-09-23.md](doc/roadmap/perf-validation-report-2026-09-23.md)
|
||||
- 映射与解释器:**v0.3 语法宪法**(`**` 代码 / `*` 返回 / 叙述声明);Phase I 树遍历 + 字节码后端;金样例在 `tests/` 与 `tests/markup-v03/`
|
||||
- **对象**:`#` = 类型/构造,`##`+ = 函数/方法;见 [objects.md](doc/design/objects.md)
|
||||
- **`marqdo view`**:文档浏览器(Structure + 函数大纲/搜索 + Execution + Variables 浮窗)
|
||||
- **`marqdo debug`**:独立调试页(断点 / 单步 / locals;默认端口 7430;页面 favicon / 品牌使用官方 Logo)
|
||||
- **`marqdo catalog` / `sync`**:OKF 风格 YAML + 模块概念页
|
||||
- **`marqdo catalog` / `sync`**:OKF 风格 YAML + 模块概念页(默认挂钩 EKC)
|
||||
- **`marqdo knowledge` / `find` / `reuse` / `impact` / `conflicts` / `stale` / `duplicate` / `verify`**:Engineering Knowledge Compiler([engineering-knowledge.md](doc/design/engineering-knowledge.md))
|
||||
- **`marqdo version --check`**:与 GitHub 最新 release 对比
|
||||
- 标准库:**内置于二进制**(v0.1.2+);磁盘 `lib/` 或 `MARQDO_LIB` 可覆盖。模块含文本、表、**浏览器效应**、文件、系统、时间、JSON、网络、数学、外联、插件、**自写回**、**子任务**;**中层 Mid M1–M6** + **Mid2 M7–M10**(datetime/url/toml/html/fs++/stats/log.fields 等,见 [stdlib-mid.md](doc/design/stdlib-mid.md) · [stdlib-mid2.md](doc/design/stdlib-mid2.md))
|
||||
- **官方扩展库 `ext/`**(**非** stdlib,本版收口):
|
||||
- **`web`**:W0–W7 + P3 + **W8**;**定制 C0–C4**;**宿主 H1–H2**(`proxy`/`invoke`,见 [ext-hosting.md](doc/roadmap/ext-hosting.md));**表驱动引言**(`compose_intro` / `引言装配`);原生层默认 **Go `libweb`**([ADR 0004](doc/adr/0004-web-plugin-go.md),`scripts/build-web-plugin.sh`);示例 [web-site](examples/web-site/) · [web-site-zh](examples/web-site-zh/) · [marqdo-blog](examples/marqdo-blog/) · [anlian-mq](examples/anlian-mq/);生产路径见 [web-asgi-servers-and-marqdo.md](doc/design/web-asgi-servers-and-marqdo.md)
|
||||
- **`web`**:**Artifact 模型(ADR [0007](doc/adr/0007-web-document-endpoint.md))** — `type: web|endpoint` Document/Endpoint;facade `page`/`route`/`serve`/`render`/`inspect`/`use`;Resource 在 `ext/data` · `ext/security` · `ext/net`;设计 [ext-web-artifact.md](doc/design/ext-web-artifact.md);原生 **Go `libweb`**([ADR 0004](doc/adr/0004-web-plugin-go.md));示例 [marqdo-blog](examples/marqdo-blog/) · [web-client-site](examples/web-client-site/) · [anlian-mq](examples/anlian-mq/)
|
||||
- **`quantum`**:Q0–Q7 + Q8a/b 主题 SVG;示例 [quantum-entanglement](examples/quantum-entanglement/)
|
||||
- **`linalg`**:L0–L6 + 公式文档面(中缀 / `declare` / 展示化简);示例 [linalg-transpose](examples/linalg-transpose/) · [linalg-svd](examples/linalg-svd/) · [linalg-least-squares](examples/linalg-least-squares/)
|
||||
- **`agent`**:A1–A4 + **MCP Server stdio**(`mcp_server`);宿主缺口见 [ext-hosting.md](doc/roadmap/ext-hosting.md) · [agent-framework-gaps-after-a4.md](doc/research/agent-framework-gaps-after-a4.md)
|
||||
- **`llm`**:OpenAI 兼容对话
|
||||
- **`agent`**:A1–A4 + **MCP Server/Client stdio** + Skill Compilation + Adaptive Routing + resume + **Engineering preflight**(`agent.preflight` / `eng_preflight`);宿主缺口见 [ext-hosting.md](doc/roadmap/ext-hosting.md) · [agent-framework-gaps-after-a4.md](doc/research/agent-framework-gaps-after-a4.md) · [engineering-knowledge.md](doc/design/engineering-knowledge.md)
|
||||
- **`llm`**:语义 `ask`/`stream`/`collect`(OpenAI/Ollama;可选原生 `plugins/llm`)
|
||||
- 安装:`marqdo ext list` / `add …` / `remove`([ext-cli.md](doc/design/ext-cli.md);默认 `~/.marqdo/ext`)。原生插件先 `cargo build -p marqdo_plugin_*` 再 `ext add`
|
||||
- **原生插件 ABI**:[`include/marqdo_abi.h`](include/marqdo_abi.h) · [ext-abi.md](doc/design/ext-abi.md);`plugins/{demo,agent,web,quantum,linalg}`(`web` 为 Go;其余为 Rust)
|
||||
- **原生插件 ABI**:[`include/marqdo_abi.h`](include/marqdo_abi.h) · [ext-abi.md](doc/design/ext-abi.md);`plugins/{demo,agent,web,quantum,linalg,llm}`(`web` 为 Go;其余为 Rust)
|
||||
- **用户静态站**:`public/` → `view output` → CI 发布 [gh-pages](https://cflmy.github.io/marqdo/)
|
||||
- **VS Code 扩展**:分支 **`vscode-extension`**(`main` 不跟踪源码;见 [vscode-extension-commit.md](doc/design/vscode-extension-commit.md));Release 附带 `.vsix`
|
||||
- **浏览器 Marqdo(WASM)**:`marqdo wasm build` + 官方 bridge 自启(作者零业务 JS;桥内可含列表/路由/storage/ws/文件/Canvas/音频/Observer/拖放);`lib/browser` + GFM;`web.client_embed`;示例 [browser-hello](examples/browser-hello/) · [browser-app](examples/browser-app/) · [browser-media](examples/browser-media/) · [web-client-site](examples/web-client-site/)([ADR 0002](doc/adr/0002-browser-marqdo-wasm.md) · [D](doc/roadmap/browser-wasm-d.md) · [E](doc/roadmap/browser-wasm-e.md) · [F](doc/roadmap/browser-wasm-f.md))
|
||||
- 选型:[ADR 0001 — Rust](doc/adr/0001-implementation-language.md) · [ADR 0002 — 浏览器 WASM](doc/adr/0002-browser-marqdo-wasm.md)(C0–C5 完结,见 [roadmap/browser-wasm.md](doc/roadmap/browser-wasm.md))· [ADR 0003 — 异步效应](doc/adr/0003-browser-async-effects.md)
|
||||
|
||||
### 如何使用最新 Marqdo(v1.1.0)
|
||||
### 如何使用最新 Marqdo(v1.3.0)
|
||||
|
||||
**安装解释器(任选其一)**
|
||||
|
||||
@@ -180,7 +191,7 @@ marqdo catalog public -o .marqdo
|
||||
```bash
|
||||
# 1) 源码安装(跟 tag 或 main)
|
||||
git clone https://github.com/cflmy/marqdo.git && cd marqdo
|
||||
git checkout v1.1.0 # 或留在 main
|
||||
git checkout v1.3.0 # 或留在 main
|
||||
cargo build --release
|
||||
export PATH="$PWD/target/release:$PATH"
|
||||
marqdo version
|
||||
@@ -193,7 +204,7 @@ marqdo view public --no-open
|
||||
marqdo debug public --no-open
|
||||
marqdo catalog public -o .marqdo
|
||||
|
||||
# 2b) AI 原生闭环(v1.1.0):契约检查 / 结构化诊断 / MLSP 查询
|
||||
# 2b) AI 原生闭环(v1.1.0+):契约检查 / 结构化诊断 / MLSP 查询
|
||||
marqdo check tests/contracts/ok_contract.mq.md # 静态契约防漂移
|
||||
marqdo run tests/structure/hello.mq.md --json # 结构化诊断(错误即数据)
|
||||
printf '{"id":1,"method":"syntax","params":{"query":"返回"}}\n' | marqdo mlsp
|
||||
@@ -203,14 +214,14 @@ marqdo ext add web # 或:网页 — 自动下载 L1 + 预编译 native(
|
||||
marqdo ext add agent # 或:智能体
|
||||
marqdo ext add quantum # 或:量子
|
||||
marqdo ext add linalg # 或:线性代数
|
||||
marqdo ext add llm # 或:大模型(纯 .mq.md)
|
||||
# 开发者:cargo build --release -p marqdo_plugin_{agent,quantum,linalg}
|
||||
marqdo ext add llm # 或:大模型(.mq.md + 可选 native llm)
|
||||
# 开发者:cargo build --release -p marqdo_plugin_{agent,quantum,linalg,llm}
|
||||
# bash ./scripts/build-web-plugin.sh # 需 Go + cgo → libweb
|
||||
# 离线:解压 Windows zip(已含 ext/native)或手动放下 native zip;MARQDO_EXT_NO_DOWNLOAD=1
|
||||
|
||||
# 4) 动态站示例(扩展装好后)
|
||||
marqdo run examples/web-site-zh/index.mq.md # 表驱动引言 Demo
|
||||
marqdo run examples/marqdo-blog/index.mq.md
|
||||
marqdo run examples/marqdo-blog/serve.mq.md
|
||||
marqdo run examples/anlian-mq/index.mq.md # W-G14 验收站
|
||||
# 浏览器打开终端打印的 listen 地址;/favicon.ico 与 logo 装配见 W8
|
||||
|
||||
@@ -219,7 +230,7 @@ marqdo wasm build
|
||||
# → dist/wasm/ … 见 examples/browser-hello/
|
||||
|
||||
# 6) Releases:Windows exe/zip/vsix;Linux CLI zip + native `.so` zip
|
||||
# https://github.com/cflmy/marqdo/releases/tag/v1.1.0
|
||||
# https://github.com/cflmy/marqdo/releases/tag/v1.3.0
|
||||
```
|
||||
|
||||
开发期也可用 `cargo run -- …` 代替已安装的 `marqdo`:
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "marqdo-wasm"
|
||||
version = "1.1.0"
|
||||
version = "1.3.0"
|
||||
edition = "2021"
|
||||
license = "Apache-2.0"
|
||||
description = "Marqdo interpreter as WebAssembly (browser host bridge)"
|
||||
|
||||
Vendored
+24
@@ -1,3 +1,27 @@
|
||||
marqdo (1.3.0-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 1.3.0: Engineering Knowledge Compiler (EKC) — L0–L4 catalog,
|
||||
marqdo knowledge/find/reuse/duplicate/verify, Agent preflight
|
||||
(REUSE→ADAPT→CREATE), graph + context packs.
|
||||
|
||||
-- cflmy <pingan@cflmy.cn> Mon, 28 Sep 2026 16:50:00 +0800
|
||||
|
||||
marqdo (1.2.0-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 1.2.0: Web Artifact rewrite (ADR 0007) — Document/Endpoint/Resource,
|
||||
type:web|endpoint, web.serve/render/inspect, ext/data|security|net split,
|
||||
compose_* author surface removed; marqdo-blog Artifact example.
|
||||
|
||||
-- cflmy <pingan@cflmy.cn> Mon, 28 Sep 2026 11:39:04 +0800
|
||||
|
||||
marqdo (1.1.1-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 1.1.1: AI stack closure — llm ask/stream/collect, Metadata Binding,
|
||||
Skill Compilation + Adaptive Routing, MCP stdio client + resume checkpoints,
|
||||
optional plugins/llm Transport ABI (fallback lib/net).
|
||||
|
||||
-- cflmy <pingan@cflmy.cn> Thu, 24 Sep 2026 20:32:00 +0800
|
||||
|
||||
marqdo (1.1.0-1ppa1) noble; urgency=medium
|
||||
|
||||
* Marqdo 1.1.0: core surface guard + progressive contracts + MLSP for AI.
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
| [design/return-hr-and-code-surface.md](design/return-hr-and-code-surface.md) | 语句/返回/输出决议 |
|
||||
| [design/generated-yaml-manifest.md](design/generated-yaml-manifest.md) | 生成式 OKF 清单(早期方向) |
|
||||
| [design/okf.md](design/okf.md) | **Marqdo OKF 实现规范**(catalog v0 + 任务知识包) |
|
||||
| [design/engineering-knowledge.md](design/engineering-knowledge.md) | **Engineering Knowledge Compiler(EKC)**:L0–L4、reuse、EFI |
|
||||
| [design/catalog-cli.md](design/catalog-cli.md) | **`marqdo catalog` / `sync` OKF 清单命令** |
|
||||
| [design/tech-stack.md](design/tech-stack.md) | 实现栈对比 |
|
||||
| [adr/0001-implementation-language.md](adr/0001-implementation-language.md) | **Accepted:Rust 参考解释器;不用 Flex/Bison** |
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
# ADR 0006:Artifact Metadata Binding(`${…}` 进核心 · Metadata-only)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Status | **Accepted** |
|
||||
| Date | 2026-09-24 |
|
||||
| Related | [binding.md](../design/binding.md) · [core-surface.md](../design/core-surface.md) · [layers.md](../design/layers.md) · [doc/next/006.md](../next/006.md) |
|
||||
|
||||
## Context
|
||||
|
||||
- Executable Document 需要声明「依赖哪些外部运行时资源」(模型名、API 密钥、base URL),而不是在 `ext/*` 里各自 `sys.env_get`。
|
||||
- [doc/next/006.md](../next/006.md) 提出 **Marqdo Binding**(`${namespace.name}`)与 Artifact Metadata 统一解析。
|
||||
- [layers.md](../design/layers.md):Language 层变更须 ADR;核心构造计数只许持平或下降;默认进 ext。
|
||||
- 正文插值会与 `$$…$$` 公式面冲突,且会新增内联构造压力。
|
||||
|
||||
## Decision
|
||||
|
||||
1. 采纳 **Artifact Metadata Binding** 为 Language 层能力;正式规范见 [binding.md](../design/binding.md)。
|
||||
2. **`${…}` 仅**在文件头 `---` Metadata 中生效;正文 / 粗体 / 调用 / 表 / 叙述 **永不**写 Binding Expression(否决 [006 §17](../next/006.md))。
|
||||
3. **不**把 Binding 记入 [core-surface.md](../design/core-surface.md) 的 19 构造守卫表——与 `import` 同属 §4「其它语言面」。
|
||||
4. 核心只做声明与解析;扩展(如 `ext/ai/llm`)消费已绑定的 `Module.metadata` / `sys.meta*` / 入口提升变量。
|
||||
5. 第一版 namespace:`env` · `arg` · `sys` · `secret`。`config` / `file` 推迟。
|
||||
6. 加载期一次 resolve → Bound Artifact;`Value::Secret` 遮掩展示;`sys.env_get` 保留为逃逸口。
|
||||
7. **Phase 2**:入口执行前把 metadata 键提升为普通变量(形参/赋值可覆盖);正文用 `` `model` ``,不再 `${env…}`。
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- 配置成为文档的一部分;LLM / agent / 其它 ext 共用同一套 Binding,避免各写一套 getenv。
|
||||
- 缺失 `!` 在加载期失败,而不是运行到 HTTP 401。
|
||||
- 不膨胀 CORE_CONSTRUCTS;与公式 `$$` 无冲突。
|
||||
- 正文保持普通变量语义,不引入第二套环境变量语法。
|
||||
|
||||
### Negative / Risks
|
||||
|
||||
- Secret 在 `Text + Secret` 拼接时会揭示明文(供 Authorization);须约束日志面。
|
||||
- metadata 键与业务变量同名时依赖「形参/赋值覆盖」规则,作者需注意命名。
|
||||
|
||||
### Neutral
|
||||
|
||||
- `#` / `##` / `**` / `*` 等核心标记不变;frontmatter `import` 句法不变。
|
||||
@@ -0,0 +1,54 @@
|
||||
# ADR 0007:Web Document / Endpoint / Resource(破坏性 · 代码即文档)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Status | **Accepted** |
|
||||
| Date | 2026-09-28 |
|
||||
| Related | [ext-web-artifact.md](../design/ext-web-artifact.md) · [ext-web.md](../design/ext-web.md) · [binding.md](../design/binding.md) · ADR [0006](0006-artifact-metadata-binding.md) · [layers.md](../design/layers.md) · [doc/next/007.md](../next/007.md) |
|
||||
|
||||
## Context
|
||||
|
||||
- [`ext/web/web.mq.md`](../../ext/web/web.mq.md) 已膨胀为小型 Web Framework(Page God Object、`configure` 垃圾抽屉、碎片 Table DSL、隐式路由),与 Marqdo「代码即文档」冲突。
|
||||
- [doc/next/007.md](../next/007.md) 提出 Document / Endpoint / Resource 三原语与拆分 `ext/web` · `ext/net` · `ext/data` · `ext/security`。
|
||||
- 现行 [ext-web.md](../design/ext-web.md) **C1** 禁止为 web 改核心;Document/Endpoint 装载识别需要 Language/Runtime 参与(复用 Artifact Metadata `type`,**不**新增 CORE_CONSTRUCTS 标记)。
|
||||
- Binding(ADR 0006)已提供文件级 Metadata;`type: prompt` 先例在扩展侧消费 metadata。
|
||||
|
||||
## Decision
|
||||
|
||||
1. **采纳** Document / Endpoint / Resource 为 Web 模型一等公民;正式规范见 [ext-web-artifact.md](../design/ext-web-artifact.md)。
|
||||
2. **修订 ext-web C1**:允许核心在装载期识别 `type: web|endpoint`(及中文 `类型: 网页|端点`)并暴露 artifact 分类;**装配 / HTTP / DB 语义仍在 ext + plugin**。单元格求值、import 句法、CORE_CONSTRUCTS 计数不变。
|
||||
3. **文件级 Artifact only**:不在 `##` 函数体嵌套 `---` frontmatter(与 Binding Phase 1 / 解析器一致)。
|
||||
4. **破坏性重写作者面**:删除 `compose_*` 碎片 DSL、god `# page` 变异堆、`app.configure` 多参数抽屉、作者 `ensure_plugin`;examples 与 `tests/ext/web*` 同步重写。
|
||||
5. **目录定稿**(007 §三十五 + §四十):
|
||||
- `ext/web/` — page · route · component · dom · client · facade
|
||||
- `ext/net/` — http · websocket · url
|
||||
- `ext/data/` — db · cache · storage
|
||||
- `ext/security/` — auth · oidc · rbac
|
||||
6. **统一 View 表**:`| type | slot | value | attrs | style |`;Table = Data,禁止每函数私有方言。
|
||||
7. **declared vs system routes**:用户 route 须声明;`/_part` · `/_form` 等为 system,经 `web.inspect` 可见。
|
||||
8. **import = capability**:`import web:…` 由 facade 自动加载 native plugin;作者程序不写 `ensure_plugin` / `host_*`。
|
||||
9. **Metadata v1 锁定(原 007 未决项)**:
|
||||
- 发现:`web.serve root=` 扫描 `root/**/*.mq.md`,仅注册 `type`/`类型` ∈ {web,网页,endpoint,端点} 的文件;无 type 的库文件忽略。
|
||||
- Data:仅 flat 键 `data_source` · `data_order` · `data_where`(text;where 为 `col=val` 逗号分隔)。嵌套 YAML `data:` 推迟。
|
||||
- Result:`{ok: bool, value: any, error: text|None}`。
|
||||
- Realtime 模块名:`ext/net/websocket.mq.md`(ZH:`实时.mq.md`)。
|
||||
- ZH metadata:`类型`←type,`路由`←route,`方法`←method,`路径`←path,`请求`←request,`响应`←response,`鉴权`←auth,`数据源`←data_source,`排序`←data_order,`条件`←data_where。
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- `.mq.md` 同时是文档、路由、实现与鉴权声明。
|
||||
- 与 LLM/Agent Artifact Metadata 统一心智。
|
||||
- 职责边界清晰,可独立演化 data/security/net。
|
||||
|
||||
### Negative / Risks
|
||||
|
||||
- 破坏性:全部现有 web 作者程序与 smokes 必须改写。
|
||||
- Endpoint 热路径仍经 plugin → `CallLibPath` / 文件装载回调;须保持 ABI 稳定演进。
|
||||
- flat data 键表达力弱于嵌套 YAML;后续另 ADR。
|
||||
|
||||
### Neutral
|
||||
|
||||
- 不新增 Markdown 核心标记;`${…}` 仍仅 Metadata。
|
||||
- Go `plugins/web` 可分波删除死 compose ABI;作者面先断。
|
||||
@@ -0,0 +1,186 @@
|
||||
# Marqdo Binding · Artifact Metadata
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Accepted · Phase 1 + Phase 2(scope lift)** |
|
||||
| 日期 | 2026-09-24 |
|
||||
| ADR | [0006-artifact-metadata-binding.md](../adr/0006-artifact-metadata-binding.md) |
|
||||
| 讨论稿 | [doc/next/006.md](../next/006.md) |
|
||||
| 实现 | `src/binding/` · `Module.metadata` · entry env lift · `sys.meta` / `sys.meta_get` · `marqdo run --bind` |
|
||||
|
||||
> **边界**:核心负责「声明与解析配置」;扩展负责「解释配置的语义」。
|
||||
> **`${…}` 只出现在文件头 Artifact Metadata**——正文 / 粗体 / 调用 / 表 **永不**写 `${env…}`(否决讨论稿 §17)。正文使用已绑定的普通名字。
|
||||
|
||||
---
|
||||
|
||||
## 1. Artifact Metadata
|
||||
|
||||
文件开头成对 `---` 之间为 **Artifact Metadata**(声明式元信息环境),不是「随便贴的 YAML」。
|
||||
|
||||
```text
|
||||
.mq.md
|
||||
├── Metadata ← Identity / Configuration / Dependencies / Bindings
|
||||
└── Executable Body
|
||||
```
|
||||
|
||||
- `import bind:…` / `导入` 行:模块依赖(既有语义不变)。
|
||||
- 其它 `key: value` 行:元信息;值可为字面量或 **Binding Expression**。
|
||||
- Phase 1 仅支持 **flat** 键(同级 `key: value`);块标量 `>` / `|` 收成 Text。
|
||||
|
||||
例:
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: demo
|
||||
type: llm
|
||||
model: ${env.OPENAI_MODEL ?? "gpt-4o-mini"}
|
||||
api_key: ${secret.OPENAI_API_KEY!}
|
||||
base_url: ${env.OPENAI_BASE_URL ?? "https://api.openai.com/v1"}
|
||||
import sys:lib/sys.mq.md
|
||||
---
|
||||
```
|
||||
|
||||
加载后内部为已绑定的 `Module.metadata`(确定性快照),执行期不再对同一键反复 `getenv`。
|
||||
|
||||
---
|
||||
|
||||
## 2. Binding Expression
|
||||
|
||||
外形:
|
||||
|
||||
```text
|
||||
${expression}
|
||||
```
|
||||
|
||||
### 2.1 路径
|
||||
|
||||
```text
|
||||
${namespace.name}
|
||||
```
|
||||
|
||||
| Namespace | 意义 | Phase 1 |
|
||||
|-----------|------|---------|
|
||||
| `env` | 进程环境变量 | ✅ |
|
||||
| `arg` | CLI / Runtime 绑定(`marqdo run --bind KEY=VALUE`) | ✅ |
|
||||
| `sys` | 系统信息:`cwd` · `platform` | ✅ |
|
||||
| `secret` | 敏感环境变量 → `Value::Secret` | ✅ |
|
||||
| `config` | 配置 Artifact | 推迟 |
|
||||
| `file` | 外部文件内容 | 推迟 |
|
||||
|
||||
### 2.2 默认与必需
|
||||
|
||||
```text
|
||||
${env.OPENAI_MODEL ?? "gpt-4o-mini"}
|
||||
${secret.OPENAI_API_KEY!}
|
||||
```
|
||||
|
||||
- `A ?? B`:A 缺失(未设置 / 空)则用 B。
|
||||
- `A!`:A 必须存在,否则加载失败(`config.required_binding_missing`)。
|
||||
|
||||
### 2.3 类型转换
|
||||
|
||||
```text
|
||||
${int(env.MAX_TOKENS ?? "4096")}
|
||||
${float(env.TEMPERATURE ?? "0.2")}
|
||||
${bool(env.STREAM ?? "false")}
|
||||
```
|
||||
|
||||
### 2.4 独占 value vs 插值
|
||||
|
||||
| 形态 | 语义 |
|
||||
|------|------|
|
||||
| `model: ${env.OPENAI_MODEL}` | **Value binding** — 保留解析后的类型(含 Secret) |
|
||||
| `title: "Hello ${env.USER}"` | **Interpolation** — 结果为 Text |
|
||||
| `model: gpt-4o-mini` | **Literal** — 字面量;**不会**被环境变量静默覆盖 |
|
||||
|
||||
---
|
||||
|
||||
## 3. Resolution Policy
|
||||
|
||||
对每个 metadata 键:
|
||||
|
||||
1. 解析该键的字面量 / Binding / 插值(字面量定稿后不受 env 覆盖)。
|
||||
2. 若 CLI 提供了同名 `--bind KEY=VALUE`,**覆盖**该键的最终值(Text)。
|
||||
3. Binding 内部查找顺序(仅当表达式写出时):`arg` / `env` / `secret` / `sys` 按表达式 namespace;`??` 链从左到右。
|
||||
|
||||
推荐作者意图:
|
||||
|
||||
```text
|
||||
CLI --bind
|
||||
→ Artifact 字面量 / 已写死的 Binding 结果
|
||||
→ Binding 源(env / secret / …)
|
||||
→ ?? 默认
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Secret
|
||||
|
||||
`${secret.NAME}` → `Value::Secret`。
|
||||
|
||||
- `print` / `str` / Display → `<secret>`(不泄露)。
|
||||
- `type` → `secret`。
|
||||
- 与 Text 拼接(如 `bearer + api_key`)时**揭示**明文,供 HTTP Authorization。
|
||||
- 不得进入普通 episode / 错误信息明文(扩展作者责任;核心 Display 已遮掩)。
|
||||
|
||||
---
|
||||
|
||||
## 5. Runtime API
|
||||
|
||||
| API | 语义 |
|
||||
|-----|------|
|
||||
| `sys.meta` / `系统.元信息` | 入口模块已绑定 metadata(map) |
|
||||
| `sys.meta_get` / `系统.取元信息` | `key=` → 值或 `None` |
|
||||
| `marqdo run FILE --bind KEY=VALUE` | 可重复;写入 `arg` 并覆盖同名 metadata 键 |
|
||||
| `sys.env_get` | **保留**逃逸口;新代码优先 Metadata Binding |
|
||||
|
||||
---
|
||||
|
||||
## 6. 错误
|
||||
|
||||
| code | 何时 |
|
||||
|------|------|
|
||||
| `config.required_binding_missing` | `${ns.name!}` 且源缺失 |
|
||||
| `config.unknown_binding_ns` | 未知 namespace |
|
||||
| `config.bad_binding` | 表达式非法 / cast 失败 |
|
||||
|
||||
诊断尽量带 frontmatter 行号。
|
||||
|
||||
---
|
||||
|
||||
## 7. 与扩展的分工
|
||||
|
||||
- **核心**:解析 Metadata、resolve Binding、暴露 `Module.metadata` / `sys.meta*`,并在入口把 metadata **提升为普通变量**。
|
||||
- **`ext/ai/llm`**:构造次序为 **显式实参 → 入口作用域同名变量 / `sys.meta_get` → `sys.env_get` 回退 → 默认**。
|
||||
|
||||
---
|
||||
|
||||
## 8. Phase 2 · Metadata scope lift(Accepted)
|
||||
|
||||
加载期绑定完成后,入口执行(`# main` 或唯一无参顶层对象)前,把 `Module.metadata` 的每个键注入 entry 环境,成为**普通变量**:
|
||||
|
||||
```yaml
|
||||
---
|
||||
model: ${env.ModelName ?? "gpt-4o-mini"}
|
||||
---
|
||||
```
|
||||
|
||||
```marqdo
|
||||
# main
|
||||
|
||||
> print text=`model`
|
||||
```
|
||||
|
||||
### 规则
|
||||
|
||||
1. 注入时机:进入入口函数之前。
|
||||
2. 冲突:形参 / 调用实参 **覆盖** 同名 metadata;程序内赋值亦然。metadata **不**覆盖已存在的绑定。
|
||||
3. `sys.meta` / `sys.meta_get` 仍可用(显式通道)。
|
||||
4. **拒绝** 正文 / 粗体 / 表 / 叙述中的 `${…}`(讨论稿 006 §17 否决)。普通语言也不在语句里 `${ENV}`。
|
||||
|
||||
### 仍推迟(P3+)
|
||||
|
||||
- `config` / `file` namespace
|
||||
- 嵌套 YAML map
|
||||
- 显式 `override:` 糖
|
||||
- 计入 `CORE_CONSTRUCTS`
|
||||
@@ -23,15 +23,20 @@ YAML / OKF 风格清单是**从 `.mq.md` 派生的生成物**,不是手写配
|
||||
marqdo catalog [PATH] -o OUT_DIR
|
||||
# 别名(与早期草案一致)
|
||||
marqdo sync [PATH] -o OUT_DIR
|
||||
# 跳过 Engineering Knowledge 投影(仅旧式 catalog)
|
||||
marqdo catalog [PATH] -o OUT_DIR --no-knowledge
|
||||
```
|
||||
|
||||
| 参数 | 默认 | 说明 |
|
||||
|------|------|------|
|
||||
| `PATH` | `.` | 工程根或含 `.mq.md` 的目录(递归扫描) |
|
||||
| `-o` / `--out` | `.marqdo` | 生成目录(写入前创建;**覆盖**同名生成文件;扫描时跳过该目录) |
|
||||
| `--no-knowledge` | off | 不跑 EKC;默认在 catalog 成功后编译 L0–L4 / graph |
|
||||
|
||||
退出码:成功 0;无可扫描文件或 IO/解析失败非 0(单文件解析失败时写入 `diagnostics` 并继续其余文件,最终若有失败则非 0)。
|
||||
|
||||
Engineering Knowledge 命令见 [engineering-knowledge.md](engineering-knowledge.md)(`find` / `reuse` / `duplicate` / …)。
|
||||
|
||||
---
|
||||
|
||||
## 3. 生成物
|
||||
|
||||
@@ -44,4 +44,4 @@
|
||||
|
||||
## 4. 范围外(进 Language 层,但不在本守卫)
|
||||
|
||||
frontmatter YAML(元信息)与 `import` 行(模块装载)属于 Language 层的其它语言面,随 [layers.md](layers.md) 准入流程登记,暂不进本清单守卫;如需一并守卫,走 ADR 扩表。
|
||||
frontmatter / Artifact Metadata(元信息)、`import` 行(模块装载)、以及 Metadata 内的 **Binding Expression**(`${ns.name}`,见 [binding.md](binding.md) · [ADR 0006](../adr/0006-artifact-metadata-binding.md))属于 Language 层的其它语言面,随 [layers.md](layers.md) 准入流程登记,**暂不进本清单守卫**(不增 CORE_CONSTRUCTS 计数)。Phase 2 将已绑定 metadata **提升为入口普通变量**(仍非新标记)。如需一并守卫,走 ADR 扩表。
|
||||
@@ -0,0 +1,97 @@
|
||||
# Engineering Knowledge Compiler(EKC)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Accepted(v1 已落地)** |
|
||||
| 日期 | 2026-09-28 |
|
||||
| 提案 | [next/008.md](../next/008.md) |
|
||||
| 相关 | [okf.md](okf.md) · [catalog-cli.md](catalog-cli.md) · [ext-agent.md](ext-agent.md) |
|
||||
|
||||
## 1. 定位
|
||||
|
||||
Marqdo 把仓库投影为 **可查询、可验证、可复用** 的工程知识系统。Source of Truth 仍是 `*.mq.md`;`.marqdo/**` 是 GENERATED 投影。
|
||||
|
||||
```text
|
||||
.mq.md ──► EKC ──► Executable + Catalog L0–L4 + Graph + Context Pack
|
||||
│
|
||||
▼
|
||||
Agent preflight (REUSE → ADAPT → CREATE)
|
||||
```
|
||||
|
||||
**Engineering First Information (EFI)**:生成代码前先获得已有能力、约束、决策、失败经验与推荐复用。
|
||||
|
||||
## 2. 五层 Catalog
|
||||
|
||||
| 层 | 内容 | 落盘 |
|
||||
|----|------|------|
|
||||
| L0 Repository | 语言/规模 | `.marqdo/catalog/repository.mq.md` |
|
||||
| L1 Module | imports/exports/responsibility | `.marqdo/catalog/modules/` |
|
||||
| L2 Symbol | signature/calls/fingerprint | `.marqdo/catalog/symbols/` |
|
||||
| L3 Capability | 工程能做什么 | `.marqdo/catalog/capabilities/` |
|
||||
| L4 Knowledge | Decision/Constraint/Failure/… | `.marqdo/knowledge/{decisions,constraints,…}/` |
|
||||
|
||||
既有 `marqdo catalog` 根级 `catalog.yaml` / `index.md` / `modules/` / `concepts/` **保留**;`catalog` 成功后默认再跑 EKC(`--no-knowledge` 可关)。
|
||||
|
||||
## 3. 知识类型(OKF `type`)
|
||||
|
||||
| type | 含义 |
|
||||
|------|------|
|
||||
| `Capability` / `Function` | 能力与符号投影 |
|
||||
| `Decision` | 为什么这样做 |
|
||||
| `Constraint` | 必须/禁止 |
|
||||
| `Failure` / `AntiPattern` | 负面知识 |
|
||||
| `Pattern` / `Migration` / `Fact` | 模式、迁移、事实 |
|
||||
| `Marqdo Context Pack` | Agent 上下文制品 |
|
||||
| `Marqdo Policy` | 复用决议编译结果 |
|
||||
| `Marqdo Engineering Knowledge` | 总览 / `engineering.yaml` |
|
||||
|
||||
未知 type 优雅降级(可读,不专属逻辑)。
|
||||
|
||||
## 4. 图边
|
||||
|
||||
`implements` · `uses` · `depends_on` · `tested_by` · `documented_by` · `constrained_by` · `decided_by` · `supersedes` · `conflicts_with` · `deprecated_by` · `derived_from` · `failed_by` · `related_to`
|
||||
|
||||
落盘:`.marqdo/graph/graph.json` · `edges.json` · `index.json`。
|
||||
|
||||
## 5. CLI
|
||||
|
||||
```bash
|
||||
marqdo knowledge [PATH] -o .marqdo # compile(默认)
|
||||
marqdo knowledge preflight "task"
|
||||
marqdo knowledge record REUSE
|
||||
marqdo knowledge learn --task ... --failure ...
|
||||
marqdo find "authentication"
|
||||
marqdo reuse "implement oauth login"
|
||||
marqdo impact path/to/file.mq.md
|
||||
marqdo conflicts | stale | duplicate [--fail] | verify
|
||||
```
|
||||
|
||||
统一 `--json`。`duplicate` 默认阈值 `0.75`;`--fail` 作 CI Gate。
|
||||
|
||||
## 6. Reuse 协议
|
||||
|
||||
三态:**REUSE → ADAPT → CREATE**(硬顺序)。Reuse budget:`min_candidates` / `allow_create_after`。仅 `create_allowed: true` 时允许盲生成。
|
||||
|
||||
## 7. Context Pack
|
||||
|
||||
`.marqdo/agent/contexts/task-<id>.mq.md`:Task / Existing Capabilities / APIs / Constraints / Decisions / Failures / Recommended / Forbidden。
|
||||
|
||||
## 8. Agent
|
||||
|
||||
- Plugin:`agent_eng_preflight` / `agent_eng_reuse` / `agent_eng_record`
|
||||
- Facade:`agent.preflight` · `agent.eng_reuse` · `agent.eng_record`
|
||||
- `agent.plan … eng_preflight=True` 在 agent-kb 之前跑 EFI;缺知识图时 `missing_knowledge` 放行(兼容纯 agent-kb 测试)
|
||||
|
||||
## 9. 指标
|
||||
|
||||
`.marqdo/agent/episodes/reuse_metrics.json`:`reuse_ratio` / `adapt_ratio` / `novel_ratio` / `duplication_rate`。
|
||||
|
||||
## 10. 原则(五条)
|
||||
|
||||
1. Discover before Create
|
||||
2. Reuse before Adapt
|
||||
3. Adapt before Duplicate
|
||||
4. Verify before Promote
|
||||
5. Code changes update knowledge
|
||||
|
||||
**不做**:以向量库为知识模型;embedding 仅可选检索实现,默认关闭。Hybrid = lexical + symbol + capability + graph。
|
||||
+19
-3
@@ -4,7 +4,7 @@
|
||||
|---|---|
|
||||
| Status | **Accepted · redesign**(取代「thin TOOL: loop」方案) |
|
||||
| Date | 2026-08-07 |
|
||||
| Related | [ext-llm.md](ext-llm.md) · [ext-cli.md](ext-cli.md) · [ext-abi.md](ext-abi.md) · [**ext-agent-plan.md**](ext-agent-plan.md)(**多步锁定设计**) · [**ext-agent-parent.md**](ext-agent-parent.md)(**父 Plan-and-Move**) · [**okf.md**](okf.md)(**OKF / 任务知识包**) · [**调研:主流框架与 Marqdo 优势**](../research/agent-frameworks-and-marqdo.md) · [**缺口:A0–A4 之后**](../research/agent-framework-gaps-after-a4.md) · [module-namespace.md](module-namespace.md) · [objects.md](objects.md) · [stdlib-writeback.md](stdlib-writeback.md) · [stdlib-subtask.md](stdlib-subtask.md) · [markdown-mapping.md](markdown-mapping.md) |
|
||||
| Related | [ext-llm.md](ext-llm.md) · [ext-cli.md](ext-cli.md) · [ext-abi.md](ext-abi.md) · [**ext-agent-plan.md**](ext-agent-plan.md)(**多步锁定设计**) · [**ext-agent-parent.md**](ext-agent-parent.md)(**父 Plan-and-Move**) · [**okf.md**](okf.md)(**OKF / 任务知识包**) · [**engineering-knowledge.md**](engineering-knowledge.md)(**EKC / EFI preflight**) · [**调研:主流框架与 Marqdo 优势**](../research/agent-frameworks-and-marqdo.md) · [**缺口:A0–A4 之后**](../research/agent-framework-gaps-after-a4.md) · [module-namespace.md](module-namespace.md) · [objects.md](objects.md) · [stdlib-writeback.md](stdlib-writeback.md) · [stdlib-subtask.md](stdlib-subtask.md) · [markdown-mapping.md](markdown-mapping.md) |
|
||||
|
||||
## 0. 为什么要改掉旧方案
|
||||
|
||||
@@ -167,8 +167,9 @@ ext/
|
||||
**分层纪律(硬规则,勿破)**:
|
||||
|
||||
1. `ext/**/*.mq.md` **禁止**出现任何 `host_*` / `host_` 调用。
|
||||
2. Agent 专属能力(含 OKF agent-kb:`agent_goal_sig` / `agent_kb_lookup` / `agent_kb_promote` / …)只进 **`plugins/agent` 注册名**,由 `ext/ai/agent` 在 `plugin.load` 后调用。
|
||||
3. **禁止**向 `src/host/`(`HostFn` 表)添加 agent / OKF 领域原语,以免核心包体膨胀。通用 L0.5 仅保留 fs/json/subtask 等标准库底层。
|
||||
2. Agent 专属能力(含 OKF agent-kb:`agent_goal_sig` / `agent_kb_lookup` / `agent_kb_promote` / …;以及 EKC:`agent_eng_preflight` / `agent_eng_reuse` / `agent_eng_record`)只进 **`plugins/agent` 注册名**,由 `ext/ai/agent` 在 `plugin.load` 后调用。
|
||||
3. **禁止**向 `src/host/`(`HostFn` 表)添加 agent / OKF / EKC 领域原语,以免核心包体膨胀。通用 L0.5 仅保留 fs/json/subtask 等标准库底层。
|
||||
4. **Engineering preflight**:`agent.preflight` / `plan … eng_preflight=True` 在 agent-kb 之前解析 REUSE/ADAPT/CREATE(见 [engineering-knowledge.md](engineering-knowledge.md))。缺 `.marqdo` 知识图时返回 `missing_knowledge` 并放行,以兼容纯任务包测试。
|
||||
4. 进程退出用 `sys.exit` / `系统.退出`,不用裸 `exit`。
|
||||
|
||||
官方 `ext/ai/` 用 `lib/*` 包装或插件注册名;见 [module-namespace.md](module-namespace.md) §8 · [ext-abi.md](ext-abi.md)。
|
||||
@@ -180,11 +181,26 @@ ext/
|
||||
| 中文 | 英文 | 角色 |
|
||||
|------|------|------|
|
||||
| `# 智能体` | `# agent` | 构造:`大模型`/`model`,`工具`/`tools`,可选 `站立提示`/`standing` |
|
||||
| `## 运行` | `## run` | **v2 主入口**:Adaptive `route` → execute → record episode → maybe_learn(技能 + 策略) |
|
||||
| `## 单步` | `## step` | 原子执行;注入源码 + 位置;可选工具;返回结构化 map;**默认自动写回**(`写回=` / `writeback=` 可关) |
|
||||
| `## 多步` | `## plan` | 建/续工作簿 → subtask 跑文件 → 修订循环 → 汇总(见 [ext-agent-plan.md](ext-agent-plan.md)) |
|
||||
| `## 解析` | `## resolve` | 技能解析:exact → alias → canonical → near |
|
||||
| `## 路由` | `## route` | **P4 Adaptive Routing**:`marqdo` \| `small-llm` \| `llm` \| `auto`(`jev` 为 `small-llm` 兼容别名) |
|
||||
| `## 应用路由` | `## route_apply` | 将小模型/大模型路由回复(`SKILL:<slug>` / `EXPLORE`)落到 kb 技能 |
|
||||
| `## 编译策略` | `## compile_policy` | Policy Compilation → `policies/router.mq.md` |
|
||||
| `## 记录情景` | `## record_episode` | 每次执行沉淀为 `.mq.md` episode(`.marqdo/agent-memory/`) |
|
||||
| `## 尝试学习` | `## maybe_learn` | 情景足够时归纳 → 合成 → 编译技能,并尝试编译策略 |
|
||||
| `## 编译` | `## compile` | 候选技能 → 版本化 `skills/<slug>/vN.mq.md` + 可选 OKF promote |
|
||||
| `## 指标` | `## metrics` | Reasoning amortization(`avg_llm_calls` / `compiled_runs`) |
|
||||
| `## 分解` | `## decompose` |(可公开或内部)目标 → 子任务表 |
|
||||
| `## 清空历史` | `## clear_history` | 若仍保留会话式辅助状态则清空;**不以隐藏 chat 袋为真相源**——真相在文档写回 |
|
||||
|
||||
**Skill Compilation(v2)**:Agent 的记忆不是向量袋,而是可执行 `.mq.md`。路径:`episodes/` → `experiences/` → `candidates/` → `skills/vN.mq.md` → OKF `agent-kb`(`llm_free`)。编译技能正文为合法 Marqdo(`**result = …**` + 斜体返回);金样 `tests/ext/agent-zero-llm.mq.md` 验收 policy+skill 全路径 `execution.llm_calls=0`。`plan` 对已存在的 `resources/<slug>.mq.md` **不覆盖**(`agent-plan-preserve`)。
|
||||
|
||||
**LLM 成本**:`model_turn` 走 `model.ask` / `stream`+`collect`,把 `LLMResult.tokens` 写入 episode / `execution`。
|
||||
|
||||
**Adaptive Routing(P4)**:Router 是抽象能力,backend = `marqdo`(已编译 `policies/router.mq.md`)/ `small-llm`(`router_model` 任意 llm 句柄)/ `llm` / `auto` 级联。`jev` 仅为 `small-llm` 兼容别名,**不进入框架专属设计**。热路径在 **`plugins/agent`**(`route.rs`);**禁止**向 `src/host/` 加领域原语。设计:[doc/next/003.md](../next/003.md) · [doc/next/004.md](../next/004.md)。
|
||||
|
||||
构造参数(v1):
|
||||
|
||||
| 参数 | 说明 |
|
||||
|
||||
+74
-51
@@ -2,13 +2,21 @@
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Status | Accepted (v0.1.x) |
|
||||
| Date | 2026-08-06 |
|
||||
| Related | [objects.md](objects.md) · [ext-abi.md](ext-abi.md) · [ext-agent.md](ext-agent.md) · [ext-cli.md](ext-cli.md) |
|
||||
| Status | **Accepted · v0.3 semantic**(005:Intelligence Primitive) |
|
||||
| Date | 2026-09-24 |
|
||||
| Related | [objects.md](objects.md) · [ext-agent.md](ext-agent.md) · [ext-cli.md](ext-cli.md) · [ai-stack.md](../roadmap/ai-stack.md) · [doc/next/005.md](../next/005.md) |
|
||||
|
||||
## Scope
|
||||
|
||||
`ext/ai/` holds **official LLM + agent** extensions. Import paths:
|
||||
`ext/ai/llm` is Marqdo’s **intelligence primitive**, not an OpenAI SDK wrapper.
|
||||
|
||||
```text
|
||||
Author surface: ask / stream / collect / create / fast / reasoning / prompt_load
|
||||
Transport: ext/ai/llm/openai.mq.md (+ ollama defaults)
|
||||
Compat: complete / chat
|
||||
```
|
||||
|
||||
Import:
|
||||
|
||||
```markdown
|
||||
---
|
||||
@@ -16,70 +24,85 @@ import llm:ext/ai/llm.mq.md
|
||||
---
|
||||
```
|
||||
|
||||
Resolution: `MARQDO_EXT/ai/…`, repo `ext/ai/…`, cwd `ext/ai/…` (see [ext-cli.md](ext-cli.md)).
|
||||
|
||||
Other official ext: [ext-agent.md](ext-agent.md) / [ext-agent-plan.md](ext-agent-plan.md) (**document-driven agent** — step with default writeback / plan workbook; thin `TOOL:` loop superseded). Installer: [ext-cli.md](ext-cli.md). Native plugins: [ext-abi.md](ext-abi.md).
|
||||
|
||||
Streaming: `complete stream=True` returns an event list (`reasoning` / `delta` / `done` / `error`); `echo=True` prints thinking (`reasoning_content`) and answer tokens as they arrive. See [agent-streaming.md](../roadmap/agent-streaming.md). Default (`stream=False`) is unchanged (answer string).
|
||||
|
||||
**Note:** Prefer `marqdo ext add llm`. Import `import llm:ext/ai/llm.mq.md` from repo or install root.
|
||||
|
||||
## Platform prerequisites
|
||||
|
||||
| Capability | Surface |
|
||||
|------------|---------|
|
||||
| HTTPS + headers | `lib/net` → `http_post` / `headers=` map |
|
||||
| Dotenv | `## load_env` in ext (or `load_dotenv` / `lib/sys`) |
|
||||
| JSON quote | `lib/json` → `quote` |
|
||||
|
||||
## `ext/ai/llm` / `ext/ai/大模型` (object handles)
|
||||
## Public API
|
||||
|
||||
| English | Chinese | Role |
|
||||
|---------|---------|------|
|
||||
| `## load_env` | `## 加载环境` | Free function: load `.env` |
|
||||
| `## stream_result` | `## 流式结果` | Collapse event list → final text |
|
||||
| `# llm` | `# 大模型` | Object ctor → handle map |
|
||||
| `## complete` / `## chat` | `## 运行` / `## 聊天` | Methods; optional `stream` / `流式`, `echo` / `打印增量` |
|
||||
| `## load_env` | `## 加载环境` | Load `.env` |
|
||||
| `## create` | `## 创建` | Factory → LLM handle |
|
||||
| `## fast` / `## reasoning` | `## 快速` / `## 推理` | Named handles |
|
||||
| `## prompt_load` / `## prompt` / `## prompt_body` | `## 加载提示` / `## 提示` / `## 提示正文` | Prompt artifacts (`type: prompt` → body) |
|
||||
| `## ask` (module) | `## 提问` | Convenience → **text** (`prompt=` or `path=`) |
|
||||
| `## collect` | `## 收集` | Events → text |
|
||||
| `## stream_result` | `## 流式结果` | Alias of `collect` |
|
||||
| `# llm` | `# 大模型` | Handle ctor (`backend=` openai-compatible \| ollama) |
|
||||
| `## ask` (method) | `## 提问` | → **LLMResult** map |
|
||||
| `## stream` | `## 流式` | → event list |
|
||||
| `## complete` / `## chat` | `## 运行` / `## 聊天` | Compat / transport primitive |
|
||||
|
||||
### Preferred
|
||||
|
||||
```markdown
|
||||
---
|
||||
import llm:ext/ai/llm.mq.md
|
||||
---
|
||||
**answer = > llm.ask prompt="What is Marqdo?"**
|
||||
*answer*
|
||||
|
||||
# main
|
||||
**model = > llm.create**
|
||||
**result = > `model`.ask prompt=`prompt`**
|
||||
*[text](result)*
|
||||
|
||||
> load_env
|
||||
**fast = > llm.fast**
|
||||
**events = > `fast`.stream prompt=`prompt`**
|
||||
**text = > llm.collect events=`events`**
|
||||
|
||||
*`model` = > llm *
|
||||
*`reply` = > `model`.complete prompt=Say hi in one word *
|
||||
> print text=`reply`
|
||||
|
||||
*`events` = > `model`.complete prompt=Say hi stream=True *
|
||||
- [`ev`](`events`)
|
||||
*`t` = > json.get value=`ev` key=type *
|
||||
1. `t` == "delta"
|
||||
*`chunk` = > json.get value=`ev` key=text *
|
||||
> print text=`chunk`
|
||||
2. *
|
||||
*`_` = 1*
|
||||
**body = > llm.prompt path="fixtures/prompt-marqdo.md"**
|
||||
**body2 = > llm.prompt text=`raw`**
|
||||
```
|
||||
|
||||
Environment (create a **project-local** `.env`; do not commit secrets):
|
||||
### LLMResult
|
||||
|
||||
| Field | Meaning |
|
||||
|-------|---------|
|
||||
| `text` | Answer string |
|
||||
| `model` | Model id |
|
||||
| `finish` | Finish reason (`stop`, …) |
|
||||
| `backend` | e.g. `openai-compatible` / `ollama` |
|
||||
| `usage` | `{prompt_tokens,completion_tokens,total_tokens}` |
|
||||
| `tokens` | Alias of `usage.total_tokens` (agent metrics) |
|
||||
| `name` | Optional handle tag (`fast` / `reasoning`) |
|
||||
| `llm_calls` | `1` per ask |
|
||||
| `tool_calls` | List of OpenAI-shaped tool call objects (empty when none) |
|
||||
|
||||
### Compat
|
||||
|
||||
`complete` / `chat` still return a bare string (or event list when `stream=True`) so `ext/ai/agent` and older runbooks keep working. **New code should use `ask` / `stream` / `collect`.**
|
||||
|
||||
## Environment
|
||||
|
||||
```env
|
||||
OPENAI_API_KEY=sk-...
|
||||
OPENAI_BASE_URL=https://api.openai.com/v1
|
||||
OPENAI_MODEL=gpt-4o-mini
|
||||
MARQDO_LLM_FAST_MODEL=...
|
||||
MARQDO_LLM_REASONING_MODEL=...
|
||||
OLLAMA_HOST=http://127.0.0.1:11434/v1
|
||||
```
|
||||
|
||||
Fallbacks: `MARQDO_LLM_API_KEY`, `MARQDO_LLM_BASE_URL`, `MARQDO_LLM_MODEL`.
|
||||
Fallbacks: `MARQDO_LLM_API_KEY`, `MARQDO_LLM_BASE_URL`, `MARQDO_LLM_MODEL`. Ctor also accepts `api_key=` / `base_url=` / `model=` / `backend=`.
|
||||
|
||||
## Backends
|
||||
|
||||
| `backend=` | Module | Notes |
|
||||
|------------|--------|-------|
|
||||
| `openai-compatible` (default) | `ext/ai/llm/openai.mq.md` | Chat Completions HTTP + SSE |
|
||||
| `ollama` | defaults in ctor + same transport | Key defaults to `ollama`; host from `OLLAMA_HOST` |
|
||||
|
||||
## Tests
|
||||
|
||||
- Import smoke: `tests/ext/llm-import.mq.md`
|
||||
- Offline stream: `tests/ext/llm-stream-offline.mq.md` + `tests/lib/net-openai-sse.mq.md`
|
||||
- Live complete (local): `tests/ext/llm-complete.mq.md` + `tests/ext/.env`
|
||||
- Live agent `## step`: `tests/ext/agent-run-live.mq.md` (DeepSeek; gold: `ext_agent_run_live`)
|
||||
- Agent smoke (real `llm` handle): `tests/ext/agent-smoke.mq.md` (gold: `ext_agent_framework_smoke`)
|
||||
- Live complete: `tests/ext/llm-complete.mq.md` (gold: `ext_llm_complete_live`)
|
||||
|
||||
- `tests/ext/llm-import.mq.md`
|
||||
- `tests/ext/llm-ctor-offline.mq.md`
|
||||
- `tests/ext/llm-ask-offline.mq.md` (create + collect)
|
||||
- `tests/ext/llm-stream-offline.mq.md`
|
||||
- `tests/ext/llm-named-offline.mq.md` (fast/reasoning + prompt_load)
|
||||
- `tests/ext/llm-tool-calls-offline.mq.md` (`tool_calls_from` / Result field)
|
||||
- `tests/ext/llm-meta-offline.mq.md` · `llm-entry-prompt-offline.mq.md`
|
||||
- Live: `llm-complete.mq.md` · `llm-stream-live.mq.md`
|
||||
@@ -0,0 +1,197 @@
|
||||
# 官方扩展:Web Artifact(Document / Endpoint / Resource)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Accepted · 破坏性(取代 page/app God Object 作者面)** |
|
||||
| 日期 | 2026-09-28 |
|
||||
| ADR | [0007-web-document-endpoint.md](../adr/0007-web-document-endpoint.md) |
|
||||
| 讨论稿 | [doc/next/007.md](../next/007.md)(superseded) |
|
||||
| 安装 | `marqdo ext add web`(`web` / `网页`) |
|
||||
| 相关 | [binding.md](binding.md) · [ext-web.md](ext-web.md) · [stdlib-i18n.md](stdlib-i18n.md) · [layers.md](layers.md) |
|
||||
|
||||
---
|
||||
|
||||
## 0. 一句话
|
||||
|
||||
> **Document describes the page. Table describes the data. Endpoint describes the interaction. Metadata describes the runtime. Marqdo executes all of them.**
|
||||
|
||||
---
|
||||
|
||||
## 1. 三原语
|
||||
|
||||
| Artifact | Metadata `type` / `类型` | 含义 |
|
||||
|----------|--------------------------|------|
|
||||
| **Document** | `web` / `网页` | Markdown 正文 + View → HTML;`route`/`路由` 注册 declared HTTP |
|
||||
| **Endpoint** | `endpoint` / `端点` | 可执行正文为 handler;`path`+`method` 注册 API |
|
||||
| **Resource** | (绑定/导入) | `ext/data` · `ext/security` · 未来 LLM 等,经 metadata / import 声明 |
|
||||
|
||||
核心装载识别 `type`/`类型`(见 `src/artifact/`);渲染与 listen 在 `ext/web` + `plugins/web`。
|
||||
|
||||
---
|
||||
|
||||
## 2. 目录
|
||||
|
||||
```text
|
||||
ext/web/
|
||||
web.mq.md · 网页.mq.md # facade: page route serve render inspect
|
||||
page.mq.md · route.mq.md · component.mq.md · dom.mq.md · client.mq.md
|
||||
ext/net/ http.mq.md · websocket.mq.md · url.mq.md
|
||||
ext/data/ db.mq.md · cache.mq.md · storage.mq.md
|
||||
ext/security/ auth.mq.md · oidc.mq.md · rbac.mq.md
|
||||
```
|
||||
|
||||
作者入口通常:
|
||||
|
||||
```text
|
||||
site/
|
||||
index.mq.md # type: web · route: / · 或 # main → web.serve
|
||||
pages/*.mq.md # type: web
|
||||
api/*.mq.md # type: endpoint
|
||||
data/ # sqlite runtime (gitignore)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Document(`type: web`)
|
||||
|
||||
```markdown
|
||||
---
|
||||
type: web
|
||||
title: Marqdo Blog
|
||||
route: /
|
||||
method: GET
|
||||
data_source: posts
|
||||
data_order: -created_at
|
||||
import web:ext/web/web.mq.md
|
||||
import data:ext/data/db.mq.md
|
||||
---
|
||||
|
||||
# Marqdo
|
||||
|
||||
Welcome to the executable web.
|
||||
```
|
||||
|
||||
| 键 (EN) | 键 (ZH) | 说明 |
|
||||
|---------|---------|------|
|
||||
| `type` | `类型` | `web` / `网页` |
|
||||
| `route` | `路由` | 路径,可含 `{param}` |
|
||||
| `method` | `方法` | 默认 `GET` |
|
||||
| `title` | `标题` | 文档标题 / `<title>` |
|
||||
| `data_source` | `数据源` | 表名 |
|
||||
| `data_order` | `排序` | 如 `-created_at` |
|
||||
| `data_where` | `条件` | `col=val,col2=val2` |
|
||||
|
||||
正文 Markdown 经 `dom` 变为 DOM Artifact,再 `web.render`。可选 View 表:
|
||||
|
||||
```markdown
|
||||
`nodes` =
|
||||
|
||||
| type | slot | value | attrs | style |
|
||||
|------|------|-------|-------|-------|
|
||||
| text | kicker | AI Native | | eyebrow |
|
||||
| title | title | Marqdo | | title |
|
||||
```
|
||||
|
||||
`page.render nodes=` 是**唯一**主装配路径。禁止 `compose_main` / `compose_intro` 等碎片 DSL。
|
||||
|
||||
---
|
||||
|
||||
## 4. Endpoint(`type: endpoint`)
|
||||
|
||||
```markdown
|
||||
---
|
||||
type: endpoint
|
||||
method: POST
|
||||
path: /api/posts
|
||||
request: json
|
||||
response: json
|
||||
auth: member
|
||||
import data:ext/data/db.mq.md
|
||||
---
|
||||
|
||||
**post = > data.db.insert …**
|
||||
|
||||
*post*
|
||||
```
|
||||
|
||||
| 键 (EN) | 键 (ZH) | 说明 |
|
||||
|---------|---------|------|
|
||||
| `type` | `类型` | `endpoint` / `端点` |
|
||||
| `path` | `路径` | URL |
|
||||
| `method` | `方法` | GET/POST/… |
|
||||
| `request` | `请求` | `json` / `form` / `none` |
|
||||
| `response` | `响应` | `json` / `text` / `html` |
|
||||
| `auth` | `鉴权` | 角色名或 `none` |
|
||||
|
||||
正文在请求时执行;返回值按 `response` 序列化。Result 协议见 §8。
|
||||
|
||||
**Handler 约定(锁定)**:Endpoint 文件用 `# main` 作为请求处理器。请求体/query/path 参数经 `run_artifact` 以 Metadata Binding 覆盖注入(与 `--bind` 同源),正文用已提升的普通变量名。无 `# main` 时返回 Artifact Result 摘要(`ok`/`value`/`error`),适合作文档型 ping。
|
||||
|
||||
---
|
||||
|
||||
## 5. Facade API(EN / ZH)
|
||||
|
||||
| EN | ZH | 作用 |
|
||||
|----|-----|------|
|
||||
| `web.page` | `网页.页面` | 从 metadata/正文/nodes 构造 Document 句柄 |
|
||||
| `web.route` | `网页.路由` | 注册 declared Document 或 Endpoint |
|
||||
| `web.serve` | `网页.服务` | `root=` 扫描注册 + listen |
|
||||
| `web.render` | `网页.渲染` | Document → HTML(离线/预览) |
|
||||
| `web.inspect` | `网页.检视` | `{declared, system}` 路由表 |
|
||||
|
||||
Middleware 一等:`web.use` / `route.use`,表驱动;**禁止** `configure(cors, security, …)` 垃圾抽屉。
|
||||
|
||||
---
|
||||
|
||||
## 6. 发现算法(锁定)
|
||||
|
||||
`web.serve root=DIR host=… port=…`:
|
||||
|
||||
1. 递归枚举 `DIR` 下 `*.mq.md`(跳过 `data/`、隐藏目录、`node_modules`)。
|
||||
2. 解析文件头 Metadata;无 `type`/`类型` 或非 web/endpoint → 跳过。
|
||||
3. Document → declared route(缺 `route` 则相对 `root` 的路径映射,`index.mq.md` → `/`)。
|
||||
4. Endpoint → declared path+method(`path`/`路径` 必需)。
|
||||
5. System routes(`/_part/*`、`/_form/*`、静态等)由 runtime 挂载,仅出现在 `inspect.system`。
|
||||
6. 调用 native listen。
|
||||
|
||||
---
|
||||
|
||||
## 7. Table 规则
|
||||
|
||||
- **View**:仅 `|type|slot|value|attrs|style|`(及列名 ZH 别名:`类型|槽|值|属性|样式`)。
|
||||
- **Resource schema**:`|字段|类型|可空|` 等 — 专用且文档化,不是函数私有方言。
|
||||
- **禁止** `|属性|值|样式|` / `|front|back|css|` / `|组件|样式|` 作为页面主装配(已删除)。
|
||||
- Raw HTML/CSS = escape hatch(`dom.raw` / `page.css`),不是默认语言。
|
||||
|
||||
---
|
||||
|
||||
## 8. Result
|
||||
|
||||
所有易失败的 Resource / Endpoint 边界返回 map:
|
||||
|
||||
| 键 | 类型 | 说明 |
|
||||
|----|------|------|
|
||||
| `ok` | bool | 成功 |
|
||||
| `value` | any | 成功载荷 |
|
||||
| `error` | text \| None | 失败信息 |
|
||||
|
||||
---
|
||||
|
||||
## 9. 硬约束(取代旧 C1–C7 作者面)
|
||||
|
||||
| # | 约束 |
|
||||
|---|------|
|
||||
| A1 | Document/Endpoint 由核心识别 `type`;装配在 ext(ADR 0007) |
|
||||
| A2 | 禁止 `json.set` 袋胶水;Table = Data |
|
||||
| A3 | 中英分文件;同文件禁止混 API |
|
||||
| A4 | 禁止作者 `ensure_plugin` / `host_*` |
|
||||
| A5 | 禁止隐式用户 route;system 须 `inspect` 可见 |
|
||||
| A6 | 禁止恢复 `compose_*` / `app.configure` / god page 变异堆 |
|
||||
| A7 | 能力只许迁移/提升;删除须本设计明示 + tests 同步 |
|
||||
| A8 | 设计未锁定处停工提问,禁止自行发明 DSL |
|
||||
|
||||
---
|
||||
|
||||
## 10. 与旧 ext-web.md
|
||||
|
||||
旧「页面表 + compose_* + app.route」作者面 **废止**。历史能力矩阵见 [web-net-capabilities.md](web-net-capabilities.md);迁移后以本文件与 ADR 0007 为准。
|
||||
+17
-13
@@ -2,18 +2,21 @@
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Accepted · W0–W7 + P3 + W8(站点图标/Head/图片装配)完结**(SQLite/Postgres 作者面:页面装配 / db CRUD+`where` / form / 嵌入主区 / `app.route` / 路由 `/_part` / admin UI / RBAC / 上传相册 / SEO·RSS·sitemap / **favicon·头资源·图片装配**;Postgres 等见 §9) |
|
||||
| 日期 | 2026-08-11(W8:2026-08-29;请求上下文:2026-09-18) |
|
||||
| 相关 | [markdown-mapping.md](markdown-mapping.md) · [module-namespace.md](module-namespace.md) · [objects.md](objects.md) · [ext-abi.md](ext-abi.md) · [ext-cli.md](ext-cli.md) · [stdlib-i18n.md](stdlib-i18n.md) · [web-assets-and-images.md](web-assets-and-images.md) · **[ext-web-request-context.md](ext-web-request-context.md)** |
|
||||
| 状态 | **Superseded(作者面)· 历史能力档案** — 现行作者面见 **[ext-web-artifact.md](ext-web-artifact.md)** · ADR [0007](../adr/0007-web-document-endpoint.md) |
|
||||
| 日期 | 2026-08-11(W8:2026-08-29;请求上下文:2026-09-18;Artifact 重构:2026-09-28) |
|
||||
| 相关 | [ext-web-artifact.md](ext-web-artifact.md) · [markdown-mapping.md](markdown-mapping.md) · [module-namespace.md](module-namespace.md) · [objects.md](objects.md) · [ext-abi.md](ext-abi.md) · [ext-cli.md](ext-cli.md) · [stdlib-i18n.md](stdlib-i18n.md) · [web-assets-and-images.md](web-assets-and-images.md) · **[ext-web-request-context.md](ext-web-request-context.md)** |
|
||||
| 安装(目标) | `marqdo ext add web`(中英:`web` / `网页`) |
|
||||
| 本文目的 | 锁定**作者面**与**类 API**;指导实现 |
|
||||
| 本文目的 | **历史**类 API 与能力矩阵;新站请勿按本文 `compose_*` / `app.configure` 写作 |
|
||||
|
||||
> **2026-09-28**:作者面迁移到 Document / Endpoint / Resource。C1 已由 ADR 0007 修订。下文保留作迁移对照,**不是**现行推荐 API。
|
||||
|
||||
---
|
||||
|
||||
## 0. 一句话
|
||||
|
||||
`ext/web` 用 **GFM 表格 + Marqdo 类方法** 描述动态网站:表仍是字典/列表嵌套;类方法把表装配成可渲染页面。
|
||||
**不改 Marqdo 核心语法**;扩展库存在的理由,正是让「文档即代码」在网站场景下成立。
|
||||
**现行**:`.mq.md` Artifact(`type: web|endpoint`)描述页面与交互;见 [ext-web-artifact.md](ext-web-artifact.md)。
|
||||
|
||||
**历史(废止)**:`ext/web` 曾用 GFM 表格 + `# page`/`# app` 类方法拼装站点。该模型因 God Object / 碎片 DSL / 隐式路由被破坏性替换。
|
||||
|
||||
---
|
||||
|
||||
@@ -41,13 +44,14 @@
|
||||
|
||||
| # | 约束 |
|
||||
|---|------|
|
||||
| C1 | **禁止**为 web 改动 Marqdo 核心:单元格求值、import、点号值读取等保持不变 |
|
||||
| C2 | 作者面**禁止** `json.parse` / `json.set` 袋胶水、手写 part JSON、手写 assemble 袋 |
|
||||
| C3 | 能力以 **`#` 类 + `##` 方法** 暴露;**英文库文件只用英文名,中文库文件只用中文名**(见 [stdlib-i18n.md](stdlib-i18n.md));同一 `.mq.md` 内禁止中英 API 混排 |
|
||||
| C4 | 首页在 **`index.mq.md`**:页面表在此;可复用片段进 `components/`;主体可就地写 |
|
||||
| C5 | **旧有网络扩展库全部废弃**:`plugins/web`、`ext/web/**`(含现网示例/gold 中依赖旧 API 者)评审通过后**删除或清空后重写**;禁止在旧代码上打补丁演进。**落实**:原生层 Go 全量重写见 [ext-web-go-rewrite.md](ext-web-go-rewrite.md);作者面 ABI 名保持兼容 |
|
||||
| C6 | `# db` / `# 数据库` 必须提供完整**增删改查**;列表查询英文方法名为 **`select`**(中文 **`查询`**);后台 `/admin` 与表单提交走同一套写库 API |
|
||||
| C7 | 用户输入经 `# form` / `# 表单`:字段表 + 校验表;**服务端校验必须在写库前执行**(见 §5.5) |
|
||||
| C1 | **修订(ADR 0007)**:核心可识别 Artifact `type: web\|endpoint`(及 ZH);**不**新增 CORE_CONSTRUCTS 标记;单元格求值 / import 句法不变;装配语义仍在 ext + plugin。旧文「禁止为 web 改核心」作废 |
|
||||
| C2 | 作者面**禁止** `json.parse` / `json.set` 袋胶水(现行见 ext-web-artifact A2) |
|
||||
| C3 | 中英分文件;同文件禁止混 API(现行 A3) |
|
||||
| C4 | **废止**「页面表 + compose」心智 → Document Artifact + `pages/` · `api/` |
|
||||
| C5 | 破坏性重写作者面(ADR 0007);原生 Go 演进见 [ext-web-go-rewrite.md](ext-web-go-rewrite.md) |
|
||||
| C6 | DB 迁至 **`ext/data`**;CRUD 仍须完整(`select` / `查询`) |
|
||||
| C7 | 表单校验仍须在写库前执行;挂载经 declared Endpoint / system `/_form`(inspect 可见) |
|
||||
| A1–A8 | **现行硬约束**见 [ext-web-artifact.md](ext-web-artifact.md) §9 |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -616,3 +616,4 @@ FnDef.params:
|
||||
| 2026-09-13 | 决议空返回(§7.2)与提及性反引号(§4.4);形参改为「升参」与绑定图分离(§8.2) |
|
||||
| 2026-09-14 | §7.2:`---` 不再收束;返回语句结束函数体解析;空返回首选 `*None*` |
|
||||
| 2026-09-14 | 取元主语法改为 `[键](集合)`;脚注取元过渡保留 |
|
||||
| 2026-09-24 | Artifact Metadata Binding(`${…}` Phase 1,仅 frontmatter):见 [binding.md](binding.md) · ADR 0006 |
|
||||
@@ -130,7 +130,7 @@ Markdown **标记即语法**。输出**不是**语言架构原语,而是内置
|
||||
|
||||
## 5. Frontmatter 与跨文件
|
||||
|
||||
文件头 `---` 对为元信息。其中 `import bind:path.mq.md` / `导入` 为**文件导入**;`import bind:lib.member` 为**短名**(废除独立 `use` 与 frontmatter `>` 导入)。调用经裸点号路径(`time.parse`、`agent.agent`);实例方法在独立 `>` 调用须 `` > `var`.method ``(`*…*` 段内接收者裸名)。详见 [module-namespace.md](module-namespace.md)。
|
||||
文件头 `---` 对为 **Artifact Metadata**。其中 `import bind:path.mq.md` / `导入` 为**文件导入**;`import bind:lib.member` 为**短名**(废除独立 `use` 与 frontmatter `>` 导入)。其它 `key: value` 可为字面量或 **Binding**(`${env.NAME}` 等,加载期解析);见 [binding.md](binding.md)。调用经裸点号路径(`time.parse`、`agent.agent`);实例方法在独立 `>` 调用须 `` > `var`.method ``(`*…*` 段内接收者裸名)。详见 [module-namespace.md](module-namespace.md)。
|
||||
|
||||
**函数体内的** `---` / `***` 不解释为 frontmatter(见 §11)。
|
||||
|
||||
|
||||
+10
-5
@@ -143,13 +143,18 @@ Marqdo 对齐的判断是:
|
||||
|
||||
| `type` | 状态 | 含义 | 主要生产者 |
|
||||
|--------|------|------|------------|
|
||||
| `Marqdo Catalog` | **v0 已实现** | 包总览(`catalog.yaml` / `index.md`) | `marqdo catalog` |
|
||||
| `Marqdo Module` | **v0 已实现** | 一个 `.mq.md` 模块的投影 | `marqdo catalog` |
|
||||
| `Marqdo Task` | **规范已定;实现待做** | 一项可匹配的任务意图 | `plan` 晋升 |
|
||||
| `Marqdo Agent Skill` | **规范已定;实现待做** | 指向可执行工作簿的技能 | `plan` 晋升 |
|
||||
| `Marqdo Catalog` | **已实现** | 包总览(`catalog.yaml` / `index.md`) | `marqdo catalog` |
|
||||
| `Marqdo Module` | **已实现** | 一个 `.mq.md` 模块的投影 | `marqdo catalog` |
|
||||
| `Marqdo Task` | **已实现** | 一项可匹配的任务意图 | `plan` 晋升 |
|
||||
| `Marqdo Agent Skill` | **已实现** | 指向可执行工作簿的技能 | `plan` 晋升 |
|
||||
| `Capability` / `Function` | **EKC** | 工程能力 / 符号投影 | `marqdo knowledge` |
|
||||
| `Decision` / `Constraint` / `Failure` / `AntiPattern` / `Pattern` / `Migration` / `Fact` | **EKC** | L4 工程知识 | 源 frontmatter + learn |
|
||||
| `Marqdo Context Pack` | **EKC** | Agent EFI 上下文制品 | `reuse` / `preflight` |
|
||||
| `Marqdo Policy` | **EKC** | 复用决议编译 | `marqdo reuse` |
|
||||
| `Marqdo Engineering Knowledge` | **EKC** | 工程知识总览 | `engineering.yaml` / `index.mq.md` |
|
||||
| 其他 | 开放 | 第三方/未来扩展 | — |
|
||||
|
||||
**不**引入 Google OKF 的 `Metric` / BigQuery 表类型作为 Marqdo 核心词汇;需要时可由用户包自行使用,运行时降级。
|
||||
完整 EKC 规范见 [engineering-knowledge.md](engineering-knowledge.md)。
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -2,16 +2,23 @@
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 状态 | **Accepted · W0–W7 + P3 完结**(2026-08-28 复核) |
|
||||
| 日期 | 2026-08-26(初稿)· 2026-08-28(完结复核) |
|
||||
| 相关 | [ext-web.md](ext-web.md) · [ext-web-net.md](ext-web-net.md) · [ext-abi.md](ext-abi.md) · [stdlib-modules.md](stdlib-modules.md) |
|
||||
| 目标 | 盘点 `ext/web` + `plugins/web` + `lib/net` 现状,对照主流语言网络栈,给出「开发一个完整 Web 项目」所需能力的差距清单与补强路线 |
|
||||
| 状态 | **Historical capability matrix** — 作者面以 **[ext-web-artifact.md](ext-web-artifact.md)** / ADR [0007](../adr/0007-web-document-endpoint.md) 为准(2026-09-28) |
|
||||
| 日期 | 2026-08-26(初稿)· 2026-08-28(完结复核)· 2026-09-28(Artifact 标注) |
|
||||
| 相关 | [ext-web-artifact.md](ext-web-artifact.md) · [ext-web.md](ext-web.md) · [ext-web-net.md](ext-web-net.md) · [ext-abi.md](ext-abi.md) · [stdlib-modules.md](stdlib-modules.md) |
|
||||
| 目标 | 盘点 `ext/web` + `plugins/web` + `lib/net` 现状;**下文 API 名多为 W0–W7 历史面**(`compose_*` / `configure` 已废止) |
|
||||
|
||||
> **2026-09-28**:作者面迁移到 Document / Endpoint / Resource。
|
||||
> - Document/Endpoint:`type: web|endpoint` + `web.serve` / `web.render` / `web.inspect`
|
||||
> - DB/cache/storage → `ext/data`;auth/oidc/rbac → `ext/security`;ws → `ext/net`
|
||||
> - 统一 View:`|type|slot|value|attrs|style|`;中间件:`web.use`
|
||||
> 本文件保留为**原生能力矩阵**对照,勿再按 `compose_*` / `app.configure` 写作。
|
||||
|
||||
---
|
||||
|
||||
## 0. 一句话结论
|
||||
|
||||
**`ext/web` + `plugins/web` 已完成 W0–W7 + P3 路线图**:中间件/JSON API、数据层(连接池/事务/分页/FTS/迁移/外键/审计时间戳)、安全硬化(argon2/CSRF/会话持久化/限速/RBAC)、内容站点标配(SEO/RSS/Markdown/分页 UI)、上传下载/相册/ETag、sitemap/robots/错误页/重定向、WebSocket 广播与访问日志。足以在 Marqdo 上实现博客/CMS/中小型 API 站点;**未内置**的仅剩标签页模板(D6,可用路由+`db.count` 自建)与应用层反垃圾。边界不变:**纯解析进 `lib/net`,HTTP 服务器与领域能力进 `plugins/web` + `ext/web` 作者面**。
|
||||
**能力层(plugins/web)**:中间件/JSON API、数据层、安全、内容站点、上传下载、sitemap、WebSocket 等仍在原生实现中。
|
||||
**作者面(2026-09-28)**:改为 Artifact Metadata + facade;`compose_*` / `configure` **已删除**。边界不变:**纯解析进 `lib/net`,HTTP 服务器进 `plugins/web`,作者 API 进 `ext/web` + `ext/data` + `ext/security` + `ext/net`**。
|
||||
|
||||
**宿主集成下一刀**(同域 LLM SSE 中继、HTTP→用户 `##`、与 MCP Server 协作)不在本 W 表内,见 [ext-hosting-gaps.md](../research/ext-hosting-gaps.md) · [ext-hosting-fill.md](ext-hosting-fill.md) · [ext-hosting.md](../roadmap/ext-hosting.md)。
|
||||
|
||||
|
||||
+2626
File diff suppressed because it is too large.
Load diff
+541
@@ -0,0 +1,541 @@
|
||||
有时候可能用户不太好接入Jev,所以阶段4应该要允许用户使用小参数的模型代替,你认为呢?
|
||||
|
||||
我非常赞同,而且我甚至建议**不要把 Jev 作为阶段 4 的必选组件**。
|
||||
|
||||
更准确的设计应该是:
|
||||
|
||||
> **Jev 是一种可选的决策器(Decision Engine),小参数模型是另一种可选实现;Marqdo Agent 本身只规定“需要一个 Router/Policy”,而不规定 Router 必须由什么实现。**
|
||||
|
||||
这样会让整个框架开放很多。
|
||||
|
||||
### 1. 阶段 4 应该改成“Adaptive Routing”
|
||||
|
||||
原来的:
|
||||
|
||||
```text
|
||||
Phase 4
|
||||
↓
|
||||
Jev Router
|
||||
```
|
||||
|
||||
我建议改成:
|
||||
|
||||
```text
|
||||
Phase 4: Adaptive Routing
|
||||
│
|
||||
┌─────────┼─────────┐
|
||||
↓ ↓ ↓
|
||||
Marqdo Rule Jev Small LLM
|
||||
│ │ │
|
||||
└─────────┼─────────┘
|
||||
↓
|
||||
Route Policy
|
||||
```
|
||||
|
||||
甚至还可以允许:
|
||||
|
||||
```text
|
||||
Rule
|
||||
Small Model
|
||||
Jev
|
||||
Full LLM
|
||||
```
|
||||
|
||||
混合使用。
|
||||
|
||||
---
|
||||
|
||||
# 2. 为什么这个设计更合理
|
||||
|
||||
因为用户真正需要的不是:
|
||||
|
||||
> “我要接入 Jev。”
|
||||
|
||||
而是:
|
||||
|
||||
> “我希望 Agent 学会以后,不要每次都调用大模型。”
|
||||
|
||||
这两个概念应该分开。
|
||||
|
||||
所以 Marqdo 只定义接口:
|
||||
|
||||
```text
|
||||
agent.route(task, candidates)
|
||||
```
|
||||
|
||||
至于内部怎么实现,可以是:
|
||||
|
||||
### A. 纯 Marqdo
|
||||
|
||||
```text
|
||||
if task.type == "json":
|
||||
return json-repair
|
||||
|
||||
if task.type == "search":
|
||||
return corpus-search
|
||||
```
|
||||
|
||||
完全不需要模型。
|
||||
|
||||
---
|
||||
|
||||
### B. 小参数模型
|
||||
|
||||
比如用户已经有:
|
||||
|
||||
```text
|
||||
Qwen 0.5B
|
||||
Qwen 1.5B
|
||||
Llama 1B
|
||||
某个本地模型
|
||||
```
|
||||
|
||||
就可以:
|
||||
|
||||
```text
|
||||
task
|
||||
↓
|
||||
small model
|
||||
↓
|
||||
"json-repair"
|
||||
```
|
||||
|
||||
甚至可以让用户直接配置:
|
||||
|
||||
```yaml
|
||||
router:
|
||||
type: llm
|
||||
|
||||
provider: openai-compatible
|
||||
base_url: ...
|
||||
api_key: ...
|
||||
model: ...
|
||||
```
|
||||
|
||||
这和你之前设计 AI 斗蛐蛐时提出的 **OpenAI-compatible URL + Key** 思路其实也很契合。
|
||||
|
||||
---
|
||||
|
||||
### C. Jev
|
||||
|
||||
如果用户安装了 Jev:
|
||||
|
||||
```text
|
||||
task
|
||||
↓
|
||||
Jev
|
||||
↓
|
||||
policy
|
||||
```
|
||||
|
||||
那么就用 Jev。
|
||||
|
||||
---
|
||||
|
||||
### D. 大模型
|
||||
|
||||
甚至最开始没有任何 Router:
|
||||
|
||||
```text
|
||||
task
|
||||
↓
|
||||
large LLM
|
||||
↓
|
||||
decide
|
||||
```
|
||||
|
||||
也是合法的。
|
||||
|
||||
所以:
|
||||
|
||||
> **Router 是抽象能力,不是具体模型。**
|
||||
|
||||
---
|
||||
|
||||
# 3. 更重要的是:Router 本身也可以被学习
|
||||
|
||||
这里我觉得还能进一步提升你的设计。
|
||||
|
||||
一开始:
|
||||
|
||||
```text
|
||||
Task
|
||||
↓
|
||||
Large LLM
|
||||
↓
|
||||
选择 skill
|
||||
```
|
||||
|
||||
系统观察:
|
||||
|
||||
```text
|
||||
task A → json-repair
|
||||
task B → json-repair
|
||||
task C → json-repair
|
||||
task D → corpus-search
|
||||
```
|
||||
|
||||
之后可以自动生成:
|
||||
|
||||
```text
|
||||
router.mq.md
|
||||
```
|
||||
|
||||
例如:
|
||||
|
||||
```markdown
|
||||
---
|
||||
type: policy
|
||||
status: compiled
|
||||
---
|
||||
|
||||
## route
|
||||
|
||||
1. `task.type` == "json"
|
||||
*"json-repair"*
|
||||
|
||||
2. `task.type` == "search"
|
||||
*"corpus-search"*
|
||||
|
||||
3. *
|
||||
*"llm"*
|
||||
```
|
||||
|
||||
那么你甚至不需要 Jev。
|
||||
|
||||
**Marqdo 自己的控制流就是 Router。**
|
||||
|
||||
---
|
||||
|
||||
# 4. 所以我会重新设计 Cost Ladder
|
||||
|
||||
之前的:
|
||||
|
||||
```text
|
||||
L0 deterministic
|
||||
L1 rule
|
||||
L2 small model
|
||||
L3 small LLM
|
||||
L4 large LLM
|
||||
L5 exploration
|
||||
```
|
||||
|
||||
我建议改成:
|
||||
|
||||
```text
|
||||
┌────────────────────────┐
|
||||
│ Adaptive Router │
|
||||
└───────────┬────────────┘
|
||||
│
|
||||
┌─────────────────┼─────────────────┐
|
||||
↓ ↓ ↓
|
||||
Deterministic Small Model Large LLM
|
||||
│ │ │
|
||||
↓ ↓ ↓
|
||||
L0/L1 L2/L3 L4
|
||||
│
|
||||
↓
|
||||
Exploration L5
|
||||
```
|
||||
|
||||
这里最关键的是:
|
||||
|
||||
> **L2/L3 不应该规定具体模型。**
|
||||
|
||||
可以是:
|
||||
|
||||
```text
|
||||
Marqdo rules
|
||||
OR
|
||||
Jev
|
||||
OR
|
||||
0.5B model
|
||||
OR
|
||||
1B model
|
||||
OR
|
||||
3B model
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# 5. 我甚至建议用户可以配置多个 Router
|
||||
|
||||
例如:
|
||||
|
||||
```yaml
|
||||
router:
|
||||
candidates:
|
||||
|
||||
- name: rules
|
||||
type: marqdo
|
||||
|
||||
- name: local
|
||||
type: llm
|
||||
model: qwen-0.5b
|
||||
|
||||
- name: jev
|
||||
type: jev
|
||||
|
||||
- name: cloud
|
||||
type: llm
|
||||
model: ...
|
||||
```
|
||||
|
||||
然后 Agent 自己选择:
|
||||
|
||||
```text
|
||||
Task
|
||||
↓
|
||||
Router Resolver
|
||||
↓
|
||||
rules?
|
||||
↓
|
||||
confidence insufficient
|
||||
↓
|
||||
small model?
|
||||
↓
|
||||
confidence insufficient
|
||||
↓
|
||||
Jev?
|
||||
↓
|
||||
confidence insufficient
|
||||
↓
|
||||
large LLM
|
||||
```
|
||||
|
||||
这就形成一个**分级决策系统**。
|
||||
|
||||
---
|
||||
|
||||
# 6. 最漂亮的地方:Router 也可以被编译
|
||||
|
||||
最终甚至可以:
|
||||
|
||||
```text
|
||||
第一次:
|
||||
|
||||
LLM
|
||||
↓
|
||||
决定调用 Skill A
|
||||
|
||||
第二次:
|
||||
|
||||
LLM
|
||||
↓
|
||||
决定调用 Skill A
|
||||
|
||||
第三次:
|
||||
|
||||
LLM
|
||||
↓
|
||||
决定调用 Skill A
|
||||
|
||||
Learning
|
||||
↓
|
||||
发现稳定模式
|
||||
|
||||
Compile
|
||||
↓
|
||||
router.mq.md
|
||||
```
|
||||
|
||||
以后:
|
||||
|
||||
```text
|
||||
Task
|
||||
↓
|
||||
router.mq.md
|
||||
↓
|
||||
skill A
|
||||
↓
|
||||
execute
|
||||
```
|
||||
|
||||
**0 LLM。**
|
||||
|
||||
于是整个 Agent 的学习就不只是:
|
||||
|
||||
```text
|
||||
学习如何解决任务
|
||||
```
|
||||
|
||||
而是同时学习:
|
||||
|
||||
```text
|
||||
1. 学习如何解决任务
|
||||
2. 学习什么时候使用什么技能
|
||||
3. 学习什么时候需要大模型
|
||||
```
|
||||
|
||||
这三个层次组合起来,我认为比单纯引入 Jev 更有意思。
|
||||
|
||||
---
|
||||
|
||||
# 7. 最终可以形成三级编译
|
||||
|
||||
我现在反而觉得你的 Agent Framework 可以把整个“学习”定义成三个阶段:
|
||||
|
||||
```text
|
||||
Experience
|
||||
↓
|
||||
Skill Compilation
|
||||
↓
|
||||
Policy Compilation
|
||||
↓
|
||||
Execution Compilation
|
||||
```
|
||||
|
||||
### 第一层:Skill Compilation
|
||||
|
||||
```text
|
||||
LLM experience
|
||||
↓
|
||||
skill.mq.md
|
||||
```
|
||||
|
||||
解决:
|
||||
|
||||
> **怎么做?**
|
||||
|
||||
### 第二层:Policy Compilation
|
||||
|
||||
```text
|
||||
many task → skill mappings
|
||||
↓
|
||||
policy.mq.md
|
||||
```
|
||||
|
||||
解决:
|
||||
|
||||
> **什么时候做什么?**
|
||||
|
||||
### 第三层:Execution Compilation
|
||||
|
||||
```text
|
||||
skill + policy
|
||||
↓
|
||||
pure Marqdo execution
|
||||
```
|
||||
|
||||
解决:
|
||||
|
||||
> **能不能完全不思考直接执行?**
|
||||
|
||||
这时候整个 Agent 就变成:
|
||||
|
||||
```text
|
||||
Large LLM
|
||||
│
|
||||
Exploration
|
||||
↓
|
||||
Experience
|
||||
↓
|
||||
┌───────────────┐
|
||||
│ Skill Compile │
|
||||
└───────┬───────┘
|
||||
↓
|
||||
Skill MQ
|
||||
│
|
||||
┌───────┴───────┐
|
||||
↓ ↓
|
||||
Policy Compile Direct Execute
|
||||
↓
|
||||
Router MQ
|
||||
│
|
||||
↓
|
||||
Skill Selection
|
||||
│
|
||||
↓
|
||||
Execute
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# 8. 因此 Phase 4 我建议正式改名
|
||||
|
||||
不要叫:
|
||||
|
||||
> Jev Router
|
||||
|
||||
而叫:
|
||||
|
||||
> **Adaptive Policy / Routing Layer**
|
||||
|
||||
然后提供四种 backend:
|
||||
|
||||
```text
|
||||
router.backend = marqdo
|
||||
router.backend = small-llm
|
||||
router.backend = jev
|
||||
router.backend = llm
|
||||
```
|
||||
|
||||
甚至:
|
||||
|
||||
```text
|
||||
router.backend = auto
|
||||
```
|
||||
|
||||
让 Agent 根据可用环境自动选择。
|
||||
|
||||
---
|
||||
|
||||
## 9. 用户体验也会好很多
|
||||
|
||||
用户第一次安装 Marqdo:
|
||||
|
||||
```text
|
||||
marqdo agent run ...
|
||||
```
|
||||
|
||||
没有 Jev,也没有小模型。
|
||||
|
||||
完全没问题:
|
||||
|
||||
```text
|
||||
→ 使用配置的大模型
|
||||
```
|
||||
|
||||
用户后来配置:
|
||||
|
||||
```yaml
|
||||
router:
|
||||
backend: small-llm
|
||||
provider: openai-compatible
|
||||
model: qwen-0.5b
|
||||
```
|
||||
|
||||
Agent 就自动变成:
|
||||
|
||||
```text
|
||||
大模型:负责复杂推理 / 学习
|
||||
小模型:负责日常路由
|
||||
Marqdo:负责确定性执行
|
||||
```
|
||||
|
||||
如果用户安装 Jev:
|
||||
|
||||
```yaml
|
||||
router:
|
||||
backend: jev
|
||||
```
|
||||
|
||||
则:
|
||||
|
||||
```text
|
||||
Jev:负责路由
|
||||
大模型:负责复杂任务
|
||||
Marqdo:负责执行
|
||||
```
|
||||
|
||||
**用户完全不需要改变 Agent 的上层 API。**
|
||||
|
||||
---
|
||||
|
||||
我认为这个调整非常重要,因为它让 **Marqdo Agent 的核心创新从“依赖某个新模型”变成“模型无关的认知编译架构”**。
|
||||
|
||||
最终可以浓缩成一句话:
|
||||
|
||||
> **大模型负责探索,小模型/Jev 负责决策,Marqdo 负责执行;随着经验积累,探索和决策逐渐被编译成可执行的 `.mq.md`。**
|
||||
|
||||
这其实比“接入 Jev”本身更符合你最开始提出的 **Think → Execute → Learn → Compile → Reuse** 路线。
|
||||
+1173
File diff suppressed because it is too large.
Load diff
+1335
File diff suppressed because it is too large.
Load diff
+2182
File diff suppressed because it is too large.
Load diff
+2818
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,77 @@
|
||||
# Marqdo AI 栈统一开发方案(llm + agent)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Status | **Active** |
|
||||
| Date | 2026-09-24 |
|
||||
| Source | [doc/next/002–005](../next/) · [ext-llm.md](../design/ext-llm.md) · [ext-agent.md](../design/ext-agent.md) |
|
||||
|
||||
## 1. 一句话
|
||||
|
||||
> **Document → Prompt → LLM → Agent → Skill → Policy → Executable Knowledge**
|
||||
|
||||
大模型负责探索,小模型(`router_model` / 任意 llm 句柄)负责决策,Marqdo 负责执行;经验编译进 `.mq.md` 后推理成本下降。
|
||||
|
||||
## 2. 分层(勿混)
|
||||
|
||||
| 层 | 职责 | 落点 |
|
||||
|----|------|------|
|
||||
| **Intelligence primitive** | `ask` / `stream` / `collect`;结果对象 | `ext/ai/llm` |
|
||||
| **Agent runtime** | `run` / `step` / `plan` / `route` | `ext/ai/agent` |
|
||||
| **Memory / compile** | episode → skill → policy | `plugins/agent`(ABI;**不进** `src/host`) |
|
||||
| **Transport** | HTTP / SSE / provider | `ext/ai/llm/openai` · `ollama`;用户不可见 |
|
||||
|
||||
硬规则:`ext/**` 不调 `host_*`;agent/OKF/路由热路径只进 `plugins/agent`。
|
||||
|
||||
## 3. LLM(005)— 已/待
|
||||
|
||||
| 项 | 状态 |
|
||||
|----|------|
|
||||
| 语义 API:`ask` / `stream` / `collect`(`complete` 降为兼容原语) | **done** |
|
||||
| `create` 工厂;handle 带 `backend` | **done** |
|
||||
| LLMResult:`text` / `model` / `finish` / `backend` / `usage` / `tokens` | **done** |
|
||||
| Named models(`llm.fast` / `llm.reasoning`) | **done** |
|
||||
| Prompt document:`prompt` / `prompt_load` + `type: prompt` 正文 | **done**(无参 `ask`/`stream` → `sys.module_source` + `prompt_body`) |
|
||||
| 拆 `ext/ai/llm/openai.mq.md` + `ollama` 后端 | **done** |
|
||||
| tool_calls 完整 Result | **done**(`openai.tool_calls_from` + `LLMResult.tool_calls`;`llm-tool-calls-offline`) |
|
||||
| Transport → 可选 ABI 插件 (`plugins/llm`) | **done** |
|
||||
| Artifact Metadata Binding(`${env.*}` / `secret` · `sys.meta`) | **done**(Phase 1 Metadata-only + Phase 2 scope lift;[binding.md](../design/binding.md)) |
|
||||
|
||||
作者面优先:
|
||||
|
||||
```marqdo
|
||||
**answer = > llm.ask prompt="What is Marqdo?"**
|
||||
**events = > model.stream prompt=`p`**
|
||||
**text = > llm.collect events=`events`**
|
||||
**fast = > llm.fast**
|
||||
**prompt = > llm.prompt path="prompts/task.md"**
|
||||
```
|
||||
|
||||
## 4. Agent(002–004)— 已/待
|
||||
|
||||
| 项 | 状态 |
|
||||
|----|------|
|
||||
| Skill Compilation:episode → maybe_learn → `llm_free` | **done** |
|
||||
| Adaptive Routing:`marqdo` \| `small-llm` \| `llm` \| `auto`(`jev`→`small-llm` 别名) | **done** |
|
||||
| Policy Compilation → `policies/router.mq.md` | **done** |
|
||||
| `run` 串联 route → execute → record → learn | **done** |
|
||||
| Agent 默认走 `model.ask` + Result.usage 记成本 | **done**(`model_turn`) |
|
||||
| Prompt artifact / document-as-prompt | **done**(经 llm `path=` / `prompt`) |
|
||||
| 零 LLM 执行(policy+skill → `llm_calls=0`) | **done** |
|
||||
| `plan` 不覆盖已有 `llm_free` resource | **done** |
|
||||
| 真 MCP client(stdio) / resume checkpoint / 小模型路由请求面 | **done**(live 路由仍可选 skip) |
|
||||
|
||||
## 5. 联调顺序
|
||||
|
||||
1. **LLM Result → Agent metrics** — **done**
|
||||
2. **Named handles** — **done**
|
||||
3. **Prompt document** — **done**(`type: prompt` 剥 frontmatter;无参 `ask`/`stream` 经 `sys.module_source`)
|
||||
4. **Backend split** — **done**
|
||||
5. **Execution compilation** — **done**(`agent-zero-llm` 金样 + harness)
|
||||
6. **Metadata Binding** — **done**(Phase 1 + Phase 2 scope lift;`llm-meta-offline` · `meta-lift`)
|
||||
|
||||
## 6. 验收金样
|
||||
|
||||
- Agent:`tests/ext/agent-v2-learn.mq.md` · `agent-p4-route.mq.md` · `agent-zero-llm.mq.md` · `agent-plan-preserve.mq.md` · `agent-mcp-client.mq.md` · `agent-resume.mq.md` · `agent-small-llm-route.mq.md` · `scripts/agent-harness.sh`
|
||||
- LLM:`tests/ext/llm-ask-offline.mq.md` · `llm-stream-offline.mq.md` · `llm-ctor-offline.mq.md` · `llm-named-offline.mq.md` · `llm-meta-offline.mq.md` · `llm-entry-prompt-offline.mq.md` · `llm-tool-calls-offline.mq.md`
|
||||
- Binding:`tests/structure/meta-binding.mq.md` · `meta-lift.mq.md`
|
||||
@@ -39,6 +39,12 @@
|
||||
| **B0** | 官方示例演示 call_site/源码/OKF;Skill/文档;上下文管道可读 | **done** |
|
||||
| **B0.5** | 本机 live(`.env` → 开发模型)验证「模型用上了文档」 | **done**(`AGENT_LIVE` / `AGENT_HARNESS_LIVE`) |
|
||||
| **B1** | 评测 harness 锁差异化指标(constitution / dump / writeback 遮蔽) | **Active** |
|
||||
| **B2–B5** | 真 MCP / resume+HITL / stats / 并行 | 暂缓 |
|
||||
| **B1.5** | **Skill Compilation v2(Phase 1–3)**:episode → maybe_learn → compile → `llm_free`;`agent.run` / `resolve` / `metrics`;ABI 在 `plugins/agent`(`memory.rs`);金样 `agent-v2-learn` | **done** |
|
||||
| **B1.6 / P4** | **Adaptive Routing**(原 Jev Router 更名):`route` backends = `marqdo` \| `small-llm` \| `jev?` \| `llm` \| `auto`;Policy Compilation → `policies/router.mq.md`;金样 `agent-p4-route`;设计 [004.md](../next/004.md) | **done** |
|
||||
| **L0** | **LLM semantic surface(005)**:`ask` / `stream` / `collect` / `create`;`complete` 降为兼容;统一方案 [ai-stack.md](ai-stack.md) | **done** |
|
||||
| **B2** | 真 MCP stdio client(`agent_mcp_client` + fixture 仍作离线证据) | **done** |
|
||||
| **B3** | resume checkpoint(`.marqdo/agent-resume/`;HITL view 仍暂缓) | **done**(HITL 暂缓) |
|
||||
| **B4–B5** | stats / 并行;小模型 live 路由金样(无 key skip) | 部分 |
|
||||
| — | LLM named models / prompt-document | **done** |
|
||||
|
||||
示例:`examples/agent-pong/` · `examples/agent-okf-flywheel/`。
|
||||
示例:`examples/agent-pong/` · `examples/agent-okf-flywheel/`。v2 设计:[doc/next/002.md](../next/002.md) · [doc/next/003.md](../next/003.md) · P4:[doc/next/004.md](../next/004.md) · LLM:[doc/next/005.md](../next/005.md)。
|
||||
+85
-78
@@ -1,19 +1,22 @@
|
||||
# 第 14 章:网络与 Web 扩展
|
||||
|
||||
Marqdo 的网络能力分两层:**标准库 `lib/net`**(HTTP 客户端、cookie / multipart 解析)与**官方扩展 `ext/web`**(网页应用、数据库、表单、登录鉴权、WebSocket)。分层的依据是「代码即文档」的可读性——基础、通用、低依赖的进标准库;领域性强、依赖重、只在 Web 场景才用的进扩展。
|
||||
> **现行作者面(2026-09-28 · ADR 0007)**:Document / Endpoint / Resource —— 见 [ext-web-artifact.md](../design/ext-web-artifact.md)。
|
||||
> 下文若仍出现 `compose_*` / `app.configure` / `web.db` 同文件混写,视为**历史示例**;DB → `ext/data`,鉴权 → `ext/security`,实时 → `ext/net`。
|
||||
|
||||
Marqdo 的网络能力分两层:**标准库 `lib/net`**(HTTP 客户端、cookie / multipart 解析)与**官方扩展 `ext/web`**(Document/Endpoint facade + 拆分后的 data/security/net)。分层的依据是「代码即文档」。
|
||||
|
||||
## 14.1 概览
|
||||
|
||||
| 能力 | 位置 | 典型函数 |
|
||||
| 能力 | 位置 | 典型 API |
|
||||
|------|------|----------|
|
||||
| HTTP(S) 客户端 | `lib/net` | `http_get` / `http_post` / `http_request` |
|
||||
| Cookie 解析 | `lib/net` | `cookie_parse` |
|
||||
| multipart/form-data 解析 | `lib/net` | `multipart_parse` |
|
||||
| 网页应用 / 路由 / 静态 | `ext/web` | `web.page` / `web.app` |
|
||||
| SQLite 数据库 | `ext/web` | `web.db` |
|
||||
| 表单 | `ext/web` | `web.form` |
|
||||
| 登录鉴权(session) | `ext/web` | `web.auth` / `app.auth` |
|
||||
| WebSocket 客户端 / 端点 | `ext/web` | `web.ws` / `app.route_ws` |
|
||||
| Cookie / multipart | `lib/net` | `cookie_parse` / `multipart_parse` |
|
||||
| Document / Endpoint | `ext/web` | `type: web\|endpoint` · `web.serve` · `web.render` · `web.inspect` |
|
||||
| View 装配 | `ext/web` | `|type|slot|value|attrs|style|` → `web.render` |
|
||||
| 数据库 | `ext/data` | `db` CRUD / txn |
|
||||
| 表单 | `ext/data` | `form` |
|
||||
| 鉴权 / RBAC / OIDC | `ext/security` | `auth` · `rbac` · `oidc` |
|
||||
| WebSocket | `ext/net` | `websocket` |
|
||||
|
||||
```bash
|
||||
# 运行使用 ext/web 的程序前,先安装扩展
|
||||
@@ -67,107 +70,111 @@ import net:lib/net.mq.md
|
||||
> print text=[name](`field`)
|
||||
```
|
||||
|
||||
## 14.3 网页应用(`ext/web`)
|
||||
## 14.3 网页应用(Artifact · ADR 0007)
|
||||
|
||||
`ext/web` 用类 + 方法装配页面。导入 `web:ext/web/web.mq.md`(中文 `网页:ext/web/网页.mq.md`)。
|
||||
站点是一堆 `.mq.md`:**Document**(`type: web`)与 **Endpoint**(`type: endpoint`)。入口调用 `web.serve` 扫描目录并监听。完整设计见 [ext-web-artifact.md](../design/ext-web-artifact.md);示例 [marqdo-blog](../../examples/marqdo-blog/)。
|
||||
|
||||
### 页面、组件、主体
|
||||
### Document 首页
|
||||
|
||||
```markdown
|
||||
---
|
||||
type: web
|
||||
title: 我的站点
|
||||
route: /
|
||||
method: GET
|
||||
data_source: posts
|
||||
data_order: -created_at
|
||||
import web:ext/web/web.mq.md
|
||||
---
|
||||
|
||||
# 我的站点
|
||||
|
||||
欢迎。下方卡片列表由 `data_source` 在服务时从数据库加载。
|
||||
```
|
||||
|
||||
### Endpoint
|
||||
|
||||
```markdown
|
||||
---
|
||||
type: endpoint
|
||||
method: GET
|
||||
path: /api/ping
|
||||
request: none
|
||||
response: json
|
||||
---
|
||||
|
||||
# main
|
||||
|
||||
`首页` =
|
||||
`out` =
|
||||
|
||||
| 组件 | 样式 |
|
||||
|------|------|
|
||||
| nav.`导航` | shell.`顶栏` |
|
||||
| ok | service |
|
||||
|----|---------|
|
||||
| True | demo |
|
||||
|
||||
**page = > web.page title="我的站点" intro="<h1>你好</h1>"**
|
||||
**page = > `page`.compose_components components=`首页`**
|
||||
**html = > `page`.render**
|
||||
*out*
|
||||
```
|
||||
|
||||
### 服务入口
|
||||
|
||||
```markdown
|
||||
---
|
||||
import web:ext/web/web.mq.md
|
||||
import posts:db/posts.mq.md
|
||||
---
|
||||
|
||||
# main
|
||||
|
||||
**store = > posts.open url="sqlite:data/site.db"**
|
||||
> web.serve root="." host="127.0.0.1" port=18081 db=`store` static_dir="public"
|
||||
```
|
||||
|
||||
离线预览可用 View 表 + `web.render`:
|
||||
|
||||
```markdown
|
||||
`nodes` =
|
||||
|
||||
| type | slot | value | attrs | style |
|
||||
|------|------|-------|-------|-------|
|
||||
| title | title | Marqdo | | title |
|
||||
|
||||
**html = > web.render nodes=`nodes` title="preview"**
|
||||
> print text=`html`
|
||||
```
|
||||
|
||||
### 应用 + 路由 + 监听
|
||||
### 数据库(`ext/data`)
|
||||
|
||||
```markdown
|
||||
**app = > web.app page=`page` host=127.0.0.1 port=18081**
|
||||
**app = > `app`.route path="/about" page=`关于`**
|
||||
> web_listen app=`app`
|
||||
```
|
||||
---
|
||||
import data:ext/data/db.mq.md
|
||||
---
|
||||
|
||||
`listen` 提供 `/`、路由页、表单端点 `/_form/{id}`、可选静态目录与 `/admin`。
|
||||
# main
|
||||
|
||||
### 数据库(SQLite)
|
||||
|
||||
```markdown
|
||||
**store = > web.db url="sqlite:site.db"**
|
||||
**store = > data.db url="sqlite:site.db"**
|
||||
> `store`.init name=articles fields=`字段表`
|
||||
> `store`.insert table=articles rows=`数据`
|
||||
**rows = > `store`.select table=articles limit=10**
|
||||
- [行](rows)
|
||||
> print text=[title](`行`)
|
||||
```
|
||||
|
||||
## 14.4 登录鉴权(session / cookie)
|
||||
## 14.4 鉴权(`ext/security`)
|
||||
|
||||
给 `/admin*` 加登录门禁:`app.auth` 挂用户表,未登录重定向到 `/admin/login`。
|
||||
HTTP 门禁与登录走 `ext/security`(`auth` / `rbac` / `oidc`),再用 `web.use` 挂中间件表;勿再写 `app.configure`。独立脚本可用:
|
||||
|
||||
```markdown
|
||||
`管理员` =
|
||||
---
|
||||
import rbac:ext/security/rbac.mq.md
|
||||
---
|
||||
|
||||
| 行 | 用户名 | 密码 |
|
||||
|----|--------|------|
|
||||
| 1 | admin | secret |
|
||||
| 2 | 站长 | pw123 |
|
||||
# main
|
||||
|
||||
**app = > web.app page=`page` admin=True**
|
||||
**app = > `app`.auth users=`管理员` session_ttl=3600**
|
||||
> web_listen app=`app`
|
||||
**gate = > rbac.rbac**
|
||||
**r = > `gate`.can permissions="desk:access" needed="desk:access"**
|
||||
> print text=[allowed](`r`)
|
||||
```
|
||||
|
||||
`session_ttl` 是会话有效期(秒),过期后重新登录。
|
||||
## 14.5 WebSocket(`ext/net`)
|
||||
|
||||
**独立鉴权工具** `web.auth`:登录、校验、登出(不依赖页面),返回 `{ok, session_id, username}`:
|
||||
|
||||
```markdown
|
||||
**auth = > web.auth users=`管理员` session_ttl=3600**
|
||||
**login = > `auth`.login username="admin" password="secret"**
|
||||
1. [ok](`login`)
|
||||
> print text=登录成功:[username](`login`)
|
||||
2. *
|
||||
> print text=登录失败
|
||||
|
||||
**sid = [session_id](login)**
|
||||
**check = > `auth`.check session_id=`sid`**
|
||||
> print text=[username](`check`)
|
||||
|
||||
> `auth`.logout session_id=`sid`
|
||||
```
|
||||
|
||||
## 14.5 WebSocket
|
||||
|
||||
`app.route_ws` 注册端点,`web.ws.connect` 单次请求–响应:
|
||||
|
||||
```markdown
|
||||
**app = > `app`.route_ws path="/live" echo=True**
|
||||
> web_listen app=`app`
|
||||
```
|
||||
|
||||
客户端(另开终端):
|
||||
|
||||
```markdown
|
||||
**ws = > web.ws timeout_sec=30**
|
||||
**echo = > `ws`.connect url="ws://127.0.0.1:18081/live" message="hi"**
|
||||
1. [ok](`echo`)
|
||||
> print text=[messages](`echo`)
|
||||
2. *
|
||||
> print text=连接失败:[error](`echo`)
|
||||
```
|
||||
实时能力在 `ext/net`(如 `websocket`),与 Document 站点并列导入,不再挂在 `web.app` 的 junk drawer 上。
|
||||
|
||||
## 14.6 字典操作:用表格与取元,不用 `json.get`/`json.set`
|
||||
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
---
|
||||
title: resolve_config
|
||||
capability: config.resolve
|
||||
status: stable
|
||||
---
|
||||
|
||||
Use when: final normalized configuration is required.
|
||||
|
||||
# config
|
||||
|
||||
## resolve_config
|
||||
+ `path`
|
||||
|
||||
`defaults` =
|
||||
|
||||
| host | port |
|
||||
|------|------|
|
||||
| localhost | 8080 |
|
||||
|
||||
* defaults
|
||||
@@ -0,0 +1,21 @@
|
||||
---
|
||||
title: Engineering Knowledge demo
|
||||
description: Tiny project demonstrating Capability Catalog + reuse + decisions.
|
||||
---
|
||||
|
||||
# Engineering Knowledge demo
|
||||
|
||||
This folder is a miniature “large engineering” slice: config resolution already exists.
|
||||
Before generating a new loader, run:
|
||||
|
||||
```bash
|
||||
marqdo knowledge examples/engineering-knowledge -o examples/engineering-knowledge/.marqdo
|
||||
marqdo reuse "load and resolve configuration" examples/engineering-knowledge -o examples/engineering-knowledge/.marqdo
|
||||
marqdo duplicate examples/engineering-knowledge -o examples/engineering-knowledge/.marqdo
|
||||
```
|
||||
|
||||
Expected: **REUSE config.resolve**, not a new `load_configuration`.
|
||||
|
||||
# main
|
||||
|
||||
* "see README via catalog/reuse CLI"
|
||||
@@ -0,0 +1,18 @@
|
||||
---
|
||||
type: Decision
|
||||
id: ADR-022
|
||||
title: Cache is not persistence
|
||||
status: accepted
|
||||
applies_to:
|
||||
- config.resolve
|
||||
---
|
||||
|
||||
# Cache is not persistence
|
||||
|
||||
## Decision
|
||||
|
||||
Redis is only used for ephemeral cache.
|
||||
|
||||
## Reason
|
||||
|
||||
The application must remain correct when Redis is unavailable.
|
||||
@@ -0,0 +1,12 @@
|
||||
---
|
||||
type: Constraint
|
||||
id: C-014
|
||||
title: Centralized config resolution
|
||||
status: stable
|
||||
applies_to:
|
||||
- config.resolve
|
||||
---
|
||||
|
||||
# Centralized config resolution
|
||||
|
||||
All configuration resolution must go through `resolve_config`.
|
||||
@@ -0,0 +1,20 @@
|
||||
---
|
||||
type: Failure
|
||||
id: F-008
|
||||
title: Do not invent a second config loader
|
||||
status: resolved
|
||||
---
|
||||
|
||||
# Do not invent a second config loader
|
||||
|
||||
## Failed approach
|
||||
|
||||
Added `load_configuration` beside `resolve_config`.
|
||||
|
||||
## Failure
|
||||
|
||||
Divergent defaults and error handling.
|
||||
|
||||
## Correct approach
|
||||
|
||||
Call `resolve_config`.
|
||||
@@ -1,54 +1,39 @@
|
||||
# examples/marqdo-blog
|
||||
|
||||
一个用 Marqdo 的 `ext/web` 网络扩展库搭建的博客系统示例:美观大气的现代化主题、
|
||||
SQLite 驱动的文章/标签、动态路由、WebSocket 实时终端、以及受会话门禁保护的后台 CRUD。
|
||||
Marqdo **Web Artifact** blog (ADR 0007).
|
||||
|
||||
## 运行
|
||||
## Layout
|
||||
|
||||
```bash
|
||||
cargo build -p marqdo_plugin_web
|
||||
cargo run -- examples/marqdo-blog/index.mq.md
|
||||
```text
|
||||
index.mq.md # type: web · / · data_source: posts
|
||||
pages/about.mq.md # 类型: 网页 · /about
|
||||
pages/post.mq.md # type: web · /post/{slug} · data_where
|
||||
api/ping.mq.md # type: endpoint · GET /api/ping
|
||||
api/echo.mq.md # type: endpoint · POST /api/echo
|
||||
db/posts.mq.md # ext/data schema + seed + open
|
||||
public/ # static (mounted at /static)
|
||||
serve.mq.md # web.serve + db open
|
||||
smoke.sh # curl checks against a running serve
|
||||
```
|
||||
|
||||
打开 http://127.0.0.1:18085/:
|
||||
|
||||
- `/` — 首页:文章卡片网格 + 实时 WebSocket 终端
|
||||
- `/post/{slug}` — 动态路由渲染单篇文章(标题/日期/标签/正文)
|
||||
- `/tags` — 标签归档;`/tag/{slug}` 查看某标签下的文章
|
||||
- `/about` — 关于页
|
||||
- `/admin` — 会话门禁后台,登录 `admin` / `marqdo`,可增删改文章与标签
|
||||
- `/live` — WebSocket 回显端点
|
||||
- `/static/*` — 静态资源(`public/` 下的 `live.js`、favicon、logo)
|
||||
- `/favicon.ico` — 站点图标(`应用.图标` 表装配)
|
||||
|
||||
## 设计亮点(文档即代码)
|
||||
|
||||
- **样式即数据表格,装配即函数**(`styles/theme.mq.md`):
|
||||
主题不再是一大段手写 CSS 字符串,而是由一张张 GFM 样式表组成 ——
|
||||
`|选择器|属性|值|` 规则行(同一选择器多行自动合并成一条规则),
|
||||
响应式用 `|媒体|选择器|属性|值|` 分组进 `@media` 块。
|
||||
每个 `## 段`(基础/顶栏/侧栏/主体/卡片/页脚/表单/响应式)导出一张样式表,
|
||||
`## 全局` 用 `网页.样式装配` 函数把每张表装配成 CSS 文本,再 `text.str_join` 拼成完整样式表,
|
||||
最后经 `页面.样式` 注入页面。样式保持可读的数据表格,不写胶水代码。
|
||||
- **图标 / 头资源 / 图片装配**(W8):`应用.图标` 表挂 `/favicon.ico` 与 SVG;
|
||||
`页面.头装配` 写 apple-touch-icon;`页面.图片装配` 把 logo 表注入主区(`mq-images`)。
|
||||
含 `/` 的路径与 MIME 单元格需加引号(如 `"/static/logo.svg"`、`"image/png"`)。
|
||||
- **页面 = 组件装配 + 主体绑定**(`index.mq.md`):
|
||||
导航/侧栏/页脚用 `|组件|样式|` 表装配,文章列表用 `|属性|值|` 绑定 SQLite 字段。
|
||||
- **动态路由**:`/post/{slug}` 的 `slug` 注入查询条件,`页面.详情` 把首行渲染成文章。
|
||||
- **数据模型**(`db/`):`schema.mq.md` 定义 `posts`/`tags`/`post_tags` 三张表,
|
||||
`seed.mq.md` 提供种子数据,`index.mq.md` 幂等建表 + 灌数据。
|
||||
|
||||
## 冒烟测试
|
||||
|
||||
服务器启动后(端口 18085),运行:
|
||||
## Run
|
||||
|
||||
```bash
|
||||
bash scripts/build-web-plugin.sh
|
||||
export MARQDO_EXT=$PWD/ext # if ~/.marqdo/ext is stale
|
||||
export MARQDO_WEB_PLUGIN=$PWD/plugins/web/build/libweb.so
|
||||
cd examples/marqdo-blog && marqdo run serve.mq.md
|
||||
# other terminal:
|
||||
bash examples/marqdo-blog/smoke.sh
|
||||
```
|
||||
|
||||
覆盖:首页/卡片/主题 CSS/媒体查询、静态资源、动态路由详情、标签归档、鉴权门禁、
|
||||
登录会话、后台表单、WebSocket 回显。全部通过时输出 `通过 N / N`。
|
||||
| Path | Artifact |
|
||||
|------|----------|
|
||||
| `/` | Document home + post cards |
|
||||
| `/about` | ZH Document |
|
||||
| `/post/{slug}` | Document detail |
|
||||
| `/api/ping` | Endpoint JSON |
|
||||
| `/api/echo` | Endpoint JSON echo |
|
||||
| `/static/*` | public assets |
|
||||
|
||||
可从仓库根目录或本目录运行:相对 `static_dir`(`public`)与相对数据库路径
|
||||
(`data/marqdo-blog.db`)都按**入口脚本所在目录**解析,不依赖终端工作目录。
|
||||
Design: [ext-web-artifact.md](../../doc/design/ext-web-artifact.md).
|
||||
@@ -0,0 +1,25 @@
|
||||
---
|
||||
type: endpoint
|
||||
method: POST
|
||||
path: /api/echo
|
||||
request: json
|
||||
response: json
|
||||
auth: none
|
||||
---
|
||||
|
||||
# main
|
||||
|
||||
Echo request fields bound via Artifact Metadata (`arg` overlays).
|
||||
|
||||
**msg = message**
|
||||
1. not `msg`
|
||||
**msg = ""**
|
||||
2. *
|
||||
|
||||
`out` =
|
||||
|
||||
| ok | message |
|
||||
|----|---------|
|
||||
| True | `msg` |
|
||||
|
||||
*out*
|
||||
@@ -0,0 +1,19 @@
|
||||
---
|
||||
type: endpoint
|
||||
method: GET
|
||||
path: /api/ping
|
||||
request: none
|
||||
response: json
|
||||
auth: none
|
||||
description: Health ping — documentation is the API (ADR 0007).
|
||||
---
|
||||
|
||||
# main
|
||||
|
||||
`out` =
|
||||
|
||||
| ok | service |
|
||||
|----|---------|
|
||||
| True | marqdo-blog |
|
||||
|
||||
*out*
|
||||
@@ -1,28 +0,0 @@
|
||||
---
|
||||
title: db/index
|
||||
description: 打开 sqlite、建三张表、幂等种子。
|
||||
导入 网页:ext/web/网页.mq.md
|
||||
import schema:schema.mq.md
|
||||
import seed:seed.mq.md
|
||||
---
|
||||
|
||||
## 打开
|
||||
|
||||
**store = > 网页.数据库 地址="sqlite:data/marqdo-blog.db"**
|
||||
**字段 = > schema.posts**
|
||||
**标签字段 = > schema.tags**
|
||||
**关联字段 = > schema.post_tags**
|
||||
> `store`.初始化 名=posts 字段=`字段`
|
||||
> `store`.初始化 名=tags 字段=`标签字段`
|
||||
> `store`.初始化 名=post_tags 字段=`关联字段`
|
||||
**行 = > store.查询 表="posts" 上限=1**
|
||||
1. `行`
|
||||
*store*
|
||||
2. *
|
||||
**文章 = > seed.posts**
|
||||
**标签 = > seed.tags**
|
||||
**关联 = > seed.post_tags**
|
||||
> `store`.插入 表=posts 行=`文章`
|
||||
> `store`.插入 表=tags 行=`标签`
|
||||
> `store`.插入 表=post_tags 行=`关联`
|
||||
*store*
|
||||
@@ -0,0 +1,44 @@
|
||||
---
|
||||
title: blog posts schema
|
||||
import data:ext/data/db.mq.md
|
||||
---
|
||||
|
||||
## schema
|
||||
|
||||
`fields` =
|
||||
|
||||
| 字段 | 类型 | 可空 |
|
||||
|------|------|------|
|
||||
| title | text | False |
|
||||
| slug | text | False |
|
||||
| summary | text | True |
|
||||
| content | text | True |
|
||||
| tag | text | True |
|
||||
| created_at | text | True |
|
||||
|
||||
*fields*
|
||||
|
||||
## seed
|
||||
|
||||
`rows` =
|
||||
|
||||
| title | slug | summary | content | tag | created_at |
|
||||
|-------|------|---------|---------|-----|------------|
|
||||
| Hello Marqdo | hello | Welcome | # Hello\n\nExecutable documents. | intro | 2026-09-01 |
|
||||
| Artifact Web | artifact-web | Document first | Pages are mq.md files. | design | 2026-09-28 |
|
||||
|
||||
*rows*
|
||||
|
||||
## open
|
||||
+ `url`="sqlite:data/blog.db"
|
||||
|
||||
**store = > data.db url=`url`**
|
||||
**fields = > schema**
|
||||
> `store`.init name=posts fields=`fields`
|
||||
**existing = > `store`.select table="posts" limit=1**
|
||||
1. `existing`
|
||||
*store*
|
||||
2. *
|
||||
**seed = > seed**
|
||||
> `store`.insert table=posts rows=`seed`
|
||||
*store*
|
||||
@@ -1,131 +1,15 @@
|
||||
---
|
||||
type: web
|
||||
title: Marqdo 博客
|
||||
description: 用 web 扩展库搭建的博客系统示例:首页列表、动态文章详情、标签归档、登录门禁后台。
|
||||
route: /
|
||||
method: GET
|
||||
data_source: posts
|
||||
data_order: -created_at
|
||||
description: >-
|
||||
Document Artifact 首页(ADR 0007):正文即文档,列表来自 data_source。
|
||||
导入 网页:ext/web/网页.mq.md
|
||||
import theme:styles/theme.mq.md
|
||||
import nav:components/nav.mq.md
|
||||
import side:components/side.mq.md
|
||||
import foot:components/foot.mq.md
|
||||
import db:db/index.mq.md
|
||||
---
|
||||
|
||||
# main
|
||||
# Marqdo 博客
|
||||
|
||||
`首页` =
|
||||
|
||||
| 组件 | 样式 |
|
||||
|------|------|
|
||||
| nav.`导航` | |
|
||||
| side.`侧栏` | |
|
||||
| foot.`页脚` | |
|
||||
|
||||
`列表` =
|
||||
|
||||
| 属性 | 值 | 样式 |
|
||||
|------|-----|------|
|
||||
| title | posts.title | |
|
||||
| body | posts.summary | |
|
||||
| meta | posts.created_at | |
|
||||
| tag | posts.tag | |
|
||||
| href | posts.slug | |
|
||||
|
||||
`详情绑定` =
|
||||
|
||||
| 属性 | 值 | 样式 |
|
||||
|------|-----|------|
|
||||
| title | posts.title | |
|
||||
| meta | posts.created_at | |
|
||||
| tag | posts.tag | |
|
||||
| body | posts.content | |
|
||||
|
||||
`标签列表` =
|
||||
|
||||
| 属性 | 值 | 样式 |
|
||||
|------|-----|------|
|
||||
| title | tags.name | |
|
||||
| meta | 点击查看该标签下的文章 | |
|
||||
| href | tags.slug | |
|
||||
|
||||
`文章条件` =
|
||||
|
||||
| 字段 | 操作 | 值 |
|
||||
|------|------|-----|
|
||||
| slug | = | {slug} |
|
||||
|
||||
`标签条件` =
|
||||
|
||||
| 字段 | 操作 | 值 |
|
||||
|------|------|-----|
|
||||
| tag | = | {slug} |
|
||||
|
||||
`管理员` =
|
||||
|
||||
| 行 | 用户名 | 密码 |
|
||||
|----|--------|------|
|
||||
| 1 | admin | marqdo |
|
||||
|
||||
`站点图标` =
|
||||
|
||||
| 路径 | 关系 | 类型 | 尺寸 | 地址 |
|
||||
|------|------|------|------|------|
|
||||
| "public/favicon.png" | icon | "image/png" | 32x32 | "/favicon.ico" |
|
||||
| "public/favicon.svg" | icon | "image/svg+xml" | any | "/static/favicon.svg" |
|
||||
|
||||
`头资源` =
|
||||
|
||||
| 关系 | 地址 | 类型 |
|
||||
|------|------|------|
|
||||
| apple-touch-icon | "/static/favicon.png" | "image/png" |
|
||||
|
||||
`品牌图` =
|
||||
|
||||
| 源 | 替代 | 类 | 链接 | 宽度 | 加载 |
|
||||
|----|------|----|------|------|------|
|
||||
| "/static/logo.svg" | Marqdo | brand-logo | "/" | 40 | eager |
|
||||
|
||||
**store = > db.打开**
|
||||
|
||||
**首页CSS = > theme.全局**
|
||||
|
||||
**page = > 网页.页面 标题="Marqdo 博客" 引言="<h1>Marqdo 博客</h1><p>用 web 扩展库搭建的博客:表格装配页面、SQLite 数据、动态路由、实时终端。</p><h2 id=\"live\">实时终端</h2><p id=\"live-status\" class=\"live-pill\">connecting…</p><input id=\"live-msg\" placeholder=\"输入一条消息…\"><button id=\"live-send\" disabled>发送</button><div id=\"live-out\"></div><script src=\"/static/live.js\"></script>"**
|
||||
**page = > page.组件装配 组件=`首页`**
|
||||
**page = > page.主体装配 主体=`列表`**
|
||||
**page = > page.排序 排序="-created_at"**
|
||||
**page = > page.样式 样式=`首页CSS`**
|
||||
**page = > page.头装配 表=`头资源`**
|
||||
**page = > page.图片装配 表=`品牌图`**
|
||||
|
||||
**about = > 网页.页面 标题="关于" 引言="<h1>关于本站</h1><p>一个用 Marqdo 的 web 扩展库(ext/web)写成的博客系统。</p><p>它演示了:GFM 表格装配导航与侧栏、SQLite 驱动的主体绑定、动态路由 /post/{slug}、标签归档、以及受 session 保护的后台 CRUD。</p>"**
|
||||
**about = > about.组件装配 组件=`首页`**
|
||||
**about = > about.样式 样式=`首页CSS`**
|
||||
|
||||
**post = > 网页.页面 标题="文章"**
|
||||
**post = > post.组件装配 组件=`首页`**
|
||||
**post = > post.主体装配 主体=`详情绑定`**
|
||||
**post = > post.查询条件 条件=`文章条件`**
|
||||
**post = > post.详情 详情=True**
|
||||
**post = > post.样式 样式=`首页CSS`**
|
||||
|
||||
**tags = > 网页.页面 标题="标签归档" 引言="<h1>标签归档</h1><p>点击标签查看该主题下的文章。</p>"**
|
||||
**tags = > tags.组件装配 组件=`首页`**
|
||||
**tags = > tags.主体装配 主体=`标签列表`**
|
||||
**tags = > tags.链接前缀 前缀="/tag/"**
|
||||
**tags = > tags.样式 样式=`首页CSS`**
|
||||
|
||||
**tagged = > 网页.页面 标题="标签"**
|
||||
**tagged = > tagged.组件装配 组件=`首页`**
|
||||
**tagged = > tagged.主体装配 主体=`列表`**
|
||||
**tagged = > tagged.查询条件 条件=`标签条件`**
|
||||
**tagged = > tagged.排序 排序="-created_at"**
|
||||
**tagged = > tagged.样式 样式=`首页CSS`**
|
||||
|
||||
**app = > 网页.应用 页面=page 数据库=store 后台=True 主机="127.0.0.1" 端口=18085**
|
||||
**app = > app.路由 路径="/about" 页面=about**
|
||||
**app = > app.路由 路径="/post/{slug}" 页面=post**
|
||||
**app = > app.路由 路径="/tags" 页面=tags**
|
||||
**app = > app.路由 路径="/tag/{slug}" 页面=tagged**
|
||||
**app = > app.静态 目录="public" 挂载="/static"**
|
||||
**app = > app.图标 表=`站点图标`**
|
||||
**app = > app.路由实时 路径="/live" 回显=True**
|
||||
**app = > app.鉴权 用户表=`管理员` 会话时长=3600**
|
||||
> `app`.监听
|
||||
可执行文档即网页。本站用 Artifact Metadata 声明路由;文章列表由 `data_source: posts` 在服务时加载。
|
||||
@@ -0,0 +1,10 @@
|
||||
---
|
||||
类型: 网页
|
||||
路由: /about
|
||||
方法: GET
|
||||
标题: 关于
|
||||
---
|
||||
|
||||
# 关于 Marqdo
|
||||
|
||||
Marqdo 是可执行文档语言:文档即程序,程序即知识。
|
||||
@@ -0,0 +1,13 @@
|
||||
---
|
||||
type: web
|
||||
title: Post
|
||||
route: /post/{slug}
|
||||
method: GET
|
||||
data_source: posts
|
||||
data_where: slug={slug}
|
||||
description: Dynamic Document route — body is the article shell; data from Resource.
|
||||
---
|
||||
|
||||
# Post
|
||||
|
||||
Article page for `{slug}`. List and detail data are provided by `ext/data` at serve time when `data_source` is set on the Artifact.
|
||||
@@ -0,0 +1,11 @@
|
||||
---
|
||||
title: marqdo-blog serve
|
||||
description: Scan Artifact documents/endpoints and listen (ADR 0007).
|
||||
import web:ext/web/web.mq.md
|
||||
import posts:db/posts.mq.md
|
||||
---
|
||||
|
||||
# main
|
||||
|
||||
**store = > posts.open url="sqlite:data/blog.db"**
|
||||
> web.serve root="." host="127.0.0.1" port=18081 db=`store` static_dir="public"
|
||||
@@ -1,71 +1,41 @@
|
||||
#!/usr/bin/env bash
|
||||
# 冒烟测试:marqdo-blog 全功能验证。
|
||||
# 用法:bash examples/marqdo-blog/smoke.sh
|
||||
# Artifact blog smoke (ADR 0007). Usage: bash examples/marqdo-blog/smoke.sh
|
||||
# Expects serve.mq.md already listening on BASE (default 18081).
|
||||
set -uo pipefail
|
||||
BASE="http://127.0.0.1:18085"
|
||||
BASE="${BASE:-http://127.0.0.1:18081}"
|
||||
PASS=0; FAIL=0
|
||||
ok() { echo " ok: $1"; PASS=$((PASS+1)); }
|
||||
bad() { echo " FAIL: $1"; FAIL=$((FAIL+1)); }
|
||||
|
||||
echo "== 首页 =="
|
||||
echo "== Document home =="
|
||||
code=$(curl -s -o /dev/null -w "%{http_code}" "$BASE/")
|
||||
[ "$code" = "200" ] && ok "GET / -> $code" || bad "GET / -> $code"
|
||||
curl -s "$BASE/" | grep -q '<title>Marqdo 博客</title>' && ok "title" || bad "title"
|
||||
curl -s "$BASE/" | grep -q 'class="card"' && ok "卡片渲染" || bad "卡片缺失"
|
||||
curl -s "$BASE/" | grep -q 'aside.side' && ok "主题 CSS 注入" || bad "主题 CSS 缺失"
|
||||
curl -s "$BASE/" | grep -q '@media (max-width: 860px)' && ok "响应式媒体查询" || bad "媒体查询缺失"
|
||||
curl -s "$BASE/" | grep -q 'class="card"' && ok "data_source cards" || bad "cards missing"
|
||||
curl -s "$BASE/" | grep -q 'Hello Marqdo' && ok "seed title" || bad "seed title missing"
|
||||
|
||||
echo "== 静态 =="
|
||||
echo "== Static =="
|
||||
code=$(curl -s -o /dev/null -w "%{http_code}" "$BASE/static/live.js")
|
||||
[ "$code" = "200" ] && ok "GET /static/live.js -> $code" || bad "static -> $code"
|
||||
code=$(curl -s -o /dev/null -w "%{http_code}" "$BASE/favicon.ico")
|
||||
[ "$code" = "200" ] && ok "GET /favicon.ico -> $code" || bad "favicon -> $code"
|
||||
curl -s "$BASE/" | grep -q 'rel="icon"' && ok "head icon link" || bad "head icon 缺失"
|
||||
curl -s "$BASE/" | grep -q 'mq-images' && ok "图片装配 logo" || bad "mq-images 缺失"
|
||||
|
||||
echo "== 动态路由 =="
|
||||
code=$(curl -s -o /dev/null -w "%{http_code}" "$BASE/post/hello-marqdo")
|
||||
[ "$code" = "200" ] && ok "GET /post/hello-marqdo -> $code" || bad "post detail -> $code"
|
||||
curl -s "$BASE/post/hello-marqdo" | grep -q 'article-title' && ok "文章详情渲染" || bad "文章详情缺失"
|
||||
echo "== Dynamic Document =="
|
||||
code=$(curl -s -o /dev/null -w "%{http_code}" "$BASE/post/hello")
|
||||
[ "$code" = "200" ] && ok "GET /post/hello -> $code" || bad "post detail -> $code"
|
||||
curl -s "$BASE/post/hello" | grep -q 'article-title' && ok "article title" || bad "article title missing"
|
||||
|
||||
echo "== 标签 =="
|
||||
curl -s "$BASE/tags" | grep -q 'href="/tag/marqdo"' && ok "标签归档链接" || bad "标签链接缺失"
|
||||
code=$(curl -s -o /dev/null -w "%{http_code}" "$BASE/tag/web")
|
||||
[ "$code" = "200" ] && ok "GET /tag/web -> $code" || bad "tag page -> $code"
|
||||
echo "== ZH Document =="
|
||||
code=$(curl -s -o /dev/null -w "%{http_code}" "$BASE/about")
|
||||
[ "$code" = "200" ] && ok "GET /about -> $code" || bad "about -> $code"
|
||||
curl -s "$BASE/about" | grep -q '关于 Marqdo' && ok "about body" || bad "about body missing"
|
||||
|
||||
echo "== 鉴权门禁 =="
|
||||
code=$(curl -s -o /dev/null -w "%{http_code}" "$BASE/admin")
|
||||
[ "$code" = "303" ] && ok "未登录 /admin -> $code (重定向登录)" || bad "未登录 /admin -> $code"
|
||||
|
||||
echo "== 登录 =="
|
||||
rm -f /tmp/mq_smoke_cookies.txt /tmp/mq_smoke_headers.txt
|
||||
curl -s -D /tmp/mq_smoke_headers.txt -o /dev/null -d "username=admin&password=marqdo" "$BASE/admin/login"
|
||||
sid=$(grep -i "set-cookie" /tmp/mq_smoke_headers.txt | sed 's/.*marqdo_sid=\([^;]*\).*/\1/')
|
||||
[ -n "$sid" ] && ok "登录获取会话 cookie" || bad "登录未返回 cookie"
|
||||
code=$(curl -s -b "marqdo_sid=$sid" -o /dev/null -w "%{http_code}" "$BASE/admin")
|
||||
[ "$code" = "200" ] && ok "带会话访问 /admin -> $code" || bad "带会话 /admin -> $code"
|
||||
|
||||
echo "== 发布文章 (CRUD) =="
|
||||
code=$(curl -s -b "marqdo_sid=$sid" -o /dev/null -w "%{http_code}" "$BASE/admin/posts/new")
|
||||
[ "$code" = "200" ] && ok "GET /admin/posts/new -> $code" || bad "new form -> $code"
|
||||
|
||||
echo "== WebSocket =="
|
||||
wsok=$(python3 -c "
|
||||
import socket, base64, os
|
||||
s = socket.create_connection(('127.0.0.1', 18085), timeout=3)
|
||||
key = base64.b64encode(os.urandom(16)).decode()
|
||||
s.sendall((f'GET /live HTTP/1.1\r\nHost: x\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nSec-WebSocket-Key: {key}\r\nSec-WebSocket-Version: 13\r\n\r\n').encode())
|
||||
r = s.recv(1024).decode(errors='replace')
|
||||
if '101' not in r.split('\r\n')[0]:
|
||||
print('NO'); s.close(); raise SystemExit
|
||||
mask = os.urandom(4); payload = b'ping'
|
||||
m = bytes(b ^ mask[i%4] for i, b in enumerate(payload))
|
||||
s.sendall(bytes([0x81, 0x80|len(payload)]) + mask + m)
|
||||
d = s.recv(1024)
|
||||
print('OK' if b'ping' in d else 'NO')
|
||||
s.close()
|
||||
" 2>/dev/null)
|
||||
[ "$wsok" = "OK" ] && ok "WebSocket /live 回显" || bad "WebSocket 回显 $wsok"
|
||||
echo "== Endpoint =="
|
||||
code=$(curl -s -o /dev/null -w "%{http_code}" "$BASE/api/ping")
|
||||
[ "$code" = "200" ] && ok "GET /api/ping -> $code" || bad "ping -> $code"
|
||||
curl -s "$BASE/api/ping" | grep -q 'marqdo-blog' && ok "ping json" || bad "ping json missing"
|
||||
code=$(curl -s -o /tmp/mq_blog_echo.json -w "%{http_code}" \
|
||||
-H 'Content-Type: application/json' -d '{"message":"hi"}' "$BASE/api/echo")
|
||||
[ "$code" = "200" ] && ok "POST /api/echo -> $code" || bad "echo -> $code"
|
||||
grep -q '"message":"hi"' /tmp/mq_blog_echo.json && ok "echo body" || bad "echo body missing"
|
||||
|
||||
echo
|
||||
echo "通过 $PASS / $((PASS+FAIL))"
|
||||
|
||||
@@ -1,20 +1,14 @@
|
||||
# web-client-site(路线 D)
|
||||
# examples/web-client-site
|
||||
|
||||
服务端 `ext/web` + 浏览器 WASM 会话;**作者零业务 JS**。
|
||||
SSR **Document Artifact** + `web.client_embed` auto-mount(零作者 JS)。ADR 0007。
|
||||
|
||||
```bash
|
||||
# 仓库根
|
||||
cargo build -p marqdo --release
|
||||
cargo build --release -p marqdo_plugin_web
|
||||
./target/release/marqdo ext add web
|
||||
|
||||
# 构建 WASM + bridge 到本示例 static/
|
||||
./target/release/marqdo wasm build -o examples/web-client-site/static
|
||||
# 保留 client.mq.md(wasm build 只写入 wasm/bridge)
|
||||
|
||||
cd examples/web-client-site
|
||||
marqdo run index.mq.md
|
||||
# 打开 http://127.0.0.1:18090/
|
||||
bash scripts/build-web-plugin.sh
|
||||
export MARQDO_EXT=$PWD/ext # 若本机 ~/.marqdo/ext 未同步
|
||||
marqdo wasm build -o examples/web-client-site/static
|
||||
cargo run -- run examples/web-client-site/serve.mq.md
|
||||
```
|
||||
|
||||
`index.mq.md` 用 `web.client_embed source="/static/client.mq.md"` 注入自启脚本;交互在 `static/client.mq.md`。
|
||||
- `index.mq.md` — `type: web` 文档页
|
||||
- `serve.mq.md` — `web.serve` + `static_dir`
|
||||
- `static/client.mq.md` — `lib/browser` wire(不变)
|
||||
@@ -1,19 +1,14 @@
|
||||
---
|
||||
title: web-client-site
|
||||
description: SSR shell + client_embed auto-mount; author zero JS (route D)
|
||||
type: web
|
||||
title: Marqdo client site
|
||||
route: /
|
||||
method: GET
|
||||
description: SSR Document + client_embed auto-mount; author zero JS (ADR 0007).
|
||||
import web:ext/web/web.mq.md
|
||||
---
|
||||
|
||||
# main
|
||||
# Marqdo client site
|
||||
|
||||
Server page with a counter UI. Client logic lives in `static/client.mq.md`
|
||||
(loaded by official bridge via `client_embed`).
|
||||
SSR shell with WASM session. Client logic lives in `static/client.mq.md`.
|
||||
|
||||
**`embed` = > web.client_embed bridge="/static/marqdo-bridge.js" wasm="/static/marqdo_wasm.wasm" source="/static/client.mq.md" boot=True**
|
||||
|
||||
**`intro` = "<h1>Marqdo client site</h1><p>SSR + WASM session. No author JS.</p><div id=\"count\">0</div><p><button id=\"bump\" type=\"button\" disabled>Bump</button> <button id=\"reset\" type=\"button\" disabled>Reset</button></p><pre id=\"log\">booting…</pre>" + embed**
|
||||
|
||||
**page = > web.page title="Marqdo client site" intro=intro**
|
||||
**app = > web.app page=page host="127.0.0.1" port=18090**
|
||||
**app = > app.static dir="static" mount="/static"**
|
||||
> `app`.listen
|
||||
To serve with static assets, run `serve.mq.md` in this directory.
|
||||
@@ -0,0 +1,10 @@
|
||||
---
|
||||
title: web-client-site serve
|
||||
import web:ext/web/web.mq.md
|
||||
---
|
||||
|
||||
# main
|
||||
|
||||
**`embed` = > web.client_embed source="/static/client.mq.md"**
|
||||
> print text=embed-ready
|
||||
> web.serve root="." host="127.0.0.1" port=18090 static_dir="static"
|
||||
+1
-1
@@ -1 +1 @@
|
||||
1.1.0
|
||||
1.3.2
|
||||
+594
-24
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: ext/ai/agent
|
||||
description: Document-driven agent — step / plan; tools via lib/subtask; runtime via ABI v2 agent plugin.
|
||||
description: Document-driven agent v2 — step / plan / run; Skill Compilation via ABI plugin (episodes → learn → llm_free).
|
||||
import llm:ext/ai/llm.mq.md
|
||||
import json:lib/json.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
@@ -49,6 +49,37 @@ The runbook `.mq.md` is ground truth (code as documentation). Default budgets ke
|
||||
|
||||
---
|
||||
|
||||
## model_turn
|
||||
+ `model`
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
|
||||
One LLM turn via the semantic surface (`ask` / `stream`+`collect`). Returns `{text,llm_calls,tokens}` for episode metrics.
|
||||
|
||||
1. `stream`
|
||||
**evs = > model.stream prompt=`prompt` echo=`echo`**
|
||||
**text = > llm.collect events=`evs`**
|
||||
**tokens = 0**
|
||||
2. *
|
||||
**r = > model.ask prompt=`prompt`**
|
||||
**text = [text](r)**
|
||||
**tokens = [tokens](r)**
|
||||
1. `tokens`
|
||||
**_ = 1**
|
||||
2. *
|
||||
**tokens = 0**
|
||||
|
||||
`out` =
|
||||
|
||||
| text | llm_calls | tokens |
|
||||
|------|-----------|--------|
|
||||
| `text` | 1 | `tokens` |
|
||||
|
||||
*out*
|
||||
|
||||
---
|
||||
|
||||
## dump_step_context
|
||||
+ `agent`
|
||||
+ `task`=""
|
||||
@@ -424,6 +455,128 @@ Call a named tool from a MCP-shaped fixture (`results` / `calls` map). Optional
|
||||
|
||||
---
|
||||
|
||||
## mcp_connect
|
||||
+ `command`
|
||||
+ `args`=None
|
||||
+ `session`="default"
|
||||
+ `name`="mcp"
|
||||
|
||||
Open a stdio MCP client session (spawn `command` + `args`, JSON-RPC initialize). Evidence only — workbook remains authority.
|
||||
|
||||
**p = > plugin.native_path name="agent"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
2. *
|
||||
> print text=ext/ai/agent: native agent plugin not found
|
||||
> sys.exit code=1
|
||||
|
||||
1. `args`
|
||||
*> agent_mcp_client action="connect" command=`command` args=`args` session=`session` name=`name`*
|
||||
2. *
|
||||
*> agent_mcp_client action="connect" command=`command` session=`session` name=`name`*
|
||||
|
||||
## mcp_client_list
|
||||
+ `session`="default"
|
||||
+ `command`=None
|
||||
+ `args`=None
|
||||
|
||||
List tools from an open MCP session, or one-shot when `command=` is set (no session retained).
|
||||
|
||||
1. `command`
|
||||
1. `args`
|
||||
*> agent_mcp_client action="list" command=`command` args=`args`*
|
||||
2. *
|
||||
*> agent_mcp_client action="list" command=`command`*
|
||||
2. *
|
||||
*> agent_mcp_client action="list" session=`session`*
|
||||
|
||||
## mcp_client_call
|
||||
+ `name`=None
|
||||
+ `session`="default"
|
||||
+ `arguments`=None
|
||||
+ `command`=None
|
||||
+ `args`=None
|
||||
+ `tool`=None
|
||||
|
||||
Call a tool on an open MCP session, or one-shot when `command=` is set. Prefer `tool=` for oneshot when `name` would collide with server name.
|
||||
|
||||
1. `command`
|
||||
1. `tool`
|
||||
**tn = tool**
|
||||
2. *
|
||||
**tn = name**
|
||||
1. `args`
|
||||
*> agent_mcp_client action="call" command=`command` args=`args` tool=`tn` arguments=`arguments`*
|
||||
2. *
|
||||
*> agent_mcp_client action="call" command=`command` tool=`tn` arguments=`arguments`*
|
||||
2. *
|
||||
*> agent_mcp_client action="call" session=`session` name=`name` arguments=`arguments`*
|
||||
|
||||
## mcp_disconnect
|
||||
+ `session`="default"
|
||||
|
||||
Close a stdio MCP client session.
|
||||
|
||||
*> agent_mcp_client action="close" session=`session`*
|
||||
|
||||
---
|
||||
|
||||
## resume_save
|
||||
+ `id`
|
||||
+ `goal`=None
|
||||
+ `task`=None
|
||||
+ `status`=running
|
||||
+ `round`=0
|
||||
+ `workbook`=None
|
||||
+ `events`=None
|
||||
+ `route`=None
|
||||
+ `result`=None
|
||||
+ `resume_dir`=.marqdo/agent-resume
|
||||
|
||||
Persist a plan/run checkpoint as `.marqdo/agent-resume/<id>.json`.
|
||||
|
||||
**p = > plugin.native_path name="agent"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
2. *
|
||||
> print text=ext/ai/agent: native agent plugin not found
|
||||
> sys.exit code=1
|
||||
|
||||
*> agent_resume_save id=`id` goal=`goal` task=`task` status=`status` round=`round` workbook=`workbook` events=`events` route=`route` result=`result` resume_dir=`resume_dir`*
|
||||
|
||||
## resume_load
|
||||
+ `id`
|
||||
+ `resume_dir`=.marqdo/agent-resume
|
||||
|
||||
Load a resume checkpoint (or `{ok:false,found:false}`).
|
||||
|
||||
*> agent_resume_load id=`id` resume_dir=`resume_dir`*
|
||||
|
||||
## resume_clear
|
||||
+ `id`
|
||||
+ `resume_dir`=.marqdo/agent-resume
|
||||
|
||||
Delete a resume checkpoint.
|
||||
|
||||
*> agent_resume_clear id=`id` resume_dir=`resume_dir`*
|
||||
|
||||
## resume_list
|
||||
+ `resume_dir`=.marqdo/agent-resume
|
||||
|
||||
List checkpoint ids under `resume_dir`.
|
||||
|
||||
*> agent_resume_list resume_dir=`resume_dir`*
|
||||
|
||||
## resume
|
||||
+ `id`
|
||||
+ `resume_dir`=.marqdo/agent-resume
|
||||
|
||||
Author entry: load checkpoint for continuation. Returns the checkpoint map (`found`/`round`/`workbook`/…). Callers continue with `plan`/`run` using saved fields; clear when done.
|
||||
|
||||
*> resume_load id=`id` resume_dir=`resume_dir`*
|
||||
|
||||
---
|
||||
|
||||
## extract_call_args
|
||||
+ `reply`=None
|
||||
|
||||
@@ -1264,7 +1417,7 @@ Parent Plan-and-Move prompt. Short protocol only — no long monologues.
|
||||
+ `events`
|
||||
+ `max_acts`=6
|
||||
|
||||
LLM Plan → CALL/READ/DECISION loop. Returns `{decision,reply,observation,events}`.
|
||||
LLM Plan → CALL/READ/DECISION loop. Returns `{decision,reply,observation,events,llm_calls,tokens}`.
|
||||
|
||||
**model = > json.get value=`agent` key="model"**
|
||||
**last_obs = observation**
|
||||
@@ -1272,6 +1425,8 @@ LLM Plan → CALL/READ/DECISION loop. Returns `{decision,reply,observation,event
|
||||
**decision = None**
|
||||
**last_reply = None**
|
||||
**evs_all = events**
|
||||
**llm_calls = 0**
|
||||
**tokens = 0**
|
||||
|
||||
- `left` > 0
|
||||
1. `decision`
|
||||
@@ -1279,11 +1434,17 @@ LLM Plan → CALL/READ/DECISION loop. Returns `{decision,reply,observation,event
|
||||
2. *
|
||||
**ctx = > build_plan_context agent=`agent` goal=`goal` observation=`last_obs` explore_attempt=`explore_attempt` explore_n=`explore_n` phase=`phase`**
|
||||
1. `stream`
|
||||
**evs = > model.complete prompt=`ctx` stream=True echo=`echo`**
|
||||
**last_reply = > llm.stream_result events=`evs`**
|
||||
**evs = > model.stream prompt=`ctx` echo=`echo`**
|
||||
**last_reply = > llm.collect events=`evs`**
|
||||
**evs_all = > plan_merge_deltas events=`evs_all` from=`evs` stream=`stream`**
|
||||
**llm_calls = llm_calls + 1**
|
||||
2. *
|
||||
**last_reply = > model.complete prompt=`ctx`**
|
||||
**turn = > model_turn model=`model` prompt=`ctx` stream=False echo=`echo`**
|
||||
**last_reply = > json.get value=`turn` key="text"**
|
||||
**c = > json.get value=`turn` key="llm_calls"**
|
||||
**t = > json.get value=`turn` key="tokens"**
|
||||
**llm_calls = llm_calls + c**
|
||||
**tokens = tokens + t**
|
||||
**last_reply = > trim value=`last_reply`**
|
||||
**act = > extract_plan_act reply=`last_reply`**
|
||||
**kind = > json.get value=`act` key="kind"**
|
||||
@@ -1314,6 +1475,8 @@ LLM Plan → CALL/READ/DECISION loop. Returns `{decision,reply,observation,event
|
||||
**out = > json.set map=`out` key="reply" value=`last_reply`**
|
||||
**out = > json.set map=`out` key="observation" value=`last_obs`**
|
||||
**out = > json.set map=`out` key="events" value=`evs_all`**
|
||||
**out = > json.set map=`out` key="llm_calls" value=`llm_calls`**
|
||||
**out = > json.set map=`out` key="tokens" value=`tokens`**
|
||||
*out*
|
||||
|
||||
---
|
||||
@@ -1515,6 +1678,316 @@ Clear the plugin session bag for this agent id.
|
||||
> agent_history_clear id=`id`
|
||||
****
|
||||
|
||||
## resolve
|
||||
+ `task`
|
||||
+ `kb_dir`=.marqdo/agent-kb
|
||||
+ `near_match`=True
|
||||
+ `near_threshold`=0.78
|
||||
|
||||
Skill resolver (v2): exact → alias → canonical → near. Returns `mode` / `resource` / `llm_free` / `confidence`. Plugin ABI — no core host_* calls.
|
||||
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
*> agent_resolve task=`task` kb_dir=`kb_dir` near_match=`near_match` near_threshold=`near_threshold` tools=`tools`*
|
||||
|
||||
## route
|
||||
+ `task`
|
||||
+ `backend`=auto
|
||||
+ `kb_dir`=.marqdo/agent-kb
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
+ `threshold`=0.78
|
||||
|
||||
Adaptive Routing (Phase 4): abstract router — `marqdo` | `small-llm` | `llm` | `auto`. `jev` is accepted as a **compat alias** of `small-llm` (not a first-class backend). `auto` cascades policy → kb → small-llm request. May return `needs_llm=True` + `prompt` for the caller to complete with `router_model` (any llm handle), then `route_apply`.
|
||||
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
*> agent_route task=`task` backend=`backend` kb_dir=`kb_dir` memory_dir=`memory_dir` threshold=`threshold` tools=`tools`*
|
||||
|
||||
## route_apply
|
||||
+ `task`
|
||||
+ `reply`
|
||||
+ `kb_dir`=.marqdo/agent-kb
|
||||
+ `backend`=small-llm
|
||||
|
||||
Apply a router model line (`SKILL:<slug>` / `EXPLORE`) to a kb skill hit.
|
||||
|
||||
*> agent_route_apply task=`task` reply=`reply` kb_dir=`kb_dir` backend=`backend`*
|
||||
|
||||
## compile_policy
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
+ `min_evidence`=3
|
||||
+ `force`=False
|
||||
|
||||
Policy Compilation: stable episode skill selections → `policies/router.mq.md` (deterministic Marqdo rules).
|
||||
|
||||
*> agent_compile_policy memory_dir=`memory_dir` min_evidence=`min_evidence` force=`force`*
|
||||
|
||||
## maybe_compile_policy
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
+ `min_evidence`=3
|
||||
|
||||
Learn gate for the Adaptive Router policy layer.
|
||||
|
||||
*> agent_maybe_compile_policy memory_dir=`memory_dir` min_evidence=`min_evidence`*
|
||||
|
||||
## record_episode
|
||||
+ `task`
|
||||
+ `status`=ok
|
||||
+ `result`=None
|
||||
+ `mode`=explore
|
||||
+ `skill`=none
|
||||
+ `observation`=None
|
||||
+ `workbook`=None
|
||||
+ `cache`=None
|
||||
+ `match`=None
|
||||
+ `llm_calls`=0
|
||||
+ `tool_calls`=0
|
||||
+ `error`=None
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
|
||||
Persist one run as `.mq.md` under `memory_dir/episodes/YYYY/MM/` (v2 Phase 1). Memory is executable documentation, not a vector bag.
|
||||
|
||||
*> agent_record_episode task=`task` status=`status` result=`result` mode=`mode` skill=`skill` observation=`observation` workbook=`workbook` cache=`cache` match=`match` llm_calls=`llm_calls` tool_calls=`tool_calls` error=`error` memory_dir=`memory_dir`*
|
||||
|
||||
## maybe_learn
|
||||
+ `task`
|
||||
+ `kb_dir`=.marqdo/agent-kb
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
+ `improve_every`=3
|
||||
+ `promote`=True
|
||||
|
||||
Offline learn gate: when ≥ `improve_every` successful episodes share a task hash, generalize → synthesize → compile → optional OKF promote (`llm_free`). Also attempts Policy Compilation for Adaptive Routing.
|
||||
|
||||
**skill_learn = > agent_maybe_learn task=`task` kb_dir=`kb_dir` memory_dir=`memory_dir` improve_every=`improve_every` promote=`promote`**
|
||||
**policy_learn = > agent_maybe_compile_policy memory_dir=`memory_dir` min_evidence=`improve_every`**
|
||||
**out = > json.parse text={"learned":false}**
|
||||
**sl = > json.get value=`skill_learn` key="learned"**
|
||||
**out = > json.set map=`out` key="learned" value=`sl`**
|
||||
**out = > json.set map=`out` key="skill" value=`skill_learn`**
|
||||
**out = > json.set map=`out` key="policy" value=`policy_learn`**
|
||||
*out*
|
||||
|
||||
## compile
|
||||
+ `task`
|
||||
+ `kb_dir`=.marqdo/agent-kb
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
+ `min_evidence`=3
|
||||
+ `promote`=True
|
||||
+ `force`=False
|
||||
|
||||
Compile a candidate skill into versioned `skills/<slug>/vN.mq.md` and optionally promote into agent-kb.
|
||||
|
||||
*> agent_compile_skill task=`task` kb_dir=`kb_dir` memory_dir=`memory_dir` min_evidence=`min_evidence` promote=`promote` force=`force`*
|
||||
|
||||
## metrics
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
|
||||
Reasoning amortization stats over recorded episodes (`avg_llm_calls`, `compiled_runs`, …).
|
||||
|
||||
*> agent_metrics memory_dir=`memory_dir`*
|
||||
|
||||
## run
|
||||
+ `task`
|
||||
+ `kb_dir`=.marqdo/agent-kb
|
||||
+ `memory_dir`=.marqdo/agent-memory
|
||||
+ `learn`=True
|
||||
+ `verify`=True
|
||||
+ `writeback`=True
|
||||
+ `reuse`=True
|
||||
+ `near_match`=True
|
||||
+ `near_threshold`=0.78
|
||||
+ `promote`=True
|
||||
+ `improve_every`=3
|
||||
+ `explore_n`=3
|
||||
+ `max_rounds`=4
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
+ `trace`=False
|
||||
+ `force`=False
|
||||
+ `router`=auto
|
||||
+ `router_model`=None
|
||||
+ `resume_id`=None
|
||||
+ `resume_dir`=.marqdo/agent-resume
|
||||
|
||||
Primary author entry (v2 + P4): Adaptive `route` → execute (policy/kb skill) or `plan` → record episode → maybe_learn (skill + policy). `router` = `auto`|`marqdo`|`small-llm`|`llm` (`jev`→`small-llm` alias). Optional `router_model` (llm handle) answers `needs_llm` prompts — small models welcome.
|
||||
|
||||
**routed = > `self`.route task=`task` backend=`router` kb_dir=`kb_dir` memory_dir=`memory_dir` threshold=`near_threshold`**
|
||||
**route_matched = > json.get value=`routed` key="matched"**
|
||||
**needs_llm = > json.get value=`routed` key="needs_llm"**
|
||||
**exec_mode = "explore"**
|
||||
**llm_calls = 0**
|
||||
**tokens = 0**
|
||||
**out = None**
|
||||
**route_backend = > json.get value=`routed` key="backend"**
|
||||
|
||||
1. `route_matched`
|
||||
**path = > json.get value=`routed` key="resource"**
|
||||
1. `path`
|
||||
**aw = > await_workbook path=`path`**
|
||||
**code = > json.get value=`aw` key="code"**
|
||||
**child_val = > json.get value=`aw` key="value"**
|
||||
**last_obs = > json.get value=`aw` key="observation"**
|
||||
1. `code` == 0
|
||||
> agent_kb_record_hit kb_dir=`kb_dir` goal=`task` improve_every=`improve_every`
|
||||
**lf = > json.get value=`routed` key="llm_free"**
|
||||
1. `lf`
|
||||
**exec_mode = "compiled"**
|
||||
2. *
|
||||
**exec_mode = "routed"**
|
||||
**out = > json.parse text={"status":"ok"}**
|
||||
**out = > json.set map=`out` key="status" value="ok"**
|
||||
**out = > json.set map=`out` key="goal" value=`task`**
|
||||
**out = > json.set map=`out` key="result" value=`child_val`**
|
||||
**out = > json.set map=`out` key="workbook" value=`path`**
|
||||
**out = > json.set map=`out` key="cache" value="hit"**
|
||||
**rmode = > json.get value=`routed` key="mode"**
|
||||
**out = > json.set map=`out` key="match" value=`rmode`**
|
||||
**sk = > json.get value=`routed` key="skill"**
|
||||
**out = > json.set map=`out` key="skill" value=`sk`**
|
||||
**out = > json.set map=`out` key="observation" value=`last_obs`**
|
||||
**out = > json.set map=`out` key="summary" value="adaptive route hit"**
|
||||
**out = > json.set map=`out` key="route" value=`routed`**
|
||||
**llm_calls = 0**
|
||||
2. *
|
||||
**route_matched = False**
|
||||
2. *
|
||||
**route_matched = False**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `route_matched`
|
||||
**_ = 1**
|
||||
2. `needs_llm`
|
||||
1. `router_model`
|
||||
**prompt = > json.get value=`routed` key="prompt"**
|
||||
**turn = > model_turn model=`router_model` prompt=`prompt` stream=`stream` echo=`echo`**
|
||||
**reply = > json.get value=`turn` key="text"**
|
||||
**llm_calls = > json.get value=`turn` key="llm_calls"**
|
||||
**tokens = > json.get value=`turn` key="tokens"**
|
||||
**applied = > `self`.route_apply task=`task` reply=`reply` kb_dir=`kb_dir` backend=`router`**
|
||||
**am = > json.get value=`applied` key="matched"**
|
||||
1. `am`
|
||||
**path = > json.get value=`applied` key="resource"**
|
||||
**aw = > await_workbook path=`path`**
|
||||
**code = > json.get value=`aw` key="code"**
|
||||
**child_val = > json.get value=`aw` key="value"**
|
||||
**last_obs = > json.get value=`aw` key="observation"**
|
||||
1. `code` == 0
|
||||
**exec_mode = "routed"**
|
||||
**out = > json.parse text={"status":"ok"}**
|
||||
**out = > json.set map=`out` key="status" value="ok"**
|
||||
**out = > json.set map=`out` key="goal" value=`task`**
|
||||
**out = > json.set map=`out` key="result" value=`child_val`**
|
||||
**out = > json.set map=`out` key="workbook" value=`path`**
|
||||
**out = > json.set map=`out` key="cache" value="soft-hit"**
|
||||
**out = > json.set map=`out` key="match" value="routed"**
|
||||
**sk = > json.get value=`applied` key="skill"**
|
||||
**out = > json.set map=`out` key="skill" value=`sk`**
|
||||
**out = > json.set map=`out` key="observation" value=`last_obs`**
|
||||
**out = > json.set map=`out` key="summary" value="router model selected skill"**
|
||||
**out = > json.set map=`out` key="route" value=`applied`**
|
||||
**route_matched = True**
|
||||
2. *
|
||||
**route_matched = False**
|
||||
2. *
|
||||
**route_matched = False**
|
||||
2. *
|
||||
**route_matched = False**
|
||||
3. *
|
||||
**_ = 1**
|
||||
|
||||
1. `route_matched`
|
||||
**_ = 1**
|
||||
2. *
|
||||
**resolved = > `self`.resolve task=`task` kb_dir=`kb_dir` near_match=`near_match` near_threshold=`near_threshold`**
|
||||
**mode = > json.get value=`resolved` key="mode"**
|
||||
**llm_free = > json.get value=`resolved` key="llm_free"**
|
||||
1. `llm_free`
|
||||
**path = > json.get value=`resolved` key="resource"**
|
||||
**aw = > await_workbook path=`path`**
|
||||
**code = > json.get value=`aw` key="code"**
|
||||
**child_val = > json.get value=`aw` key="value"**
|
||||
**last_obs = > json.get value=`aw` key="observation"**
|
||||
1. `code` == 0
|
||||
> agent_kb_record_hit kb_dir=`kb_dir` goal=`task` improve_every=`improve_every`
|
||||
**exec_mode = "compiled"**
|
||||
**out = > json.parse text={"status":"ok"}**
|
||||
**out = > json.set map=`out` key="status" value="ok"**
|
||||
**out = > json.set map=`out` key="goal" value=`task`**
|
||||
**out = > json.set map=`out` key="result" value=`child_val`**
|
||||
**out = > json.set map=`out` key="workbook" value=`path`**
|
||||
**out = > json.set map=`out` key="cache" value="hit"**
|
||||
**out = > json.set map=`out` key="match" value=`mode`**
|
||||
**sk = > json.get value=`resolved` key="skill"**
|
||||
**out = > json.set map=`out` key="skill" value=`sk`**
|
||||
**out = > json.set map=`out` key="observation" value=`last_obs`**
|
||||
**out = > json.set map=`out` key="summary" value="compiled skill (llm_free)"**
|
||||
**llm_calls = 0**
|
||||
2. *
|
||||
**out = > `self`.plan goal=`task` kb_dir=`kb_dir` writeback=`writeback` reuse=`reuse` near_match=`near_match` near_threshold=`near_threshold` promote=`promote` improve_every=`improve_every` explore_n=`explore_n` max_rounds=`max_rounds` stream=`stream` echo=`echo` trace=`trace` force=`force`**
|
||||
**exec_mode = "explore"**
|
||||
**llm_calls = 1**
|
||||
2. *
|
||||
**out = > `self`.plan goal=`task` kb_dir=`kb_dir` writeback=`writeback` reuse=`reuse` near_match=`near_match` near_threshold=`near_threshold` promote=`promote` improve_every=`improve_every` explore_n=`explore_n` max_rounds=`max_rounds` stream=`stream` echo=`echo` trace=`trace` force=`force`**
|
||||
**cache = > json.get value=`out` key="cache"**
|
||||
1. `cache` == hit
|
||||
**exec_mode = "compiled"**
|
||||
**llm_calls = 0**
|
||||
2. `cache` == soft-hit
|
||||
**exec_mode = "near"**
|
||||
**llm_calls = 0**
|
||||
3. *
|
||||
**exec_mode = "explore"**
|
||||
**llm_calls = 1**
|
||||
|
||||
**st = > json.get value=`out` key="status"**
|
||||
**res = > json.get value=`out` key="result"**
|
||||
**wb_path = > json.get value=`out` key="workbook"**
|
||||
**obs = > json.get value=`out` key="observation"**
|
||||
**sk2 = > json.get value=`out` key="skill"**
|
||||
**cache2 = > json.get value=`out` key="cache"**
|
||||
**match2 = > json.get value=`out` key="match"**
|
||||
**err = > json.get value=`out` key="error"**
|
||||
|
||||
1. `verify`
|
||||
1. `st` == "ok"
|
||||
**verification = "status: passed"**
|
||||
2. *
|
||||
**verification = "status: failed"**
|
||||
2. *
|
||||
**verification = "status: skipped"**
|
||||
|
||||
**ep = > `self`.record_episode task=`task` status=`st` result=`res` mode=`exec_mode` skill=`sk2` observation=`obs` workbook=`wb_path` cache=`cache2` match=`match2` llm_calls=`llm_calls` tokens=`tokens` error=`err` memory_dir=`memory_dir`**
|
||||
|
||||
**learning = None**
|
||||
1. `learn`
|
||||
1. `st` == "ok"
|
||||
**learning = > `self`.maybe_learn task=`task` kb_dir=`kb_dir` memory_dir=`memory_dir` improve_every=`improve_every` promote=`promote`**
|
||||
2. *
|
||||
**learning = None**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
**execution = > json.parse text={"mode":"explore","llm_calls":0,"tokens":0}**
|
||||
**execution = > json.set map=`execution` key="mode" value=`exec_mode`**
|
||||
**execution = > json.set map=`execution` key="llm_calls" value=`llm_calls`**
|
||||
**execution = > json.set map=`execution` key="tokens" value=`tokens`**
|
||||
**execution = > json.set map=`execution` key="skill" value=`sk2`**
|
||||
**execution = > json.set map=`execution` key="router" value=`route_backend`**
|
||||
**out = > json.set map=`out` key="execution" value=`execution`**
|
||||
**out = > json.set map=`out` key="episode" value=`ep`**
|
||||
**out = > json.set map=`out` key="route" value=`routed`**
|
||||
1. `learning`
|
||||
**out = > json.set map=`out` key="learning" value=`learning`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `resume_id`
|
||||
**_ckpt = > resume_save id=`resume_id` goal=`task` task=`task` status=`st` round=1 workbook=`wb_path` result=`res` route=`routed` resume_dir=`resume_dir`**
|
||||
**out = > json.set map=`out` key="resume" value=`_ckpt`**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
*out*
|
||||
|
||||
## step
|
||||
+ `task`
|
||||
+ `writeback`=True
|
||||
@@ -1539,17 +2012,20 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
|
||||
**reads_left = max_reads**
|
||||
**reply = None**
|
||||
**decision = None**
|
||||
**llm_calls = 0**
|
||||
**tokens = 0**
|
||||
|
||||
- `reads_left` > 0
|
||||
1. `decision`
|
||||
**reads_left = 0**
|
||||
2. *
|
||||
**ctx = > build_step_context agent=`self` task=`task` source_depth=`source_depth` skill_depth=`skill_depth`**
|
||||
1. `stream`
|
||||
**evs = > model.complete prompt=`ctx` stream=True echo=`echo`**
|
||||
**reply = > llm.stream_result events=`evs`**
|
||||
2. *
|
||||
**reply = > model.complete prompt=`ctx`**
|
||||
**turn = > model_turn model=`model` prompt=`ctx` stream=`stream` echo=`echo`**
|
||||
**reply = > json.get value=`turn` key="text"**
|
||||
**c = > json.get value=`turn` key="llm_calls"**
|
||||
**t = > json.get value=`turn` key="tokens"**
|
||||
**llm_calls = llm_calls + c**
|
||||
**tokens = tokens + t**
|
||||
**reply = > trim value=`reply`**
|
||||
**rk = > extract_plan_read reply=`reply`**
|
||||
1. `rk` == source
|
||||
@@ -1578,11 +2054,12 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
|
||||
**task_s = > json.stringify value=`task`**
|
||||
**fp = "Tool CALL was rejected (not on the tools whitelist). Do NOT emit CALL. Answer the user task directly with the final answer only.\n\nUser task:\n"**
|
||||
**fp = fp + task_s**
|
||||
1. `stream`
|
||||
**evs_retry = > model.complete prompt=`fp` stream=True echo=`echo`**
|
||||
**reply = > llm.stream_result events=`evs_retry`**
|
||||
2. *
|
||||
**reply = > model.complete prompt=`fp`**
|
||||
**turn2 = > model_turn model=`model` prompt=`fp` stream=`stream` echo=`echo`**
|
||||
**reply = > json.get value=`turn2` key="text"**
|
||||
**c = > json.get value=`turn2` key="llm_calls"**
|
||||
**t = > json.get value=`turn2` key="tokens"**
|
||||
**llm_calls = llm_calls + c**
|
||||
**tokens = tokens + t**
|
||||
**out = > json.set map=`out` key="status" value="ok"**
|
||||
**out = > json.set map=`out` key="decision" value=`decision`**
|
||||
**out = > json.set map=`out` key="denied_tool" value=`tool_name`**
|
||||
@@ -1596,17 +2073,21 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
|
||||
**fp = fp + "ran via subtask and returned:"**
|
||||
**fp = fp + tool_s**
|
||||
**fp = fp + ". Reply to the user briefly."**
|
||||
1. `stream`
|
||||
**evs2 = > model.complete prompt=`fp` stream=True echo=`echo`**
|
||||
**reply = > llm.stream_result events=`evs2`**
|
||||
2. *
|
||||
**reply = > model.complete prompt=`fp`**
|
||||
**turn2 = > model_turn model=`model` prompt=`fp` stream=`stream` echo=`echo`**
|
||||
**reply = > json.get value=`turn2` key="text"**
|
||||
**c = > json.get value=`turn2` key="llm_calls"**
|
||||
**t = > json.get value=`turn2` key="tokens"**
|
||||
**llm_calls = llm_calls + c**
|
||||
**tokens = tokens + t**
|
||||
**out = > json.set map=`out` key="tool" value=`tool_name`**
|
||||
**out = > json.set map=`out` key="tool_result" value=`tool_s`**
|
||||
**out = > json.set map=`out` key="result" value=`reply`**
|
||||
2. *
|
||||
**out = > json.set map=`out` key="result" value=`reply`**
|
||||
|
||||
**out = > json.set map=`out` key="llm_calls" value=`llm_calls`**
|
||||
**out = > json.set map=`out` key="tokens" value=`tokens`**
|
||||
|
||||
**as_turn = > json.parse text={"role":"assistant"}**
|
||||
**as_turn = > json.set map=`as_turn` key="content" value=`reply`**
|
||||
> agent_history_append id=`id` item=`as_turn`
|
||||
@@ -1623,6 +2104,51 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
|
||||
|
||||
*out*
|
||||
|
||||
## preflight
|
||||
+ `task`
|
||||
+ `root`="."
|
||||
+ `marqdo_dir`=None
|
||||
+ `min_candidates`=3
|
||||
+ `allow_create_after`=2
|
||||
+ `reuse_threshold`=0.72
|
||||
+ `adapt_threshold`=0.45
|
||||
|
||||
Engineering First Information gate (008 / EKC). Reads compiled `.marqdo` knowledge.
|
||||
Returns `{status,decision,create_allowed,existing,constraints,decisions,failures,recommended,context_path}`.
|
||||
Hard rule: generate only when `create_allowed` is true (or decision is REUSE/ADAPT).
|
||||
|
||||
1. `marqdo_dir`
|
||||
**pf = > agent_eng_preflight task=`task` root=`root` marqdo_dir=`marqdo_dir` min_candidates=`min_candidates` allow_create_after=`allow_create_after` reuse_threshold=`reuse_threshold` adapt_threshold=`adapt_threshold`**
|
||||
2. *
|
||||
**pf = > agent_eng_preflight task=`task` root=`root` min_candidates=`min_candidates` allow_create_after=`allow_create_after` reuse_threshold=`reuse_threshold` adapt_threshold=`adapt_threshold`**
|
||||
|
||||
*pf*
|
||||
|
||||
## eng_reuse
|
||||
+ `task`
|
||||
+ `root`="."
|
||||
+ `marqdo_dir`=None
|
||||
|
||||
Alias of `preflight` focused on REUSE / ADAPT / CREATE payload.
|
||||
|
||||
1. `marqdo_dir`
|
||||
*> agent_eng_reuse task=`task` root=`root` marqdo_dir=`marqdo_dir`*
|
||||
2. *
|
||||
*> agent_eng_reuse task=`task` root=`root`*
|
||||
|
||||
## eng_record
|
||||
+ `decision`
|
||||
+ `root`="."
|
||||
+ `marqdo_dir`=None
|
||||
+ `duplicated`=False
|
||||
|
||||
Record REUSE/ADAPT/CREATE into `.marqdo/agent/episodes/reuse_metrics.json`.
|
||||
|
||||
1. `marqdo_dir`
|
||||
*> agent_eng_record decision=`decision` root=`root` marqdo_dir=`marqdo_dir` duplicated=`duplicated`*
|
||||
2. *
|
||||
*> agent_eng_record decision=`decision` root=`root` duplicated=`duplicated`*
|
||||
|
||||
## plan
|
||||
+ `goal`
|
||||
+ `workbook_dir`=None
|
||||
@@ -1644,9 +2170,16 @@ With `stream=True`, the model call uses SSE; `echo=True` prints delta text to st
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
+ `trace`=False
|
||||
+ `resume_id`=None
|
||||
+ `resume_dir`=.marqdo/agent-resume
|
||||
+ `eng_preflight`=False
|
||||
+ `marqdo_dir`=.marqdo
|
||||
+ `allow_create_after`=2
|
||||
|
||||
Multi-step with OKF agent-kb. Default workbook is `kb_dir/resources/<slug>.mq.md`. While task file count `< explore_n` and skill is not llm_free, force a new explore variant under `kb_dir/explore/<slug>/`. Code-first: llm_free hits skip parent LLM. File children return via `# main`; `plan` exposes that as `result`.
|
||||
|
||||
Engineering Knowledge preflight when `eng_preflight=True`: REUSE/ADAPT preferred; CREATE only if `create_allowed`. Production agents should enable this; tests that only exercise agent-kb may leave it false.
|
||||
|
||||
Reuse lookup: exact → alias → canonicalize → optional local n-gram `near` when `near_match=True` and score ≥ `near_threshold`. Non-hit path: optional `soft_match=True` parent REUSE/NEW over ranked `agent_kb_near_match` candidates; else decompose before first child spawn (`DECISION: RUN` / `CONTINUE`+patch / solidified `DONE`). Then `await` → revise loop. Process events (`decision` / `round` / `done` / tools) are always attached on the result map for audit and view process cards; `stream=True` additionally SSE-publishes them (and parent `delta`). `echo=True` prints `plan:decompose` / `plan:await` / deltas. `trace=True` writes events to writeback slot `trace`. Quiet child subtasks stay quiet.
|
||||
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
@@ -1657,6 +2190,32 @@ Reuse lookup: exact → alias → canonicalize → optional local n-gram `near`
|
||||
**explore_attempt = None**
|
||||
**skel_kind = skeleton**
|
||||
**events = > json.parse text=[]**
|
||||
**eng = None**
|
||||
|
||||
1. `eng_preflight`
|
||||
**eng = > agent_eng_preflight task=`goal` marqdo_dir=`marqdo_dir` allow_create_after=`allow_create_after`**
|
||||
**eng_status = > json.get value=`eng` key="status"**
|
||||
**eng_decision = > json.get value=`eng` key="decision"**
|
||||
**eng_allowed = > json.get value=`eng` key="create_allowed"**
|
||||
1. `eng_status` == missing_knowledge
|
||||
**_ = 1**
|
||||
2. `eng_decision` == REUSE
|
||||
> agent_eng_record decision="REUSE" marqdo_dir=`marqdo_dir`
|
||||
**_ = 1**
|
||||
3. `eng_decision` == ADAPT
|
||||
> agent_eng_record decision="ADAPT" marqdo_dir=`marqdo_dir`
|
||||
**_ = 1**
|
||||
4. `eng_allowed`
|
||||
> agent_eng_record decision="CREATE" marqdo_dir=`marqdo_dir`
|
||||
**_ = 1**
|
||||
5. *
|
||||
**out = > json.parse text={"status":"blocked","cache":"eng-block","eng":true}**
|
||||
**out = > json.set map=`out` key="decision" value="BLOCKED"**
|
||||
**out = > json.set map=`out` key="preflight" value=`eng`**
|
||||
**out = > json.set map=`out` key="events" value=`events`**
|
||||
*out*
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
**tf = > agent_kb_task_files kb_dir=`kb_dir` goal=`goal` tools=`tools`**
|
||||
**nfiles = > json.get value=`tf` key="count"**
|
||||
@@ -1799,11 +2358,12 @@ Reuse lookup: exact → alias → canonicalize → optional local n-gram `near`
|
||||
**soft_prompt = > build_soft_match_prompt goal=`goal` tasks=`tasks`**
|
||||
**model = > json.get value=`self` key="model"**
|
||||
1. `stream`
|
||||
**evs = > model.complete prompt=`soft_prompt` stream=True echo=`echo`**
|
||||
**soft_reply = > llm.stream_result events=`evs`**
|
||||
**evs = > model.stream prompt=`soft_prompt` echo=`echo`**
|
||||
**soft_reply = > llm.collect events=`evs`**
|
||||
**events = > plan_merge_deltas events=`events` from=`evs` stream=`stream`**
|
||||
2. *
|
||||
**soft_reply = > model.complete prompt=`soft_prompt`**
|
||||
**soft_turn = > model_turn model=`model` prompt=`soft_prompt` stream=False echo=`echo`**
|
||||
**soft_reply = > json.get value=`soft_turn` key="text"**
|
||||
**soft_reply = > trim value=`soft_reply`**
|
||||
**soft_dec = > extract_plan_decision reply=`soft_reply`**
|
||||
**events = > plan_append_decision events=`events` decision=`soft_dec` stream=`stream`**
|
||||
@@ -1898,18 +2458,28 @@ Reuse lookup: exact → alias → canonicalize → optional local n-gram `near`
|
||||
**b = > json.get value=`parts` key="b"**
|
||||
**c = > json.get value=`parts` key="c"**
|
||||
**path = workbook_dir + a + slug + b + ts + c**
|
||||
> fs.write_text path=`path` text=`skel`
|
||||
2. `explore`
|
||||
**parts = > json.parse text={"a":"/explore/","b":"/","c":".mq.md"}**
|
||||
**a = > json.get value=`parts` key="a"**
|
||||
**b = > json.get value=`parts` key="b"**
|
||||
**c = > json.get value=`parts` key="c"**
|
||||
**path = kb_dir + a + slug + b + explore_attempt + c**
|
||||
**ex = > fs.exists path=`path`**
|
||||
1. `ex`
|
||||
**_ = 1**
|
||||
2. *
|
||||
> fs.write_text path=`path` text=`skel`
|
||||
3. *
|
||||
**parts = > json.parse text={"a":"/resources/","b":".mq.md"}**
|
||||
**a = > json.get value=`parts` key="a"**
|
||||
**b = > json.get value=`parts` key="b"**
|
||||
**path = kb_dir + a + slug + b**
|
||||
> fs.write_text path=`path` text=`skel`
|
||||
**ex = > fs.exists path=`path`**
|
||||
1. `ex`
|
||||
**_ = 1**
|
||||
2. *
|
||||
> fs.write_text path=`path` text=`skel`
|
||||
|
||||
1. `confirm`
|
||||
**out = > json.parse text={"status":"pending"}**
|
||||
|
||||
+304
-68
@@ -1,23 +1,26 @@
|
||||
---
|
||||
title: ext/ai/llm
|
||||
description: OpenAI-compatible LLM object (English). Import ext/ai/llm.mq.md.
|
||||
description: >-
|
||||
Marqdo LLM intelligence primitive — ask / stream / collect / named handles.
|
||||
OpenAI-compatible and Ollama HTTP stay behind the semantic surface.
|
||||
import sys:lib/sys.mq.md
|
||||
import net:lib/net.mq.md
|
||||
import json:lib/json.mq.md
|
||||
import table:lib/table.mq.md
|
||||
import fs:lib/fs.mq.md
|
||||
import text:lib/text.mq.md
|
||||
import openai:ext/ai/llm/openai.mq.md
|
||||
---
|
||||
|
||||
## load_env
|
||||
+ `path`=None
|
||||
|
||||
Load `.env` from cwd (optional named arg `path=`). Does not override existing variables.
|
||||
Load `.env` from cwd (optional `path=`). Does not override existing variables.
|
||||
|
||||
*> sys.load_dotenv path=`path`*
|
||||
|
||||
## stream_result
|
||||
## collect
|
||||
+ `events`
|
||||
|
||||
Reduce a `complete stream=True` event list to the final `done.result` text (or `error.message`). Unknown types (e.g. `reasoning`) are ignored.
|
||||
Collapse a stream event list to final answer text (`done.result` or `error.message`). Prefer this over raw SSE fields. Alias: `stream_result`.
|
||||
|
||||
**answer = ""**
|
||||
|
||||
@@ -28,97 +31,330 @@ Reduce a `complete stream=True` event list to the final `done.result` text (or `
|
||||
2. `t` == "error"
|
||||
**answer = [message](ev)**
|
||||
3. `t` == "delta"
|
||||
4. *
|
||||
4. `t` == "text"
|
||||
**chunk = [text](ev)**
|
||||
1. `chunk`
|
||||
**answer = answer + chunk**
|
||||
2. *
|
||||
**_ = 1**
|
||||
5. *
|
||||
**_ = 1**
|
||||
|
||||
*answer*
|
||||
|
||||
# llm
|
||||
## stream_result
|
||||
+ `events`
|
||||
|
||||
Construct an LLM handle from `OPENAI_*` / `MARQDO_LLM_*`.
|
||||
Compatibility alias for `collect`.
|
||||
|
||||
**api_key = > sys.env_get name="OPENAI_API_KEY"**
|
||||
1. not `api_key`
|
||||
**api_key = > sys.env_get name="MARQDO_LLM_API_KEY"**
|
||||
*> collect events=`events`*
|
||||
|
||||
1. not `api_key`
|
||||
> print text=ext/ai/llm: set OPENAI_API_KEY or MARQDO_LLM_API_KEY
|
||||
## usage_from
|
||||
+ `data`
|
||||
|
||||
Extract `{prompt_tokens,completion_tokens,total_tokens}` from an OpenAI-shaped response map (or empty zeros).
|
||||
|
||||
*> openai.usage_from data=`data`*
|
||||
|
||||
## create
|
||||
+ `model`=None
|
||||
+ `base_url`=None
|
||||
+ `api_key`=None
|
||||
+ `backend`=openai-compatible
|
||||
+ `name`=None
|
||||
|
||||
Factory for an LLM handle (same as constructing `# llm`). Env fallbacks apply inside the constructor. Optional `name=` tags the handle (`fast` / `reasoning`).
|
||||
|
||||
**h = > llm model=`model` base_url=`base_url` api_key=`api_key` backend=`backend` name=`name`**
|
||||
*h*
|
||||
|
||||
## fast
|
||||
+ `model`=None
|
||||
+ `base_url`=None
|
||||
+ `api_key`=None
|
||||
|
||||
Named small/fast handle. Model from arg, else `MARQDO_LLM_FAST_MODEL`, else default create model.
|
||||
|
||||
1. `model`
|
||||
**mdl = model**
|
||||
2. *
|
||||
**mdl = > sys.env_get name="MARQDO_LLM_FAST_MODEL"**
|
||||
|
||||
*> create model=`mdl` base_url=`base_url` api_key=`api_key` name="fast" backend="openai-compatible"*
|
||||
|
||||
## reasoning
|
||||
+ `model`=None
|
||||
+ `base_url`=None
|
||||
+ `api_key`=None
|
||||
|
||||
Named strong/reasoning handle. Model from arg, else `MARQDO_LLM_REASONING_MODEL`, else default create model.
|
||||
|
||||
1. `model`
|
||||
**mdl = model**
|
||||
2. *
|
||||
**mdl = > sys.env_get name="MARQDO_LLM_REASONING_MODEL"**
|
||||
|
||||
*> create model=`mdl` base_url=`base_url` api_key=`api_key` name="reasoning" backend="openai-compatible"*
|
||||
|
||||
## prompt_body
|
||||
+ `src`
|
||||
|
||||
If `src` starts with YAML frontmatter declaring `type: prompt` / `类型: 提示`, return the body after the closing `---`; otherwise return `src` unchanged.
|
||||
|
||||
**sw = > text.starts_with text=`src` prefix="---"**
|
||||
1. not `sw`
|
||||
*src*
|
||||
2. *
|
||||
**parts = > split value=`src` sep="\n---\n"**
|
||||
**n = > len value=`parts`**
|
||||
1. `n` < 2
|
||||
*src*
|
||||
2. *
|
||||
**fm = > at value=`parts` index=0**
|
||||
**is_p = > text.contains text=`fm` sub="type: prompt"**
|
||||
1. not `is_p`
|
||||
**is_p = > text.contains text=`fm` sub="类型: 提示"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. `is_p`
|
||||
**body = > at value=`parts` index=1**
|
||||
*> text.str_trim s=`body`*
|
||||
2. *
|
||||
*src*
|
||||
|
||||
## prompt_load
|
||||
+ `path`
|
||||
|
||||
Load a prompt artifact (`.mq.md` / `.md`) from disk for `ask`.
|
||||
|
||||
**raw = > fs.read_text path=`path`**
|
||||
*> prompt_body src=`raw`*
|
||||
|
||||
## prompt
|
||||
+ `path`=None
|
||||
+ `text`=None
|
||||
|
||||
Prompt artifact helper: `path=` loads a file; `text=` strips `type: prompt` frontmatter from an in-memory string.
|
||||
|
||||
1. `path`
|
||||
*> prompt_load path=`path`*
|
||||
2. `text`
|
||||
*> prompt_body src=`text`*
|
||||
3. *
|
||||
> print text=ext/ai/llm.prompt: pass path= or text=
|
||||
> sys.exit code=1
|
||||
|
||||
**base_url = > sys.env_get name="OPENAI_BASE_URL"**
|
||||
1. not `base_url`
|
||||
**base_url = > sys.env_get name="MARQDO_LLM_BASE_URL"**
|
||||
1. not `base_url`
|
||||
**base_url = "https://api.openai.com/v1"**
|
||||
## ask
|
||||
+ `prompt`=None
|
||||
+ `path`=None
|
||||
+ `model`=None
|
||||
+ `base_url`=None
|
||||
+ `api_key`=None
|
||||
|
||||
**model = > sys.env_get name="OPENAI_MODEL"**
|
||||
1. not `model`
|
||||
**model = > sys.env_get name="MARQDO_LLM_MODEL"**
|
||||
1. not `model`
|
||||
**model = "gpt-4o-mini"**
|
||||
Module convenience: create a default handle and return **answer text**. Pass `path=` to load a prompt document, `prompt=` text, or omit both to use the entry `.mq.md` as a prompt artifact (`type: prompt` body via `sys.module_source`).
|
||||
|
||||
1. `path`
|
||||
**p = > prompt_load path=`path`**
|
||||
2. `prompt`
|
||||
**p = prompt**
|
||||
3. *
|
||||
**raw = > sys.module_source**
|
||||
1. not `raw`
|
||||
> print text=ext/ai/llm.ask: pass prompt= or path= (or run a prompt document as entry)
|
||||
> sys.exit code=1
|
||||
2. *
|
||||
**p = > prompt_body src=`raw`**
|
||||
|
||||
**m = > create model=`model` base_url=`base_url` api_key=`api_key`**
|
||||
**r = > m.ask prompt=`p`**
|
||||
*[text](r)*
|
||||
|
||||
# llm
|
||||
+ `model`=None
|
||||
+ `base_url`=None
|
||||
+ `api_key`=None
|
||||
+ `backend`=openai-compatible
|
||||
+ `name`=None
|
||||
|
||||
LLM handle — intelligence primitive, not an HTTP client.
|
||||
|
||||
Resolution: explicit args → entry-scope / Artifact Metadata (`sys.meta_get`) → `sys.env_get` → defaults.
|
||||
Prefer declaring `model` / `api_key` / `base_url` in frontmatter Binding, then use the lifted names.
|
||||
|
||||
1. `api_key`
|
||||
**key = api_key**
|
||||
2. *
|
||||
**key = > sys.meta_get key="api_key"**
|
||||
1. not `key`
|
||||
**key = > sys.env_get name="OPENAI_API_KEY"**
|
||||
1. not `key`
|
||||
**key = > sys.env_get name="MARQDO_LLM_API_KEY"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `backend` == ollama
|
||||
1. not `key`
|
||||
**key = "ollama"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. not `key`
|
||||
> print text=ext/ai/llm: set OPENAI_API_KEY or MARQDO_LLM_API_KEY (or pass api_key= / metadata api_key)
|
||||
> sys.exit code=1
|
||||
3. *
|
||||
**_ = 1**
|
||||
|
||||
1. `base_url`
|
||||
**url = base_url**
|
||||
2. *
|
||||
**url = > sys.meta_get key="base_url"**
|
||||
1. `url`
|
||||
**_ = 1**
|
||||
2. `backend` == ollama
|
||||
**url = > sys.env_get name="OLLAMA_HOST"**
|
||||
1. not `url`
|
||||
**url = "http://127.0.0.1:11434/v1"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
3. *
|
||||
**url = > sys.env_get name="OPENAI_BASE_URL"**
|
||||
1. not `url`
|
||||
**url = > sys.env_get name="MARQDO_LLM_BASE_URL"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. not `url`
|
||||
**url = "https://api.openai.com/v1"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `model`
|
||||
**mdl = model**
|
||||
2. *
|
||||
**mdl = > sys.meta_get key="model"**
|
||||
1. `mdl`
|
||||
**_ = 1**
|
||||
2. *
|
||||
**mdl = > sys.env_get name="OPENAI_MODEL"**
|
||||
1. not `mdl`
|
||||
**mdl = > sys.env_get name="MARQDO_LLM_MODEL"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. not `mdl`
|
||||
1. `backend` == ollama
|
||||
**mdl = "llama3.2"**
|
||||
2. *
|
||||
**mdl = "gpt-4o-mini"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `name`
|
||||
**nm = name**
|
||||
2. *
|
||||
**nm = None**
|
||||
|
||||
`h` =
|
||||
|
||||
| api_key | base_url | model | suffix | bearer |
|
||||
|---------|----------|-------|--------|--------|
|
||||
| `api_key` | `base_url` | `model` | /chat/completions | "Bearer " |
|
||||
| _type | backend | api_key | base_url | model | suffix | bearer | name |
|
||||
|-------|---------|---------|----------|-------|--------|--------|------|
|
||||
| llm | `backend` | `key` | `url` | `mdl` | /chat/completions | "Bearer " | `nm` |
|
||||
|
||||
*h*
|
||||
|
||||
## ask
|
||||
+ `prompt`=None
|
||||
+ `path`=None
|
||||
|
||||
Semantic ask → **LLMResult** (`text`, `model`, `finish`, `backend`, `usage`, `name`, `tool_calls`). Prefer `[text](result)` for the answer string. Omit `prompt=` / `path=` to use the entry document (`sys.module_source` + `prompt_body`).
|
||||
|
||||
1. `path`
|
||||
**p = > prompt_load path=`path`**
|
||||
2. `prompt`
|
||||
**p = prompt**
|
||||
3. *
|
||||
**raw = > sys.module_source**
|
||||
1. not `raw`
|
||||
> print text=ext/ai/llm: ask needs prompt= or path= (or entry prompt document)
|
||||
> sys.exit code=1
|
||||
2. *
|
||||
**p = > prompt_body src=`raw`**
|
||||
|
||||
**pack = > self.complete_result prompt=`p` stream=False echo=False**
|
||||
**text = > json.get value=`pack` key="text"**
|
||||
**usage = > json.get value=`pack` key="usage"**
|
||||
**finish = > json.get value=`pack` key="finish"**
|
||||
**tc = > json.get value=`pack` key="tool_calls"**
|
||||
1. not `tc`
|
||||
**tc = > json.parse text=[]**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
Hoist receiver fields before the result table — table cells can shadow `self` /
|
||||
column names (`model`, `usage`), so `[model](self)` / `[total_tokens](usage)`
|
||||
inside the table is unsafe.
|
||||
|
||||
**mdl = [model](self)**
|
||||
**b = [backend](self)**
|
||||
**nm = [name](self)**
|
||||
**tt = [total_tokens](usage)**
|
||||
|
||||
`result` =
|
||||
|
||||
| text | model | finish | backend | usage | name | llm_calls | tokens | tool_calls |
|
||||
|------|-------|--------|---------|-------|------|-----------|--------|------------|
|
||||
| `text` | `mdl` | `finish` | `b` | `usage` | `nm` | 1 | `tt` | `tc` |
|
||||
|
||||
*result*
|
||||
|
||||
## stream
|
||||
+ `prompt`=None
|
||||
+ `path`=None
|
||||
+ `echo`=False
|
||||
|
||||
Semantic stream: return the event list (use `llm.collect` to reduce to text). Omit `prompt=` / `path=` to use the entry document.
|
||||
|
||||
1. `path`
|
||||
**p = > prompt_load path=`path`**
|
||||
2. `prompt`
|
||||
**p = prompt**
|
||||
3. *
|
||||
**raw = > sys.module_source**
|
||||
1. not `raw`
|
||||
> print text=ext/ai/llm: stream needs prompt= or path= (or entry prompt document)
|
||||
> sys.exit code=1
|
||||
2. *
|
||||
**p = > prompt_body src=`raw`**
|
||||
|
||||
*> self.complete prompt=`p` stream=True echo=`echo`*
|
||||
|
||||
## complete_result
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
|
||||
Transport pack via `ext/ai/llm/openai` (Ollama uses the same OpenAI-compatible path). Non-stream → `{text,usage,finish}`; stream → `{events}`.
|
||||
|
||||
*> openai.chat_completions base_url=[base_url](self) api_key=[api_key](self) model=[model](self) prompt=`prompt` stream=`stream` echo=`echo` suffix=[suffix](self) bearer=[bearer](self)*
|
||||
|
||||
## complete
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
|
||||
Chat completion using `self` handle fields.
|
||||
|
||||
Wire body is `{model, messages:[{role,content}], stream?}`. Messages are an `@` row table; headers and optional `stream` use `table.put`. `json` is only for stringify/parse.
|
||||
|
||||
**url = [base_url](self) + [suffix](self)**
|
||||
**auth = [bearer](self) + [api_key](self)**
|
||||
**headers = > table.put in=None at="Authorization" value=`auth`**
|
||||
|
||||
`messages` =
|
||||
|
||||
| @ | role | content |
|
||||
|---|------|---------|
|
||||
| 1 | user | `prompt` |
|
||||
|
||||
`req` =
|
||||
|
||||
| model | messages |
|
||||
|-------|----------|
|
||||
| [model](self) | `messages` |
|
||||
Internal / compatibility primitive. Prefer `ask` / `stream` in new code.
|
||||
|
||||
1. `stream`
|
||||
**req = > table.put in=`req` at="stream" value=True**
|
||||
**body = > json.stringify value=`req`**
|
||||
**resp = > net.http_post_sse url=`url` body=`body` headers=`headers` echo=`echo`**
|
||||
1. [status](resp) == 200
|
||||
*[events](resp)*
|
||||
2. *
|
||||
> print text=ext/ai/llm: HTTP error (stream)
|
||||
> print text=[status](resp)
|
||||
> sys.exit code=1
|
||||
**pack = > self.complete_result prompt=`prompt` stream=True echo=`echo`**
|
||||
*[events](pack)*
|
||||
2. *
|
||||
**body = > json.stringify value=`req`**
|
||||
**resp = > net.http_post url=`url` body=`body` headers=`headers`**
|
||||
1. [status](resp) == 200
|
||||
**data = > json.parse text=[body](resp)**
|
||||
*[content]([message]([1]([choices](data))))*
|
||||
2. *
|
||||
> print text=ext/ai/llm: HTTP error
|
||||
> print text=[status](resp)
|
||||
> print text=[body](resp)
|
||||
> sys.exit code=1
|
||||
|
||||
---
|
||||
**pack = > self.complete_result prompt=`prompt` stream=False echo=`echo`**
|
||||
*[text](pack)*
|
||||
|
||||
## chat
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
|
||||
Alias for `complete`.
|
||||
Alias for `complete` (compat).
|
||||
|
||||
*> self.complete prompt=`prompt` stream=`stream` echo=`echo`*
|
||||
@@ -0,0 +1,55 @@
|
||||
---
|
||||
title: ext/ai/llm/ollama
|
||||
description: >-
|
||||
Ollama backend defaults. Transport reuses openai-compatible /v1 chat completions.
|
||||
import openai:ext/ai/llm/openai.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
---
|
||||
|
||||
## resolve
|
||||
+ `model`=None
|
||||
+ `base_url`=None
|
||||
+ `api_key`=None
|
||||
|
||||
Resolve Ollama handle fields: default host `http://127.0.0.1:11434/v1`, key `ollama`, model `llama3.2`.
|
||||
|
||||
1. `api_key`
|
||||
**key = api_key**
|
||||
2. *
|
||||
**key = "ollama"**
|
||||
|
||||
1. `base_url`
|
||||
**url = base_url**
|
||||
2. *
|
||||
**url = > sys.env_get name="OLLAMA_HOST"**
|
||||
1. not `url`
|
||||
**url = "http://127.0.0.1:11434/v1"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `model`
|
||||
**mdl = model**
|
||||
2. *
|
||||
**mdl = > sys.env_get name="MARQDO_LLM_MODEL"**
|
||||
1. not `mdl`
|
||||
**mdl = "llama3.2"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
`cfg` =
|
||||
|
||||
| backend | api_key | base_url | model |
|
||||
|---------|---------|----------|-------|
|
||||
| ollama | `key` | `url` | `mdl` |
|
||||
|
||||
*cfg*
|
||||
|
||||
## chat_completions
|
||||
+ `base_url`
|
||||
+ `api_key`
|
||||
+ `model`
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
|
||||
*> openai.chat_completions base_url=`base_url` api_key=`api_key` model=`model` prompt=`prompt` stream=`stream` echo=`echo`*
|
||||
@@ -0,0 +1,145 @@
|
||||
---
|
||||
title: ext/ai/llm/openai
|
||||
description: >-
|
||||
OpenAI-compatible chat completions transport (HTTP + SSE).
|
||||
Prefers optional plugins/llm ABI when present; falls back to lib/net.
|
||||
Used by ext/ai/llm; authors should prefer llm.ask / llm.stream.
|
||||
import net:lib/net.mq.md
|
||||
import json:lib/json.mq.md
|
||||
import table:lib/table.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
import plugin:lib/plugin.mq.md
|
||||
---
|
||||
|
||||
## usage_from
|
||||
+ `data`
|
||||
|
||||
Extract `{prompt_tokens,completion_tokens,total_tokens}` from an OpenAI-shaped response map.
|
||||
|
||||
**u = > json.get value=`data` key="usage"**
|
||||
1. `u`
|
||||
**pt = > json.get value=`u` key="prompt_tokens"**
|
||||
**ct = > json.get value=`u` key="completion_tokens"**
|
||||
**tt = > json.get value=`u` key="total_tokens"**
|
||||
1. not `pt`
|
||||
**pt = 0**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. not `ct`
|
||||
**ct = 0**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. not `tt`
|
||||
**tt = pt + ct**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. *
|
||||
**pt = 0**
|
||||
**ct = 0**
|
||||
**tt = 0**
|
||||
|
||||
`usage` =
|
||||
|
||||
| prompt_tokens | completion_tokens | total_tokens |
|
||||
|---------------|-------------------|--------------|
|
||||
| `pt` | `ct` | `tt` |
|
||||
|
||||
*usage*
|
||||
|
||||
## tool_calls_from
|
||||
+ `data`
|
||||
|
||||
Extract OpenAI-shaped `choices[0].message.tool_calls` as a list (or empty list).
|
||||
|
||||
**choices = > json.get value=`data` key="choices"**
|
||||
1. not `choices`
|
||||
**empty = > json.parse text=[]**
|
||||
*empty*
|
||||
2. *
|
||||
**c0 = [1](choices)**
|
||||
**msg = > json.get value=`c0` key="message"**
|
||||
1. not `msg`
|
||||
**empty = > json.parse text=[]**
|
||||
*empty*
|
||||
2. *
|
||||
**tc = > json.get value=`msg` key="tool_calls"**
|
||||
1. `tc`
|
||||
*tc*
|
||||
2. *
|
||||
**empty = > json.parse text=[]**
|
||||
*empty*
|
||||
|
||||
## chat_completions
|
||||
+ `base_url`
|
||||
+ `api_key`
|
||||
+ `model`
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
+ `suffix`=/chat/completions
|
||||
+ `bearer`="Bearer "
|
||||
|
||||
POST `/chat/completions` (or stream SSE). Prefers `plugins/llm` ABI (`llm_chat_completions`) when the native plugin is resolvable; otherwise `lib/net` HTTP/SSE. Non-stream → `{text,usage,finish,tool_calls}`; stream → `{events,finish}`.
|
||||
|
||||
**p = > plugin.native_path name="llm"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
*> llm_chat_completions base_url=`base_url` api_key=`api_key` model=`model` prompt=`prompt` stream=`stream` echo=`echo` suffix=`suffix` bearer=`bearer`*
|
||||
2. *
|
||||
**url = base_url + suffix**
|
||||
**auth = bearer + api_key**
|
||||
**headers = > table.put in=None at="Authorization" value=`auth`**
|
||||
|
||||
`messages` =
|
||||
|
||||
| @ | role | content |
|
||||
|---|------|---------|
|
||||
| 1 | user | `prompt` |
|
||||
|
||||
`req` =
|
||||
|
||||
| model | messages |
|
||||
|-------|----------|
|
||||
| `model` | `messages` |
|
||||
|
||||
1. `stream`
|
||||
**req = > table.put in=`req` at="stream" value=True**
|
||||
**body = > json.stringify value=`req`**
|
||||
**resp = > net.http_post_sse url=`url` body=`body` headers=`headers` echo=`echo`**
|
||||
1. [status](resp) == 200
|
||||
**pack = > json.parse text={"finish":"stop"}**
|
||||
**pack = > json.set map=`pack` key="events" value=[events](resp)**
|
||||
*pack*
|
||||
2. *
|
||||
> print text=ext/ai/llm/openai: HTTP error (stream)
|
||||
> print text=[status](resp)
|
||||
> sys.exit code=1
|
||||
2. *
|
||||
**body = > json.stringify value=`req`**
|
||||
**resp = > net.http_post url=`url` body=`body` headers=`headers`**
|
||||
1. [status](resp) == 200
|
||||
**data = > json.parse text=[body](resp)**
|
||||
**msg = [message]([1]([choices](data)))**
|
||||
**text = > json.get value=`msg` key="content"**
|
||||
1. not `text`
|
||||
**text = ""**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**usage = > usage_from data=`data`**
|
||||
**fr = > json.get value=[1]([choices](data)) key="finish_reason"**
|
||||
1. not `fr`
|
||||
**fr = "stop"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**tc = > tool_calls_from data=`data`**
|
||||
**pack = > json.parse text={"finish":"stop"}**
|
||||
**pack = > json.set map=`pack` key="text" value=`text`**
|
||||
**pack = > json.set map=`pack` key="usage" value=`usage`**
|
||||
**pack = > json.set map=`pack` key="finish" value=`fr`**
|
||||
**pack = > json.set map=`pack` key="tool_calls" value=`tc`**
|
||||
*pack*
|
||||
2. *
|
||||
> print text=ext/ai/llm/openai: HTTP error
|
||||
> print text=[status](resp)
|
||||
> print text=[body](resp)
|
||||
> sys.exit code=1
|
||||
+245
-72
@@ -1,24 +1,23 @@
|
||||
---
|
||||
title: ext/ai/大模型
|
||||
description: OpenAI 兼容大模型对象(中文)。导入 ext/ai/大模型.mq.md。
|
||||
description: >-
|
||||
Marqdo 大模型智能原语 — 提问 / 流式 / 收集 / 命名句柄。
|
||||
OpenAI 兼容与 Ollama HTTP 藏在语义面之后。
|
||||
导入 系统:lib/系统.mq.md
|
||||
导入 网络:lib/网络.mq.md
|
||||
import json:lib/json.mq.md
|
||||
导入 表:lib/表.mq.md
|
||||
导入 文件:lib/文件.mq.md
|
||||
导入 文本:lib/文本.mq.md
|
||||
import openai:ext/ai/llm/openai.mq.md
|
||||
---
|
||||
|
||||
## 加载环境
|
||||
+ `路径`=None
|
||||
|
||||
从工作目录加载 `.env`(可选命名实参 `路径=`)。不覆盖已有变量。
|
||||
|
||||
*> 系统.加载环境 路径=`路径`*
|
||||
|
||||
## 流式结果
|
||||
## 收集
|
||||
+ `事件`
|
||||
|
||||
把 `运行 流式=真` 的事件列表收成最终 `done.result`(或 `error.message`)。未知类型(如 `reasoning`)忽略。
|
||||
|
||||
**answer = ""**
|
||||
|
||||
- [ev](事件)
|
||||
@@ -28,97 +27,271 @@ import json:lib/json.mq.md
|
||||
2. `t` == "error"
|
||||
**answer = [message](ev)**
|
||||
3. `t` == "delta"
|
||||
4. *
|
||||
4. `t` == "text"
|
||||
**chunk = [text](ev)**
|
||||
1. `chunk`
|
||||
**answer = answer + chunk**
|
||||
2. *
|
||||
**_ = 1**
|
||||
5. *
|
||||
**_ = 1**
|
||||
|
||||
*answer*
|
||||
|
||||
# 大模型
|
||||
## 流式结果
|
||||
+ `事件`
|
||||
|
||||
从 `OPENAI_*` / `MARQDO_LLM_*` 构造句柄。
|
||||
*> 收集 事件=`事件`*
|
||||
|
||||
**api_key = > 系统.取环境 名="OPENAI_API_KEY"**
|
||||
1. 非 `api_key`
|
||||
**api_key = > 系统.取环境 名="MARQDO_LLM_API_KEY"**
|
||||
## 用量
|
||||
+ `数据`
|
||||
|
||||
1. 非 `api_key`
|
||||
> 打印 内容=ext/ai/大模型: 请设置 OPENAI_API_KEY 或 MARQDO_LLM_API_KEY
|
||||
*> openai.usage_from data=`数据`*
|
||||
|
||||
## 创建
|
||||
+ `模型`=None
|
||||
+ `基址`=None
|
||||
+ `密钥`=None
|
||||
+ `后端`=openai-compatible
|
||||
+ `名`=None
|
||||
|
||||
**h = > 大模型 模型=`模型` 基址=`基址` 密钥=`密钥` 后端=`后端` 名=`名`**
|
||||
*h*
|
||||
|
||||
## 快速
|
||||
+ `模型`=None
|
||||
+ `基址`=None
|
||||
+ `密钥`=None
|
||||
|
||||
1. `模型`
|
||||
**mdl = 模型**
|
||||
2. *
|
||||
**mdl = > 系统.取环境 名="MARQDO_LLM_FAST_MODEL"**
|
||||
|
||||
*> 创建 模型=`mdl` 基址=`基址` 密钥=`密钥` 名="fast" 后端="openai-compatible"*
|
||||
|
||||
## 推理
|
||||
+ `模型`=None
|
||||
+ `基址`=None
|
||||
+ `密钥`=None
|
||||
|
||||
1. `模型`
|
||||
**mdl = 模型**
|
||||
2. *
|
||||
**mdl = > 系统.取环境 名="MARQDO_LLM_REASONING_MODEL"**
|
||||
|
||||
*> 创建 模型=`mdl` 基址=`基址` 密钥=`密钥` 名="reasoning" 后端="openai-compatible"*
|
||||
|
||||
## 提示正文
|
||||
+ `源`
|
||||
|
||||
**sw = > 文本.开头是 内容=`源` 前缀="---"**
|
||||
1. 非 `sw`
|
||||
*源*
|
||||
2. *
|
||||
**parts = > split value=`源` sep="\n---\n"**
|
||||
**n = > len value=`parts`**
|
||||
1. `n` < 2
|
||||
*源*
|
||||
2. *
|
||||
**fm = > at value=`parts` index=0**
|
||||
**is_p = > 文本.包含 内容=`fm` 子串="type: prompt"**
|
||||
1. 非 `is_p`
|
||||
**is_p = > 文本.包含 内容=`fm` 子串="类型: 提示"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. `is_p`
|
||||
**body = > at value=`parts` index=1**
|
||||
*> 文本.去空白 s=`body`*
|
||||
2. *
|
||||
*源*
|
||||
|
||||
## 加载提示
|
||||
+ `路径`
|
||||
|
||||
**raw = > 文件.读文本 路径=`路径`**
|
||||
*> 提示正文 源=`raw`*
|
||||
|
||||
## 提示
|
||||
+ `路径`=None
|
||||
+ `文本`=None
|
||||
|
||||
1. `路径`
|
||||
*> 加载提示 路径=`路径`*
|
||||
2. `文本`
|
||||
*> 提示正文 源=`文本`*
|
||||
3. *
|
||||
> 打印 内容=ext/ai/大模型.提示: 请传 路径= 或 文本=
|
||||
> 系统.退出 码=1
|
||||
|
||||
**base_url = > 系统.取环境 名="OPENAI_BASE_URL"**
|
||||
1. 非 `base_url`
|
||||
**base_url = > 系统.取环境 名="MARQDO_LLM_BASE_URL"**
|
||||
1. 非 `base_url`
|
||||
**base_url = "https://api.openai.com/v1"**
|
||||
## 提问
|
||||
+ `提示`=None
|
||||
+ `路径`=None
|
||||
+ `模型`=None
|
||||
+ `基址`=None
|
||||
+ `密钥`=None
|
||||
|
||||
**model = > 系统.取环境 名="OPENAI_MODEL"**
|
||||
1. 非 `model`
|
||||
**model = > 系统.取环境 名="MARQDO_LLM_MODEL"**
|
||||
1. 非 `model`
|
||||
**model = "gpt-4o-mini"**
|
||||
1. `路径`
|
||||
**p = > 加载提示 路径=`路径`**
|
||||
2. `提示`
|
||||
**p = 提示**
|
||||
3. *
|
||||
> 打印 内容=ext/ai/大模型.提问: 请传 提示= 或 路径=
|
||||
> 系统.退出 码=1
|
||||
|
||||
**m = > 创建 模型=`模型` 基址=`基址` 密钥=`密钥`**
|
||||
**r = > m.提问 提示=`p`**
|
||||
*[text](r)*
|
||||
|
||||
# 大模型
|
||||
+ `模型`=None
|
||||
+ `基址`=None
|
||||
+ `密钥`=None
|
||||
+ `后端`=openai-compatible
|
||||
+ `名`=None
|
||||
|
||||
1. `密钥`
|
||||
**key = 密钥**
|
||||
2. *
|
||||
**key = > 系统.取环境 名="OPENAI_API_KEY"**
|
||||
1. 非 `key`
|
||||
**key = > 系统.取环境 名="MARQDO_LLM_API_KEY"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `后端` == ollama
|
||||
1. 非 `key`
|
||||
**key = "ollama"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
2. 非 `key`
|
||||
> 打印 内容=ext/ai/大模型: 请设置 OPENAI_API_KEY 或 MARQDO_LLM_API_KEY
|
||||
> 系统.退出 码=1
|
||||
3. *
|
||||
**_ = 1**
|
||||
|
||||
1. `基址`
|
||||
**url = 基址**
|
||||
2. `后端` == ollama
|
||||
**url = > 系统.取环境 名="OLLAMA_HOST"**
|
||||
1. 非 `url`
|
||||
**url = "http://127.0.0.1:11434/v1"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
3. *
|
||||
**url = > 系统.取环境 名="OPENAI_BASE_URL"**
|
||||
1. 非 `url`
|
||||
**url = > 系统.取环境 名="MARQDO_LLM_BASE_URL"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. 非 `url`
|
||||
**url = "https://api.openai.com/v1"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `模型`
|
||||
**mdl = 模型**
|
||||
2. *
|
||||
**mdl = > 系统.取环境 名="OPENAI_MODEL"**
|
||||
1. 非 `mdl`
|
||||
**mdl = > 系统.取环境 名="MARQDO_LLM_MODEL"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. 非 `mdl`
|
||||
1. `后端` == ollama
|
||||
**mdl = "llama3.2"**
|
||||
2. *
|
||||
**mdl = "gpt-4o-mini"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. `名`
|
||||
**nm = 名**
|
||||
2. *
|
||||
**nm = None**
|
||||
|
||||
`h` =
|
||||
|
||||
| api_key | base_url | model | suffix | bearer |
|
||||
|---------|----------|-------|--------|--------|
|
||||
| `api_key` | `base_url` | `model` | /chat/completions | "Bearer " |
|
||||
| _type | backend | api_key | base_url | model | suffix | bearer | name |
|
||||
|-------|---------|---------|----------|-------|--------|--------|------|
|
||||
| 大模型 | `后端` | `key` | `url` | `mdl` | /chat/completions | "Bearer " | `nm` |
|
||||
|
||||
*h*
|
||||
|
||||
## 提问
|
||||
+ `提示`=None
|
||||
+ `路径`=None
|
||||
|
||||
1. `路径`
|
||||
**p = > 加载提示 路径=`路径`**
|
||||
2. `提示`
|
||||
**p = 提示**
|
||||
3. *
|
||||
> 打印 内容=ext/ai/大模型: 提问需要 提示= 或 路径=
|
||||
> 系统.退出 码=1
|
||||
|
||||
**pack = > 自.完整结果 提示=`p` 流式=False 打印增量=False**
|
||||
**text = > json.get value=`pack` key="text"**
|
||||
**usage = > json.get value=`pack` key="usage"**
|
||||
**finish = > json.get value=`pack` key="finish"**
|
||||
**tc = > json.get value=`pack` key="tool_calls"**
|
||||
1. not `tc`
|
||||
**tc = > json.parse text=[]**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
表单元格会遮蔽 `自` / 列名,先取出字段再组结果表。
|
||||
|
||||
**mdl = [model](自)**
|
||||
**b = [backend](自)**
|
||||
**nm = [name](自)**
|
||||
**tt = [total_tokens](usage)**
|
||||
|
||||
`result` =
|
||||
|
||||
| text | model | finish | backend | usage | name | llm_calls | tokens | tool_calls |
|
||||
|------|-------|--------|---------|-------|------|-----------|--------|------------|
|
||||
| `text` | `mdl` | `finish` | `b` | `usage` | `nm` | 1 | `tt` | `tc` |
|
||||
|
||||
*result*
|
||||
|
||||
## 流式
|
||||
+ `提示`=None
|
||||
+ `路径`=None
|
||||
+ `打印增量`=False
|
||||
|
||||
1. `路径`
|
||||
**p = > 加载提示 路径=`路径`**
|
||||
2. `提示`
|
||||
**p = 提示**
|
||||
3. *
|
||||
> 打印 内容=ext/ai/大模型: 流式需要 提示= 或 路径=
|
||||
> 系统.退出 码=1
|
||||
|
||||
*> 自.运行 提示=`p` 流式=True 打印增量=`打印增量`*
|
||||
|
||||
## 完整结果
|
||||
+ `提示`
|
||||
+ `流式`=False
|
||||
+ `打印增量`=False
|
||||
|
||||
*> openai.chat_completions base_url=[base_url](自) api_key=[api_key](自) model=[model](自) prompt=`提示` stream=`流式` echo=`打印增量` suffix=[suffix](自) bearer=[bearer](自)*
|
||||
|
||||
## 运行
|
||||
+ `提示`
|
||||
+ `流式`=False
|
||||
+ `打印增量`=False
|
||||
|
||||
使用 `自` 句柄字段做聊天补全。
|
||||
|
||||
线上 body 为 `{model, messages:[{role,content}], stream?}`。消息用 `@` 行向表;headers 与可选 `stream` 用 `表.改`。`json` 只做 stringify/parse。
|
||||
|
||||
**url = [base_url](自) + [suffix](自)**
|
||||
**auth = [bearer](自) + [api_key](自)**
|
||||
**headers = > 表.改 入=None 于="Authorization" 值=`auth`**
|
||||
|
||||
`messages` =
|
||||
|
||||
| @ | role | content |
|
||||
|---|------|---------|
|
||||
| 1 | user | `提示` |
|
||||
|
||||
`req` =
|
||||
|
||||
| model | messages |
|
||||
|-------|----------|
|
||||
| [model](自) | `messages` |
|
||||
|
||||
1. `流式`
|
||||
**req = > 表.改 入=`req` 于="stream" 值=True**
|
||||
**body = > json.stringify value=`req`**
|
||||
**resp = > 网络.提交流式 地址=`url` 内容=`body` headers=`headers` 打印增量=`打印增量`**
|
||||
1. [status](resp) == 200
|
||||
*[events](resp)*
|
||||
2. *
|
||||
> 打印 内容=ext/ai/大模型: HTTP 错误(流式)
|
||||
> 打印 内容=[status](resp)
|
||||
> 系统.退出 码=1
|
||||
**pack = > 自.完整结果 提示=`提示` 流式=True 打印增量=`打印增量`**
|
||||
*[events](pack)*
|
||||
2. *
|
||||
**body = > json.stringify value=`req`**
|
||||
**resp = > 网络.提交 地址=`url` 内容=`body` headers=`headers`**
|
||||
1. [status](resp) == 200
|
||||
**data = > json.parse text=[body](resp)**
|
||||
*[content]([message]([1]([choices](data))))*
|
||||
2. *
|
||||
> 打印 内容=ext/ai/大模型: HTTP 错误
|
||||
> 打印 内容=[status](resp)
|
||||
> 打印 内容=[body](resp)
|
||||
> 系统.退出 码=1
|
||||
|
||||
---
|
||||
**pack = > 自.完整结果 提示=`提示` 流式=False 打印增量=`打印增量`**
|
||||
*[text](pack)*
|
||||
|
||||
## 聊天
|
||||
+ `提示`
|
||||
+ `流式`=False
|
||||
+ `打印增量`=False
|
||||
|
||||
运行 的别名。
|
||||
|
||||
*> 自.运行 提示=`提示` 流式=`流式` 打印增量=`打印增量`*
|
||||
+123
-3
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: ext/ai/智能体
|
||||
description: 文档驱动智能体 — 单步 / 多步;工具经 lib/子任务;运行时经 ABI v2 agent 插件。
|
||||
description: 文档驱动智能体 v2 — 单步 / 多步 / 运行;技能编译经 ABI v2 agent 插件(情景 → 学习 → llm_free)。
|
||||
导入 大模型:ext/ai/大模型.mq.md
|
||||
import agent:ext/ai/agent.mq.md
|
||||
import json:lib/json.mq.md
|
||||
@@ -183,6 +183,15 @@ import json:lib/json.mq.md
|
||||
|
||||
*out*
|
||||
|
||||
## 工程预检
|
||||
+ `任务`
|
||||
+ `根目录`="."
|
||||
+ `知识目录`=None
|
||||
|
||||
Engineering First Information 门闩。转发 `agent.preflight`。
|
||||
|
||||
*> agent.preflight task=`任务` root=`根目录` marqdo_dir=`知识目录`*
|
||||
|
||||
## 多步
|
||||
+ `目标`
|
||||
+ `工作簿目录`=None
|
||||
@@ -204,11 +213,122 @@ import json:lib/json.mq.md
|
||||
+ `流式`=False
|
||||
+ `打印增量`=False
|
||||
+ `轨迹`=False
|
||||
+ `工程预检`=False
|
||||
+ `工程知识目录`=.marqdo
|
||||
|
||||
多步工作簿默认写入 OKF `知识库目录/resources/<slug>.mq.md`;仅当传入 `工作簿目录` 时才用临时 runs。转发至英文 `agent.plan`。`流式` / `打印增量` / `轨迹` 对应 `stream` / `echo` / `trace`;`软命中` → `soft_match`;`近义命中` / `近义阈值` → `near_match` / `near_threshold`。
|
||||
多步工作簿默认写入 OKF `知识库目录/resources/<slug>.mq.md`;仅当传入 `工作簿目录` 时才用临时 runs。转发至英文 `agent.plan`。`流式` / `打印增量` / `轨迹` 对应 `stream` / `echo` / `trace`;`软命中` → `soft_match`;`近义命中` / `近义阈值` → `near_match` / `near_threshold`。`工程预检` → `eng_preflight`。
|
||||
|
||||
**model = > json.get value=`self` key="model"**
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
**standing = > json.get value=`self` key="standing"**
|
||||
**en = > agent.agent model=`model` tools=`tools` standing=`standing`**
|
||||
*> en.plan goal=`目标` workbook_dir=`工作簿目录` workbook=`工作簿` max_rounds=`最大轮次` confirm=`确认分解` writeback=`写回` skeleton=`骨架` reuse=`复用` optimize=`优化` force=`强制` soft_match=`软命中` near_match=`近义命中` near_threshold=`近义阈值` promote=`晋升` kb_dir=`知识库目录` improve_every=`改进周期` explore_n=`探索次数` stream=`流式` echo=`打印增量` trace=`轨迹`*
|
||||
*> en.plan goal=`目标` workbook_dir=`工作簿目录` workbook=`工作簿` max_rounds=`最大轮次` confirm=`确认分解` writeback=`写回` skeleton=`骨架` reuse=`复用` optimize=`优化` force=`强制` soft_match=`软命中` near_match=`近义命中` near_threshold=`近义阈值` promote=`晋升` kb_dir=`知识库目录` improve_every=`改进周期` explore_n=`探索次数` stream=`流式` echo=`打印增量` trace=`轨迹` eng_preflight=`工程预检` marqdo_dir=`工程知识目录`*
|
||||
|
||||
## 解析
|
||||
+ `任务`
|
||||
+ `知识库目录`=.marqdo/agent-kb
|
||||
+ `近义命中`=True
|
||||
+ `近义阈值`=0.78
|
||||
|
||||
技能解析(v2):exact → alias → canonical → near。
|
||||
|
||||
**model = > json.get value=`self` key="model"**
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
**standing = > json.get value=`self` key="standing"**
|
||||
**en = > agent.agent model=`model` tools=`tools` standing=`standing`**
|
||||
*> en.resolve task=`任务` kb_dir=`知识库目录` near_match=`近义命中` near_threshold=`近义阈值`*
|
||||
|
||||
## 记录情景
|
||||
+ `任务`
|
||||
+ `状态`=ok
|
||||
+ `结果`=None
|
||||
+ `模式`=explore
|
||||
+ `技能`=none
|
||||
+ `观察`=None
|
||||
+ `工作簿`=None
|
||||
+ `缓存`=None
|
||||
+ `匹配`=None
|
||||
+ `大模型次数`=0
|
||||
+ `工具次数`=0
|
||||
+ `错误`=None
|
||||
+ `记忆目录`=.marqdo/agent-memory
|
||||
|
||||
将一次执行沉淀为 `.mq.md` 情景记忆。
|
||||
|
||||
**model = > json.get value=`self` key="model"**
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
**standing = > json.get value=`self` key="standing"**
|
||||
**en = > agent.agent model=`model` tools=`tools` standing=`standing`**
|
||||
*> en.record_episode task=`任务` status=`状态` result=`结果` mode=`模式` skill=`技能` observation=`观察` workbook=`工作簿` cache=`缓存` match=`匹配` llm_calls=`大模型次数` tool_calls=`工具次数` error=`错误` memory_dir=`记忆目录`*
|
||||
|
||||
## 尝试学习
|
||||
+ `任务`
|
||||
+ `知识库目录`=.marqdo/agent-kb
|
||||
+ `记忆目录`=.marqdo/agent-memory
|
||||
+ `改进周期`=3
|
||||
+ `晋升`=True
|
||||
|
||||
情景足够时:归纳 → 合成候选 → 编译 → 可选晋升为 `llm_free` 技能。
|
||||
|
||||
**model = > json.get value=`self` key="model"**
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
**standing = > json.get value=`self` key="standing"**
|
||||
**en = > agent.agent model=`model` tools=`tools` standing=`standing`**
|
||||
*> en.maybe_learn task=`任务` kb_dir=`知识库目录` memory_dir=`记忆目录` improve_every=`改进周期` promote=`晋升`*
|
||||
|
||||
## 运行
|
||||
+ `任务`
|
||||
+ `知识库目录`=.marqdo/agent-kb
|
||||
+ `记忆目录`=.marqdo/agent-memory
|
||||
+ `学习`=True
|
||||
+ `校验`=True
|
||||
+ `写回`=True
|
||||
+ `复用`=True
|
||||
+ `近义命中`=True
|
||||
+ `近义阈值`=0.78
|
||||
+ `晋升`=True
|
||||
+ `改进周期`=3
|
||||
+ `探索次数`=3
|
||||
+ `最大轮次`=4
|
||||
+ `流式`=False
|
||||
+ `打印增量`=False
|
||||
+ `轨迹`=False
|
||||
+ `强制`=False
|
||||
+ `路由`=auto
|
||||
+ `路由模型`=None
|
||||
|
||||
v2+P4 主入口:自适应路由 → 执行 → 记录情景 → 尝试学习(技能 + 策略)。`路由`=`auto`|`marqdo`|`small-llm`|`llm`(`jev` 为 `small-llm` 兼容别名);决策器用 `路由模型`(任意 llm 句柄)。
|
||||
|
||||
**model = > json.get value=`self` key="model"**
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
**standing = > json.get value=`self` key="standing"**
|
||||
**en = > agent.agent model=`model` tools=`tools` standing=`standing`**
|
||||
*> en.run task=`任务` kb_dir=`知识库目录` memory_dir=`记忆目录` learn=`学习` verify=`校验` writeback=`写回` reuse=`复用` near_match=`近义命中` near_threshold=`近义阈值` promote=`晋升` improve_every=`改进周期` explore_n=`探索次数` max_rounds=`最大轮次` stream=`流式` echo=`打印增量` trace=`轨迹` force=`强制` router=`路由` router_model=`路由模型`*
|
||||
|
||||
## 路由
|
||||
+ `任务`
|
||||
+ `后端`=auto
|
||||
+ `知识库目录`=.marqdo/agent-kb
|
||||
+ `记忆目录`=.marqdo/agent-memory
|
||||
+ `阈值`=0.78
|
||||
|
||||
自适应路由(阶段 4):`marqdo` / `small-llm` / `llm` / `auto`(`jev`→别名)。
|
||||
|
||||
**model = > json.get value=`self` key="model"**
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
**standing = > json.get value=`self` key="standing"**
|
||||
**en = > agent.agent model=`model` tools=`tools` standing=`standing`**
|
||||
*> en.route task=`任务` backend=`后端` kb_dir=`知识库目录` memory_dir=`记忆目录` threshold=`阈值`*
|
||||
|
||||
## 编译策略
|
||||
+ `记忆目录`=.marqdo/agent-memory
|
||||
+ `最少证据`=3
|
||||
+ `强制`=False
|
||||
|
||||
策略编译:稳定的 task→skill 映射 → `policies/router.mq.md`。
|
||||
|
||||
**model = > json.get value=`self` key="model"**
|
||||
**tools = > json.get value=`self` key="tools"**
|
||||
**standing = > json.get value=`self` key="standing"**
|
||||
**en = > agent.agent model=`model` tools=`tools` standing=`standing`**
|
||||
*> en.compile_policy memory_dir=`记忆目录` min_evidence=`最少证据` force=`强制`*
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
title: ext/data/cache
|
||||
description: Cache Resource (ADR 0007).
|
||||
import plugin:lib/plugin.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
import table:lib/table.mq.md
|
||||
---
|
||||
|
||||
## _load
|
||||
|
||||
**p = > plugin.native_path name="web"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
2. *
|
||||
> print text=ext/data: native web plugin not found
|
||||
> sys.exit code=1
|
||||
****
|
||||
|
||||
# cache
|
||||
+ `url`="memory:"
|
||||
|
||||
Key–value cache. Use `memory:` for in-process (tests / single process) or `redis://host:6379/0` for Redis.
|
||||
|
||||
> _load
|
||||
*> web_cache_new url=`url`*
|
||||
|
||||
## get
|
||||
+ `key`
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_cache_get url=`url` key=`key`*
|
||||
|
||||
## set
|
||||
+ `key`
|
||||
+ `value`
|
||||
+ `ttl`=None
|
||||
|
||||
Optional `ttl` is seconds until expiry.
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_cache_set url=`url` key=`key` value=`value` ttl=`ttl`*
|
||||
|
||||
## del
|
||||
+ `key`
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_cache_del url=`url` key=`key`*
|
||||
|
||||
## exists
|
||||
+ `key`
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_cache_exists url=`url` key=`key`*
|
||||
|
||||
## ttl
|
||||
+ `key`
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_cache_ttl url=`url` key=`key`*
|
||||
|
||||
@@ -0,0 +1,238 @@
|
||||
---
|
||||
title: ext/data/db
|
||||
description: Database Resource — CRUD, txn, migrate, FTS (ADR 0007).
|
||||
import plugin:lib/plugin.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
import table:lib/table.mq.md
|
||||
---
|
||||
|
||||
## _load
|
||||
|
||||
**p = > plugin.native_path name="web"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
2. *
|
||||
> print text=ext/data: native web plugin not found
|
||||
> sys.exit code=1
|
||||
****
|
||||
|
||||
# db
|
||||
+ `url`="sqlite:site.db"
|
||||
|
||||
Open a database handle. URL schemes: `sqlite:path` (default), `postgres://…` / `postgresql://…` (same CRUD methods).
|
||||
|
||||
> _load
|
||||
*> web_db_new url=`url`*
|
||||
|
||||
## init
|
||||
+ `name`
|
||||
+ `fields`
|
||||
|
||||
Create a table from a schema table. Optional columns: `唯一`/`unique`, `索引`/`index` (creates UNIQUE / INDEX), `外键`/`fk`/`references` (e.g. `posts.id` or `posts(id)`). Columns named `created_at` / `updated_at` are filled automatically on insert/update when present.
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_db_init url=`url` name=`name` fields=`fields`*
|
||||
|
||||
## insert
|
||||
+ `table`
|
||||
+ `rows`
|
||||
+ `txn`=None
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_db_insert url=`url` table=`table` rows=`rows` txn=`txn`*
|
||||
|
||||
## select
|
||||
+ `table`
|
||||
+ `where`=None
|
||||
+ `limit`=200
|
||||
+ `order`=None
|
||||
+ `txn`=None
|
||||
|
||||
Simple filters: one-row map of column→value (AND `=`), or rows `|字段|操作|值|` (`=` `!=` `>` `>=` `<` `<=` `like` `in` `between` `is null`; add `|或|` = `是` to join a row with `OR`). `order` is a column name with optional `-` prefix for descending (`"created_at"`, `"-created_at"`), comma-separated for multiple keys. Pass `txn` to read inside an open transaction. For pages (with a total), use `paginate`.
|
||||
|
||||
**url = [url](self)**
|
||||
**r = > web_db_select url=`url` table=`table` where=`where` limit=`limit` order=`order` offset=None txn=`txn`**
|
||||
*[rows](r)*
|
||||
|
||||
## paginate
|
||||
+ `table`
|
||||
+ `where`=None
|
||||
+ `limit`=200
|
||||
+ `order`=None
|
||||
+ `跳过`=0
|
||||
+ `txn`=None
|
||||
|
||||
Like `select` but returns `{ rows, total }` — the total counts rows matching `where` regardless of `limit`/`跳过`, so you can render `上一页 / 下一页`. Set `跳过` to the number of rows to skip (e.g. page 2 with 10 per page ⇒ `跳过`=10).
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_db_select url=`url` table=`table` where=`where` limit=`limit` order=`order` offset=`跳过` txn=`txn`*
|
||||
|
||||
## get
|
||||
+ `table`
|
||||
+ `id`
|
||||
+ `txn`=None
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_db_get url=`url` table=`table` id=`id` txn=`txn`*
|
||||
|
||||
## update
|
||||
+ `table`
|
||||
+ `id`
|
||||
+ `row`
|
||||
+ `txn`=None
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_db_update url=`url` table=`table` id=`id` row=`row` txn=`txn`*
|
||||
|
||||
## delete
|
||||
+ `table`
|
||||
+ `id`
|
||||
+ `txn`=None
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_db_delete url=`url` table=`table` id=`id` txn=`txn`*
|
||||
|
||||
## exec
|
||||
+ `sql`
|
||||
+ `args`=None
|
||||
+ `txn`=None
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_db_exec url=`url` sql=`sql` args=`args` txn=`txn`*
|
||||
|
||||
## query
|
||||
+ `sql`
|
||||
+ `args`=None
|
||||
+ `txn`=None
|
||||
|
||||
Run bare SQL and return the result set — count / join / group / subqueries. Returns `{ rows, count }`.
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_db_query url=`url` sql=`sql` args=`args` txn=`txn`*
|
||||
|
||||
## count
|
||||
+ `table`
|
||||
+ `where`=None
|
||||
+ `txn`=None
|
||||
|
||||
Count rows matching a `where` filter (same syntax as `select`). Returns a number.
|
||||
|
||||
**url = [url](self)**
|
||||
**r = > web_db_count url=`url` table=`table` where=`where` txn=`txn`**
|
||||
*[count](r)*
|
||||
|
||||
## migrate
|
||||
+ `steps`
|
||||
|
||||
Apply versioned SQL migrations. `steps` is a `|version|sql|` / `|版本|SQL|` table. Applied versions are recorded in `_marqdo_migrations`. Re-running is a no-op for already-applied versions. SQLite only.
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_db_migrate url=`url` steps=`steps`*
|
||||
|
||||
## fts
|
||||
+ `table`
|
||||
+ `columns`
|
||||
+ `name`=None
|
||||
|
||||
Create an FTS5 index on `table` for the listed content columns (CSV string or list). Default FTS name is `{table}_fts`. Keeps the index in sync via triggers. Requires integer `id` PK. SQLite only.
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_db_fts_create url=`url` table=`table` columns=`columns` name=`name`*
|
||||
|
||||
## search
|
||||
+ `table`
|
||||
+ `q`
|
||||
+ `limit`=20
|
||||
+ `name`=None
|
||||
|
||||
Full-text search (`MATCH`) against the FTS5 index from `fts`. Returns `{ rows, count }` with a `rank` column (bm25). SQLite only.
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_db_search url=`url` table=`table` q=`q` limit=`limit` name=`name`*
|
||||
|
||||
## 事务
|
||||
|
||||
Begin a transaction: borrows the pooled connection exclusively and returns a
|
||||
`txn` handle. Write inside it, then `提交` (commit) or `回滚` (roll back).
|
||||
Every statement runs on the same connection, so a batch is atomic.
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_db_begin url=`url`*
|
||||
|
||||
# txn
|
||||
+ `txn`
|
||||
+ `url`
|
||||
|
||||
A transaction handle from `db.事务`. All CRUD here runs on the transaction's
|
||||
connection; finish with `提交` or `回滚`.
|
||||
|
||||
*self*
|
||||
|
||||
## insert
|
||||
+ `table`
|
||||
+ `rows`
|
||||
|
||||
**url = [url](self)**
|
||||
**txn = [txn](self)**
|
||||
*> web_db_insert url=`url` table=`table` rows=`rows` txn=`txn`*
|
||||
|
||||
## select
|
||||
+ `table`
|
||||
+ `where`=None
|
||||
+ `limit`=200
|
||||
+ `order`=None
|
||||
|
||||
Same filters as `db.select`; runs inside the transaction.
|
||||
|
||||
**url = [url](self)**
|
||||
**txn = [txn](self)**
|
||||
**r = > web_db_select url=`url` table=`table` where=`where` limit=`limit` order=`order` offset=None txn=`txn`**
|
||||
*[rows](r)*
|
||||
|
||||
## get
|
||||
+ `table`
|
||||
+ `id`
|
||||
|
||||
**url = [url](self)**
|
||||
**txn = [txn](self)**
|
||||
*> web_db_get url=`url` table=`table` id=`id` txn=`txn`*
|
||||
|
||||
## update
|
||||
+ `table`
|
||||
+ `id`
|
||||
+ `row`
|
||||
|
||||
**url = [url](self)**
|
||||
**txn = [txn](self)**
|
||||
*> web_db_update url=`url` table=`table` id=`id` row=`row` txn=`txn`*
|
||||
|
||||
## delete
|
||||
+ `table`
|
||||
+ `id`
|
||||
|
||||
**url = [url](self)**
|
||||
**txn = [txn](self)**
|
||||
*> web_db_delete url=`url` table=`table` id=`id` txn=`txn`*
|
||||
|
||||
## exec
|
||||
+ `sql`
|
||||
+ `args`=None
|
||||
|
||||
**url = [url](self)**
|
||||
**txn = [txn](self)**
|
||||
*> web_db_exec url=`url` sql=`sql` args=`args` txn=`txn`*
|
||||
|
||||
## 提交
|
||||
|
||||
Commit the transaction and return its connection to the pool.
|
||||
|
||||
**txn = [txn](self)**
|
||||
*> web_db_commit txn=`txn`*
|
||||
|
||||
## 回滚
|
||||
|
||||
Roll the transaction back (undo every write) and return its connection.
|
||||
|
||||
**txn = [txn](self)**
|
||||
*> web_db_rollback txn=`txn`*
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
---
|
||||
title: ext/data/form
|
||||
description: Form Resource — fields, validate, submit (ADR 0007).
|
||||
import plugin:lib/plugin.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
import table:lib/table.mq.md
|
||||
---
|
||||
|
||||
## _load
|
||||
|
||||
**p = > plugin.native_path name="web"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
2. *
|
||||
> print text=ext/data: native web plugin not found
|
||||
> sys.exit code=1
|
||||
****
|
||||
|
||||
# form
|
||||
+ `table`=None
|
||||
+ `action`=insert
|
||||
+ `id`=None
|
||||
|
||||
Field table + rules table; submit writes through `# db`.
|
||||
|
||||
> _load
|
||||
*> web_form_new table=`table` action=`action` id=`id`*
|
||||
|
||||
## fields
|
||||
+ `fields`
|
||||
|
||||
*> web_form_fields form=`self` fields=`fields`*
|
||||
|
||||
## rules
|
||||
+ `rules`
|
||||
|
||||
*> web_form_rules form=`self` rules=`rules`*
|
||||
|
||||
## labels
|
||||
+ `submit`="Submit"
|
||||
+ `cancel`="cancel"
|
||||
+ `cancel_href`=""
|
||||
|
||||
Localized submit / cancel copy for rendered forms.
|
||||
|
||||
*> web_form_labels form=`self` submit=`submit` cancel=`cancel` cancel_href=`cancel_href`*
|
||||
|
||||
## validate
|
||||
+ `rules`=None
|
||||
+ `data`
|
||||
|
||||
*> web_form_validate form=`self` rules=`rules` data=`data`*
|
||||
|
||||
## render
|
||||
+ `id`=form
|
||||
+ `data`=None
|
||||
+ `errors`=None
|
||||
|
||||
*> web_form_render form=`self` id=`id` data=`data` errors=`errors`*
|
||||
|
||||
## submit
|
||||
+ `data`
|
||||
+ `db`
|
||||
|
||||
**url = [url](db)**
|
||||
*> web_form_submit form=`self` data=`data` url=`url`*
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
---
|
||||
title: ext/data/storage
|
||||
description: Object storage Resource (ADR 0007).
|
||||
import plugin:lib/plugin.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
import table:lib/table.mq.md
|
||||
---
|
||||
|
||||
## _load
|
||||
|
||||
**p = > plugin.native_path name="web"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
2. *
|
||||
> print text=ext/data: native web plugin not found
|
||||
> sys.exit code=1
|
||||
****
|
||||
|
||||
# storage
|
||||
+ `url`="file:data/blobs"
|
||||
|
||||
Object storage. `file:dir` stores blobs on disk (offline / gold). `s3://bucket?endpoint=http://127.0.0.1:9000&access_key=…&secret_key=…` talks to MinIO / S3.
|
||||
|
||||
> _load
|
||||
*> web_storage_new url=`url`*
|
||||
|
||||
## put
|
||||
+ `key`
|
||||
+ `body`=None
|
||||
+ `path`=None
|
||||
+ `content_type`="application/octet-stream"
|
||||
|
||||
Provide either `body` (text) or `path` (local file to upload).
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_storage_put url=`url` key=`key` body=`body` path=`path` content_type=`content_type`*
|
||||
|
||||
## get
|
||||
+ `key`
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_storage_get url=`url` key=`key`*
|
||||
|
||||
## delete
|
||||
+ `key`
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_storage_delete url=`url` key=`key`*
|
||||
|
||||
## list
|
||||
+ `prefix`=""
|
||||
|
||||
**url = [url](self)**
|
||||
*> web_storage_list url=`url` prefix=`prefix`*
|
||||
|
||||
# media
|
||||
+ `storage`=None
|
||||
|
||||
Offline helpers for upload validation and saving into `# storage` (also used by HTTP `app.upload`).
|
||||
|
||||
> _load
|
||||
*> web_media_new storage=`storage`*
|
||||
|
||||
## validate
|
||||
+ `filename`
|
||||
+ `content_type`="application/octet-stream"
|
||||
+ `size`
|
||||
+ `max_bytes`=5242880
|
||||
+ `types`=None
|
||||
|
||||
*> web_upload_validate filename=`filename` content_type=`content_type` size=`size` max_bytes=`max_bytes` types=`types`*
|
||||
|
||||
## save
|
||||
+ `path`
|
||||
+ `key`=None
|
||||
+ `content_type`="application/octet-stream"
|
||||
+ `prefix`=uploads/
|
||||
+ `storage`=None
|
||||
|
||||
**st = storage**
|
||||
1. `st` == None
|
||||
**st = [storage](self)**
|
||||
2. *
|
||||
|
||||
*> web_upload_save storage=`st` path=`path` key=`key` content_type=`content_type` prefix=`prefix`*
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
---
|
||||
title: ext/net/http
|
||||
description: HTTP primitives surface for Web Runtime (ADR 0007). Prefer Document/Endpoint Artifacts over raw calls.
|
||||
import plugin:lib/plugin.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
---
|
||||
|
||||
## _load
|
||||
|
||||
**p = > plugin.native_path name="web"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
2. *
|
||||
> print text=ext/net: native web plugin not found
|
||||
> sys.exit code=1
|
||||
****
|
||||
|
||||
## result
|
||||
+ `ok`
|
||||
+ `value`=None
|
||||
+ `error`=None
|
||||
|
||||
Unified Result map `{ok, value, error}`.
|
||||
|
||||
`out` =
|
||||
|
||||
| ok | value | error |
|
||||
|----|-------|-------|
|
||||
| `ok` | `value` | `error` |
|
||||
|
||||
*out*
|
||||
@@ -0,0 +1,9 @@
|
||||
---
|
||||
title: ext/net/url
|
||||
description: URL helpers (stdlib host wrappers). ADR 0007.
|
||||
---
|
||||
|
||||
## encode
|
||||
+ `text`
|
||||
|
||||
*> host_url_encode text=`text`*
|
||||
@@ -0,0 +1,41 @@
|
||||
---
|
||||
title: ext/net/websocket
|
||||
description: WebSocket client Resource (ADR 0007).
|
||||
import plugin:lib/plugin.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
import table:lib/table.mq.md
|
||||
---
|
||||
|
||||
## _load
|
||||
|
||||
**p = > plugin.native_path name="web"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
2. *
|
||||
> print text=ext/net: native web plugin not found
|
||||
> sys.exit code=1
|
||||
****
|
||||
|
||||
# ws
|
||||
+ `timeout_sec`=30
|
||||
|
||||
WebSocket client helper.
|
||||
|
||||
> _load
|
||||
`out` =
|
||||
|
||||
| timeout_sec | _type |
|
||||
|-------------|-------|
|
||||
| `timeout_sec` | ws |
|
||||
|
||||
*out*
|
||||
|
||||
## connect
|
||||
+ `url`
|
||||
+ `message`=""
|
||||
+ `headers`=None
|
||||
|
||||
Single request–response: connect to `url`, send `message`, collect up to 256 server text replies (each up to 1 MiB), close. Transport errors and timeouts return `{ok:false, error}` instead of looking like an empty successful reply.
|
||||
|
||||
**timeout = [timeout_sec](self)**
|
||||
*> web_ws_connect url=`url` message=`message` headers=`headers` timeout_sec=`timeout`*
|
||||
@@ -0,0 +1,58 @@
|
||||
---
|
||||
title: ext/security/auth
|
||||
description: Auth Resource (ADR 0007).
|
||||
import plugin:lib/plugin.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
import table:lib/table.mq.md
|
||||
---
|
||||
|
||||
## _load
|
||||
|
||||
**p = > plugin.native_path name="web"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
2. *
|
||||
> print text=ext/security: native web plugin not found
|
||||
> sys.exit code=1
|
||||
****
|
||||
|
||||
# auth
|
||||
+ `users`
|
||||
+ `session_ttl`=3600
|
||||
|
||||
Session/auth helper. Constructs a config object; `login` validates against the users table. To gate `/admin` on this app, use `app.auth users=…` instead.
|
||||
|
||||
> _load
|
||||
*> web_auth_new users=`users` session_ttl=`session_ttl`*
|
||||
|
||||
## login
|
||||
+ `username`
|
||||
+ `password`
|
||||
|
||||
Validate credentials against the users table and create a session. Returns `{ok, session_id, username, role}`.
|
||||
|
||||
**users = [users](self)**
|
||||
**ttl = [session_ttl](self)**
|
||||
*> web_auth_login username=`username` password=`password` users=`users` session_ttl=`ttl`*
|
||||
|
||||
## check
|
||||
+ `session_id`
|
||||
|
||||
Returns `{ok, username, role}` when the session is valid.
|
||||
|
||||
*> web_auth_check session_id=`session_id`*
|
||||
|
||||
## logout
|
||||
+ `session_id`
|
||||
|
||||
Destroy the session.
|
||||
|
||||
*> web_auth_logout session_id=`session_id`*
|
||||
|
||||
## hash_password
|
||||
+ `password`
|
||||
|
||||
Hash a plaintext password for storage in admin user tables (argon2id). Store the returned `hash` in the `password` column; login verifies automatically.
|
||||
|
||||
*> web_password_hash password=`password`*
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
---
|
||||
title: ext/security/oidc
|
||||
description: OIDC attach helper for app handles (ADR 0007). Prefer declared auth metadata on Endpoints.
|
||||
import plugin:lib/plugin.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
---
|
||||
|
||||
## _load
|
||||
|
||||
**p = > plugin.native_path name="web"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
2. *
|
||||
> print text=ext/security: native web plugin not found
|
||||
> sys.exit code=1
|
||||
****
|
||||
|
||||
## attach
|
||||
+ `app`
|
||||
+ `issuer`=None
|
||||
+ `client_id`
|
||||
+ `client_secret`
|
||||
+ `redirect_uri`
|
||||
+ `redirect_origins`=None
|
||||
+ `scopes`="openid profile email"
|
||||
+ `callback_path`=None
|
||||
+ `authorize_url`=None
|
||||
+ `token_url`=None
|
||||
+ `userinfo_url`=None
|
||||
|
||||
`redirect_uri` may be an absolute URL (`https://qdqc.com/oidc/callback`) or a path
|
||||
(`/oidc/callback`). Path mode resolves the absolute callback from the login request
|
||||
Host (`X-Forwarded-Proto` / `X-Forwarded-Host` when behind a reverse proxy) and stores
|
||||
it in the session so token exchange uses the same string.
|
||||
|
||||
Optional `redirect_origins` is a comma-separated allowlist
|
||||
(`https://qdqc.com,https://www.qdqc.com`). When set, path-mode origins must match.
|
||||
The IdP must still register every absolute callback URL that will be used.
|
||||
|
||||
> _load
|
||||
*> web_app_oidc app=`app` issuer=`issuer` client_id=`client_id` client_secret=`client_secret` redirect_uri=`redirect_uri` redirect_origins=`redirect_origins` scopes=`scopes` callback_path=`callback_path` authorize_url=`authorize_url` token_url=`token_url` userinfo_url=`userinfo_url`*
|
||||
@@ -0,0 +1,61 @@
|
||||
---
|
||||
title: ext/security/rbac
|
||||
description: RBAC Resource (ADR 0007).
|
||||
import plugin:lib/plugin.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
import table:lib/table.mq.md
|
||||
---
|
||||
|
||||
## _load
|
||||
|
||||
**p = > plugin.native_path name="web"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
2. *
|
||||
> print text=ext/security: native web plugin not found
|
||||
> sys.exit code=1
|
||||
****
|
||||
|
||||
# rbac
|
||||
|
||||
Offline helpers for role ⊥ permission. Prefer `app.enable_rbac` + gates for HTTP; use these for scripts/tests.
|
||||
|
||||
## can
|
||||
+ `permissions`
|
||||
+ `needed`
|
||||
|
||||
Return `{ok, allowed}` — whether held permission CSV includes any of `needed` (CSV). Held `*` allows all.
|
||||
|
||||
> _load
|
||||
*> web_rbac_can permissions=`permissions` needed=`needed`*
|
||||
|
||||
## assign_role
|
||||
+ `url`
|
||||
+ `username`
|
||||
+ `role`
|
||||
|
||||
Bind `username` (in `web_users`) to role name.
|
||||
|
||||
> _load
|
||||
*> web_rbac_assign_role url=`url` username=`username` role=`role`*
|
||||
|
||||
## create_role
|
||||
+ `url`
|
||||
+ `name`
|
||||
|
||||
Create a non-system role; returns `{ok, id, name}`.
|
||||
|
||||
> _load
|
||||
*> web_rbac_create_role url=`url` name=`name`*
|
||||
|
||||
## set_role_permissions
|
||||
+ `url`
|
||||
+ `role`
|
||||
+ `permissions`
|
||||
+ `grantable`=""
|
||||
|
||||
Replace permissions on a non-system role (CSV). `grantable` restricts grants (anti-escalation); empty = full catalog.
|
||||
|
||||
> _load
|
||||
*> web_rbac_set_role_permissions url=`url` role=`role` permissions=`permissions` grantable=`grantable`*
|
||||
|
||||
+17
-28
@@ -1,31 +1,20 @@
|
||||
# ext/web
|
||||
# ext/web — Document / Endpoint / Resource
|
||||
|
||||
Official dynamic website extension: **GFM tables + class methods**.
|
||||
Design: [`doc/design/ext-web.md`](../../doc/design/ext-web.md).
|
||||
Official dynamic site extension (**ADR 0007**). Design: [`doc/design/ext-web-artifact.md`](../../doc/design/ext-web-artifact.md).
|
||||
|
||||
| Surface | Import | Classes |
|
||||
|---------|--------|---------|
|
||||
| English | `> ext/web/web.mq.md` | `page` · `db` · `app` · `form` · `style` |
|
||||
| Chinese | `> ext/web/网页.mq.md` | `页面` · `数据库` · `应用` · `表单` · `样式` |
|
||||
|
||||
```bash
|
||||
cargo build -p marqdo_plugin_web
|
||||
# optional install into ~/.marqdo/ext
|
||||
marqdo ext add web
|
||||
```
|
||||
|
||||
Canonical sample (design §4): [`examples/web-site/`](../../examples/web-site/).
|
||||
|
||||
```bash
|
||||
cargo run -- run examples/web-site/index.mq.md
|
||||
# http://127.0.0.1:18081/
|
||||
# /_form/article · /admin · /admin/articles/new
|
||||
```
|
||||
|
||||
Offline smokes: `tests/ext/web-smoke.mq.md`, `web-form-smoke.mq.md`, `web-admin-smoke.mq.md`.
|
||||
|
||||
| Feature | Notes |
|
||||
| Surface | Import |
|
||||
|---------|--------|
|
||||
| Form §5.5 | field/rules tables · `validate` · `submit` · `mount_form` → `GET\|POST /_form/{id}` |
|
||||
| Admin CRUD | `admin=True` · schema → same form path · new/edit/delete under `/admin/{table}` |
|
||||
| Route | `` `app`.route path=/about page=`about` `` → `GET /about` |
|
||||
| English | `import web:ext/web/web.mq.md` |
|
||||
| Chinese | `导入 网页:ext/web/网页.mq.md` |
|
||||
|
||||
**Facade:** `page` · `route` · `serve` · `render` · `inspect` · `use`
|
||||
|
||||
**Related modules:** `ext/data` (db/cache/storage) · `ext/security` (auth/oidc/rbac) · `ext/net` (http/websocket/url)
|
||||
|
||||
```bash
|
||||
bash scripts/build-web-plugin.sh
|
||||
marqdo ext add web # optional install
|
||||
marqdo run examples/marqdo-blog/serve.mq.md
|
||||
```
|
||||
|
||||
Hard rules: no author `ensure_plugin` / `compose_*` / `configure` junk drawer; View schema `|type|slot|value|attrs|style|` only; declared vs system routes via `inspect`.
|
||||
@@ -0,0 +1,18 @@
|
||||
---
|
||||
title: ext/web/_capability
|
||||
description: Internal — load native web plugin once. Authors must not call this; import web = capability (ADR 0007).
|
||||
import plugin:lib/plugin.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
---
|
||||
|
||||
## load
|
||||
|
||||
Load ABI v2 `web` plugin if present. Fail loud when missing.
|
||||
|
||||
**p = > plugin.native_path name="web"**
|
||||
1. `p`
|
||||
> plugin.load path=`p`
|
||||
2. *
|
||||
> print text=ext/web: native web plugin not found (build plugins/web or marqdo ext add web)
|
||||
> sys.exit code=1
|
||||
****
|
||||
@@ -0,0 +1,50 @@
|
||||
---
|
||||
title: ext/web/client
|
||||
description: Browser Runtime helpers (WASM bridge). Prefer lib/browser for effect tables (ADR 0007).
|
||||
import cap:ext/web/_capability.mq.md
|
||||
import table:lib/table.mq.md
|
||||
---
|
||||
|
||||
## embed
|
||||
+ `bridge`="/static/marqdo-bridge.js"
|
||||
+ `wasm`="/static/marqdo_wasm.wasm"
|
||||
+ `source`=""
|
||||
+ `boot`=True
|
||||
|
||||
HTML snippet that loads the official browser Marqdo bridge (route C/D). With `boot` and non-empty `source`, auto-mounts. No author JS.
|
||||
|
||||
> cap.load
|
||||
1. `boot`
|
||||
1. `source` != ""
|
||||
*"<script type=\"module\" src=\"" + bridge + "\" data-mq-wasm=\"" + wasm + "\" data-mq-source-url=\"" + source + "\"></script>"*
|
||||
2. *
|
||||
*"<script type=\"module\" src=\"" + bridge + "\"></script>"*
|
||||
2. *
|
||||
*"<script type=\"module\" src=\"" + bridge + "\" data-mq-no-boot=\"1\"></script>"*
|
||||
|
||||
## text_patch
|
||||
+ `sel`
|
||||
+ `text`
|
||||
|
||||
`set_text` effects map. Prefer `lib/browser` in client `.mq.md`.
|
||||
|
||||
**`m` = > table.put in=None at=sel value=text**
|
||||
*> table.put in=None at="set_text" value=m*
|
||||
|
||||
## dom_patch
|
||||
+ `sel`
|
||||
+ `html`=""
|
||||
+ `attrs`=None
|
||||
+ `remove`=False
|
||||
|
||||
**`spec` = > table.put in=None at="html" value=html**
|
||||
1. `attrs`
|
||||
**`spec` = > table.put in=`spec` at="attrs" value=attrs**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. `remove`
|
||||
**`spec` = > table.put in=`spec` at="remove" value=True**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**`m` = > table.put in=None at=sel value=spec**
|
||||
*> table.put in=None at="dom" value=m*
|
||||
@@ -0,0 +1,20 @@
|
||||
---
|
||||
title: ext/web/component
|
||||
description: Reusable Document fragments as View node tables (ADR 0007).
|
||||
import table:lib/table.mq.md
|
||||
---
|
||||
|
||||
## nodes
|
||||
+ `table`
|
||||
|
||||
Identity helper — a component is a View table `|type|slot|value|attrs|style|`.
|
||||
|
||||
*table*
|
||||
|
||||
## merge
|
||||
+ `a`
|
||||
+ `b`
|
||||
|
||||
Concatenate two View node lists (list-of-maps or column tables). Prefer authoring one table.
|
||||
|
||||
*> table.concat a=`a` b=`b`*
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
title: ext/web/dom
|
||||
description: DOM Artifact — Markdown/body to structure; raw HTML is escape hatch only (ADR 0007).
|
||||
import cap:ext/web/_capability.mq.md
|
||||
import table:lib/table.mq.md
|
||||
---
|
||||
|
||||
## from_markdown
|
||||
+ `text`
|
||||
|
||||
Turn Markdown text into a simple DOM map `{kind:"markdown", text}` for `page.render` / `web.render`.
|
||||
|
||||
> cap.load
|
||||
**`out` = > table.put in=None at="kind" value="markdown"**
|
||||
*> table.put in=`out` at="text" value=text*
|
||||
|
||||
## raw
|
||||
+ `html`
|
||||
|
||||
Escape hatch — wrap a raw HTML string. Prefer Markdown / View nodes.
|
||||
|
||||
> cap.load
|
||||
**`out` = > table.put in=None at="kind" value="raw"**
|
||||
*> table.put in=`out` at="html" value=html*
|
||||
|
||||
## render
|
||||
+ `dom`
|
||||
|
||||
Render a DOM Artifact (markdown or raw) to HTML via native plugin.
|
||||
|
||||
> cap.load
|
||||
**kind = [kind](dom)**
|
||||
1. `kind` == "raw"
|
||||
**html = [html](dom)**
|
||||
*html*
|
||||
2. *
|
||||
**text = [text](dom)**
|
||||
*> web_dom_markdown text=`text`*
|
||||
@@ -0,0 +1,54 @@
|
||||
---
|
||||
title: ext/web/page
|
||||
description: Document → View. Sole compose path is render(nodes=) with unified View schema (ADR 0007).
|
||||
import cap:ext/web/_capability.mq.md
|
||||
import table:lib/table.mq.md
|
||||
---
|
||||
|
||||
# page
|
||||
+ `title`=""
|
||||
+ `route`="/"
|
||||
+ `method`="GET"
|
||||
+ `body`=""
|
||||
+ `nodes`=None
|
||||
+ `data_source`=""
|
||||
+ `data_order`=""
|
||||
+ `data_where`=""
|
||||
|
||||
Web Document handle. Prefer constructing from Artifact Metadata via `web.page` facade.
|
||||
View tables use only `|type|slot|value|attrs|style|` (ZH `|类型|槽|值|属性|样式|`).
|
||||
|
||||
> cap.load
|
||||
`out` =
|
||||
|
||||
| _type | title | route | method | body | nodes | data_source | data_order | data_where |
|
||||
|-------|-------|-------|--------|------|-------|-------------|------------|------------|
|
||||
| page | `title` | `route` | `method` | `body` | `nodes` | `data_source` | `data_order` | `data_where` |
|
||||
|
||||
*out*
|
||||
|
||||
## render
|
||||
+ `nodes`=None
|
||||
+ `db`=None
|
||||
|
||||
Render Document to HTML. `nodes` is the unified View table; when omitted, uses `self.nodes` or Markdown `body`.
|
||||
|
||||
> cap.load
|
||||
**n = nodes**
|
||||
1. `n` == None
|
||||
**n = [nodes](self)**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. `n`
|
||||
*> web_render_nodes page=`self` nodes=`n` db=`db`*
|
||||
2. *
|
||||
**body = [body](self)**
|
||||
*> web_render_document page=`self` body=`body` db=`db`*
|
||||
|
||||
## css
|
||||
+ `css`
|
||||
|
||||
Escape hatch — append raw CSS (not a second primary language).
|
||||
|
||||
> cap.load
|
||||
*> web_page_css page=`self` css=`css`*
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
title: ext/web/route
|
||||
description: Declared Endpoint / Document route registration (ADR 0007).
|
||||
import cap:ext/web/_capability.mq.md
|
||||
import table:lib/table.mq.md
|
||||
---
|
||||
|
||||
# route
|
||||
+ `app`
|
||||
+ `path`
|
||||
+ `method`="GET"
|
||||
+ `page`=None
|
||||
+ `fn`=""
|
||||
+ `request`="json"
|
||||
+ `response`="json"
|
||||
+ `auth`=""
|
||||
|
||||
Declared route handle. Prefer `web.route` / Artifact `type: endpoint` discovery via `web.serve`.
|
||||
|
||||
> cap.load
|
||||
`out` =
|
||||
|
||||
| _type | app | path | method | page | fn | request | response | auth |
|
||||
|-------|-----|------|--------|------|----|---------|----------|------|
|
||||
| route | `app` | `path` | `method` | `page` | `fn` | `request` | `response` | `auth` |
|
||||
|
||||
*out*
|
||||
|
||||
## use
|
||||
+ `middleware`
|
||||
|
||||
Attach middleware table to this route (`|header|value|` or named capability rows).
|
||||
|
||||
> cap.load
|
||||
*> web_route_use route=`self` middleware=`middleware`*
|
||||
|
||||
## register
|
||||
+ `app`=None
|
||||
|
||||
Register this declared route on an app handle.
|
||||
|
||||
> cap.load
|
||||
**a = app**
|
||||
1. `a` == None
|
||||
**a = [app](self)**
|
||||
2. *
|
||||
**_ = 1**
|
||||
**page = [page](self)**
|
||||
**fn = [fn](self)**
|
||||
1. `page`
|
||||
**path = [path](self)**
|
||||
*> web_app_route app=`a` path=`path` page=`page`*
|
||||
2. `fn`
|
||||
**path = [path](self)**
|
||||
**method = [method](self)**
|
||||
**request = [request](self)**
|
||||
**response = [response](self)**
|
||||
*> web_app_invoke app=`a` path=`path` method=`method` fn=`fn` body=`request` return=`response`*
|
||||
3. *
|
||||
*> table.put in=None at="ok" value=False*
|
||||
+94
-1091
File diff suppressed because it is too large.
Load diff
+47
-1113
File diff suppressed because it is too large.
Load diff
@@ -39,6 +39,31 @@ Current working directory.
|
||||
|
||||
*> host_cwd*
|
||||
|
||||
## meta
|
||||
|
||||
Bound Artifact Metadata from the entry `.mq.md` (map). See `doc/design/binding.md`.
|
||||
|
||||
*> host_meta*
|
||||
|
||||
## meta_get
|
||||
+ `key`
|
||||
|
||||
Lookup one metadata key; missing → None.
|
||||
|
||||
*> host_meta_get key=`key`*
|
||||
|
||||
## artifact_kind
|
||||
|
||||
Classify entry Artifact Metadata: `web` / `endpoint` / `prompt` / `other`, or None if no `type`/`类型`. ADR 0007.
|
||||
|
||||
*> host_artifact_kind*
|
||||
|
||||
## module_source
|
||||
|
||||
Entry `.mq.md` source text (for document-as-prompt). Empty string if unset.
|
||||
|
||||
*> host_module_source*
|
||||
|
||||
## exit
|
||||
+ `code`
|
||||
|
||||
|
||||
@@ -39,6 +39,31 @@ description: 中文系统包装(加载环境 可选 路径=)
|
||||
|
||||
*> host_cwd*
|
||||
|
||||
## 元信息
|
||||
|
||||
入口 `.mq.md` 已绑定的 Artifact Metadata(映射)。见 `doc/design/binding.md`。
|
||||
|
||||
*> host_meta*
|
||||
|
||||
## 取元信息
|
||||
+ `键`
|
||||
|
||||
按键取一条元信息;缺失 → None。
|
||||
|
||||
*> host_meta_get key=`键`*
|
||||
|
||||
## 工件种类
|
||||
|
||||
入口 Artifact 分类:`web` / `endpoint` / `prompt` / `other`;无 `type`/`类型` → None。ADR 0007。
|
||||
|
||||
*> host_artifact_kind*
|
||||
|
||||
## 模块源码
|
||||
|
||||
入口 `.mq.md` 全文(供 document-as-prompt)。未设置时为空文本。
|
||||
|
||||
*> host_module_source*
|
||||
|
||||
## 退出
|
||||
+ `码`
|
||||
|
||||
|
||||
@@ -0,0 +1,497 @@
|
||||
//! Engineering Knowledge preflight / reuse (reads `.marqdo` projections).
|
||||
//! Does not call host_* — consumes compiled knowledge graph artifacts.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use serde_json::{json, Value};
|
||||
|
||||
fn arg_text<'a>(args: &'a Value, key: &str) -> Result<&'a str, String> {
|
||||
args.get(key)
|
||||
.and_then(|v| v.as_str())
|
||||
.ok_or_else(|| format!("missing string arg `{key}`"))
|
||||
}
|
||||
|
||||
fn arg_f64(args: &Value, key: &str, default: f64) -> f64 {
|
||||
match args.get(key) {
|
||||
Some(Value::Number(n)) => n.as_f64().unwrap_or(default),
|
||||
Some(Value::String(s)) => s.parse().unwrap_or(default),
|
||||
_ => default,
|
||||
}
|
||||
}
|
||||
|
||||
fn arg_usize(args: &Value, key: &str, default: usize) -> usize {
|
||||
match args.get(key) {
|
||||
Some(Value::Number(n)) => n.as_u64().unwrap_or(default as u64) as usize,
|
||||
Some(Value::String(s)) => s.parse().unwrap_or(default),
|
||||
_ => default,
|
||||
}
|
||||
}
|
||||
|
||||
fn char_bigram_tf(text: &str) -> HashMap<String, f64> {
|
||||
let chars: Vec<char> = text.chars().filter(|c| !c.is_whitespace()).collect();
|
||||
let mut m = HashMap::new();
|
||||
if chars.is_empty() {
|
||||
return m;
|
||||
}
|
||||
if chars.len() == 1 {
|
||||
m.insert(chars[0].to_string(), 1.0);
|
||||
return m;
|
||||
}
|
||||
for w in chars.windows(2) {
|
||||
let key: String = w.iter().collect();
|
||||
*m.entry(key).or_insert(0.0) += 1.0;
|
||||
}
|
||||
m
|
||||
}
|
||||
|
||||
fn cosine_tf(a: &HashMap<String, f64>, b: &HashMap<String, f64>) -> f64 {
|
||||
let mut na = 0.0;
|
||||
let mut nb = 0.0;
|
||||
for v in a.values() {
|
||||
na += v * v;
|
||||
}
|
||||
for v in b.values() {
|
||||
nb += v * v;
|
||||
}
|
||||
if na == 0.0 || nb == 0.0 {
|
||||
return 0.0;
|
||||
}
|
||||
let mut dot = 0.0;
|
||||
for (k, va) in a {
|
||||
if let Some(vb) = b.get(k) {
|
||||
dot += va * vb;
|
||||
}
|
||||
}
|
||||
dot / (na.sqrt() * nb.sqrt())
|
||||
}
|
||||
|
||||
|
||||
fn resolve_marqdo_root(start: &str) -> PathBuf {
|
||||
let mut cur = PathBuf::from(start);
|
||||
if cur.is_file() {
|
||||
if let Some(p) = cur.parent() {
|
||||
cur = p.to_path_buf();
|
||||
}
|
||||
}
|
||||
for _ in 0..8 {
|
||||
let cand = cur.join(".marqdo");
|
||||
if cand.join("engineering.yaml").exists() || cand.join("graph/graph.json").exists() {
|
||||
return cand;
|
||||
}
|
||||
if !cur.pop() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
PathBuf::from(start).join(".marqdo")
|
||||
}
|
||||
|
||||
#[derive(Clone)]
|
||||
struct CapHit {
|
||||
id: String,
|
||||
name: String,
|
||||
score: f64,
|
||||
implemented_by: Vec<String>,
|
||||
}
|
||||
|
||||
fn load_capabilities(marqdo: &Path) -> Vec<(String, String, Vec<String>, String)> {
|
||||
// returns (id, name, implemented_by, material)
|
||||
let mut out = Vec::new();
|
||||
let eng = marqdo.join("engineering.yaml");
|
||||
if let Ok(raw) = fs::read_to_string(&eng) {
|
||||
let mut cur_id = String::new();
|
||||
let mut cur_name = String::new();
|
||||
let mut impls = Vec::new();
|
||||
let mut in_caps = false;
|
||||
let mut in_impl = false;
|
||||
for line in raw.lines() {
|
||||
let t = line.trim_end();
|
||||
if t.trim() == "capabilities:" {
|
||||
in_caps = true;
|
||||
continue;
|
||||
}
|
||||
if in_caps && !line.starts_with(' ') && !line.starts_with('\t') && !t.trim().is_empty()
|
||||
{
|
||||
if t.trim() == "knowledge:" {
|
||||
if !cur_id.is_empty() {
|
||||
out.push((
|
||||
cur_id.clone(),
|
||||
cur_name.clone(),
|
||||
impls.clone(),
|
||||
format!("{cur_name} {cur_id}"),
|
||||
));
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
if !in_caps {
|
||||
continue;
|
||||
}
|
||||
let trimmed = t.trim();
|
||||
if let Some(rest) = trimmed.strip_prefix("- id:") {
|
||||
if !cur_id.is_empty() {
|
||||
out.push((
|
||||
cur_id.clone(),
|
||||
cur_name.clone(),
|
||||
impls.clone(),
|
||||
format!("{cur_name} {cur_id}"),
|
||||
));
|
||||
}
|
||||
cur_id = rest.trim().trim_matches('"').to_string();
|
||||
cur_name.clear();
|
||||
impls.clear();
|
||||
in_impl = false;
|
||||
} else if let Some(rest) = trimmed.strip_prefix("name:") {
|
||||
cur_name = rest.trim().trim_matches('"').to_string();
|
||||
} else if trimmed == "implemented_by:" {
|
||||
in_impl = true;
|
||||
} else if in_impl {
|
||||
if let Some(rest) = trimmed.strip_prefix("- ") {
|
||||
impls.push(rest.trim().trim_matches('"').to_string());
|
||||
} else if !trimmed.is_empty() && !line.starts_with(' ') {
|
||||
in_impl = false;
|
||||
}
|
||||
}
|
||||
}
|
||||
if !cur_id.is_empty() {
|
||||
out.push((
|
||||
cur_id.clone(),
|
||||
cur_name.clone(),
|
||||
impls,
|
||||
format!("{cur_name} {cur_id}"),
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
// Also scan catalog/capabilities/*.md
|
||||
let cap_dir = marqdo.join("catalog/capabilities");
|
||||
if let Ok(rd) = fs::read_dir(cap_dir) {
|
||||
for ent in rd.flatten() {
|
||||
let p = ent.path();
|
||||
if !p
|
||||
.file_name()
|
||||
.and_then(|s| s.to_str())
|
||||
.map(|s| s.ends_with(".md"))
|
||||
.unwrap_or(false)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
let Ok(src) = fs::read_to_string(&p) else {
|
||||
continue;
|
||||
};
|
||||
let id = fm_field(&src, "id").unwrap_or_else(|| {
|
||||
p.file_stem()
|
||||
.and_then(|s| s.to_str())
|
||||
.unwrap_or("cap")
|
||||
.to_string()
|
||||
});
|
||||
let name = fm_field(&src, "name").unwrap_or_else(|| id.clone());
|
||||
if out.iter().any(|(i, _, _, _)| i == &id) {
|
||||
continue;
|
||||
}
|
||||
out.push((id.clone(), name.clone(), Vec::new(), format!("{name} {id} {src}")));
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn fm_field(source: &str, key: &str) -> Option<String> {
|
||||
let prefix = format!("{key}:");
|
||||
for line in source.lines() {
|
||||
let t = line.trim();
|
||||
if t == "---" {
|
||||
continue;
|
||||
}
|
||||
if let Some(rest) = t.strip_prefix(&prefix) {
|
||||
return Some(rest.trim().trim_matches('"').to_string());
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
fn load_knowledge_titles(marqdo: &Path, kind_subdir: &str) -> Vec<(String, String)> {
|
||||
let dir = marqdo.join("knowledge").join(kind_subdir);
|
||||
let mut out = Vec::new();
|
||||
let Ok(rd) = fs::read_dir(dir) else {
|
||||
return out;
|
||||
};
|
||||
for ent in rd.flatten() {
|
||||
let p = ent.path();
|
||||
let Ok(src) = fs::read_to_string(&p) else {
|
||||
continue;
|
||||
};
|
||||
let id = fm_field(&src, "id").unwrap_or_else(|| {
|
||||
p.file_stem()
|
||||
.and_then(|s| s.to_str())
|
||||
.unwrap_or("k")
|
||||
.to_string()
|
||||
});
|
||||
let title = fm_field(&src, "title").unwrap_or_else(|| id.clone());
|
||||
out.push((id, title));
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn rank_caps(task: &str, caps: &[(String, String, Vec<String>, String)]) -> Vec<CapHit> {
|
||||
let qv = char_bigram_tf(task);
|
||||
let mut hits = Vec::new();
|
||||
for (id, name, impls, material) in caps {
|
||||
let mut score = cosine_tf(&qv, &char_bigram_tf(material));
|
||||
if name.to_ascii_lowercase().contains(&task.to_ascii_lowercase())
|
||||
|| task
|
||||
.to_ascii_lowercase()
|
||||
.split_whitespace()
|
||||
.any(|t| t.len() > 2 && name.to_ascii_lowercase().contains(t))
|
||||
{
|
||||
score = (score + 0.35).min(1.0);
|
||||
}
|
||||
if score > 0.12 {
|
||||
hits.push(CapHit {
|
||||
id: id.clone(),
|
||||
name: name.clone(),
|
||||
score,
|
||||
implemented_by: impls.clone(),
|
||||
});
|
||||
}
|
||||
}
|
||||
hits.sort_by(|a, b| {
|
||||
b.score
|
||||
.partial_cmp(&a.score)
|
||||
.unwrap_or(std::cmp::Ordering::Equal)
|
||||
});
|
||||
hits
|
||||
}
|
||||
|
||||
fn relevant_knowledge(task: &str, items: &[(String, String)], cap_names: &[String]) -> Vec<String> {
|
||||
let task_l = task.to_ascii_lowercase();
|
||||
items
|
||||
.iter()
|
||||
.filter(|(id, title)| {
|
||||
let m = format!("{id} {title}").to_ascii_lowercase();
|
||||
task_l.split_whitespace().any(|t| t.len() > 3 && m.contains(t))
|
||||
|| cap_names.iter().any(|c| m.contains(&c.to_ascii_lowercase()))
|
||||
})
|
||||
.map(|(id, title)| format!("{title} ({id})"))
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// `agent_eng_preflight` — Engineering First Information gate.
|
||||
pub fn eng_preflight(args: &Value) -> Result<Value, String> {
|
||||
let task = arg_text(args, "task").or_else(|_| arg_text(args, "goal"))?;
|
||||
let root = args
|
||||
.get("root")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or(".");
|
||||
let marqdo = args
|
||||
.get("marqdo_dir")
|
||||
.and_then(|v| v.as_str())
|
||||
.map(PathBuf::from)
|
||||
.unwrap_or_else(|| resolve_marqdo_root(root));
|
||||
|
||||
if !marqdo.join("engineering.yaml").exists() && !marqdo.join("graph/graph.json").exists() {
|
||||
return Ok(json!({
|
||||
"status": "missing_knowledge",
|
||||
"create_allowed": false,
|
||||
"decision": "CREATE",
|
||||
"error": "run `marqdo knowledge` first",
|
||||
"existing": [],
|
||||
"recommended": [],
|
||||
}));
|
||||
}
|
||||
|
||||
let reuse_threshold = arg_f64(args, "reuse_threshold", 0.72);
|
||||
let adapt_threshold = arg_f64(args, "adapt_threshold", 0.45);
|
||||
let allow_create_after = arg_usize(args, "allow_create_after", 2);
|
||||
let min_candidates = arg_usize(args, "min_candidates", 3);
|
||||
|
||||
let caps = load_capabilities(&marqdo);
|
||||
let ranked = rank_caps(task, &caps);
|
||||
let existing: Vec<String> = ranked.iter().take(8).map(|c| c.name.clone()).collect();
|
||||
let cap_names = existing.clone();
|
||||
|
||||
let constraints = relevant_knowledge(
|
||||
task,
|
||||
&load_knowledge_titles(&marqdo, "constraints"),
|
||||
&cap_names,
|
||||
);
|
||||
let decisions = relevant_knowledge(
|
||||
task,
|
||||
&load_knowledge_titles(&marqdo, "decisions"),
|
||||
&cap_names,
|
||||
);
|
||||
let failures = relevant_knowledge(
|
||||
task,
|
||||
&load_knowledge_titles(&marqdo, "failures"),
|
||||
&cap_names,
|
||||
);
|
||||
|
||||
let best = ranked.first();
|
||||
let best_score = best.map(|c| c.score).unwrap_or(0.0);
|
||||
let analyzed = ranked.len().max(1).min(min_candidates.max(allow_create_after));
|
||||
|
||||
let (decision, create_allowed, recommended, reason) = if best_score >= reuse_threshold {
|
||||
let b = best.unwrap();
|
||||
(
|
||||
"REUSE",
|
||||
false,
|
||||
vec![format!("REUSE {}", b.name)],
|
||||
vec![
|
||||
"score meets REUSE threshold".into(),
|
||||
"same capability semantics".into(),
|
||||
],
|
||||
)
|
||||
} else if best_score >= adapt_threshold {
|
||||
let b = best.unwrap();
|
||||
(
|
||||
"ADAPT",
|
||||
false,
|
||||
vec![format!("ADAPT {}", b.name)],
|
||||
vec!["near match — wrap/compose/extend".into()],
|
||||
)
|
||||
} else if analyzed >= allow_create_after {
|
||||
(
|
||||
"CREATE",
|
||||
true,
|
||||
vec!["CREATE new capability".into()],
|
||||
vec!["no existing capability satisfies constraints".into()],
|
||||
)
|
||||
} else {
|
||||
(
|
||||
"CREATE",
|
||||
false,
|
||||
vec![],
|
||||
vec![format!(
|
||||
"insufficient candidates ({analyzed} < {allow_create_after})"
|
||||
)],
|
||||
)
|
||||
};
|
||||
|
||||
// Write context pack
|
||||
let contexts = marqdo.join("agent/contexts");
|
||||
let _ = fs::create_dir_all(&contexts);
|
||||
let mut h: u64 = 5381;
|
||||
for b in task.as_bytes() {
|
||||
h = h.wrapping_mul(33).wrapping_add(u64::from(*b));
|
||||
}
|
||||
let tid = format!("{h:016x}")[..12].to_string();
|
||||
let ctx_path = contexts.join(format!("task-{tid}.mq.md"));
|
||||
let pack = format!(
|
||||
"---\ntype: Marqdo Context Pack\ntitle: Engineering Context\ndecision: {decision}\ncreate_allowed: {create_allowed}\n---\n\n# Engineering Context\n\n## Task\n\n{task}\n\n## Existing Capabilities\n\n{existing}\n\n## Recommended Reuse\n\n{recommended}\n",
|
||||
existing = existing
|
||||
.iter()
|
||||
.map(|e| format!("- {e}"))
|
||||
.collect::<Vec<_>>()
|
||||
.join("\n"),
|
||||
recommended = recommended
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(i, r)| format!("{}. {r}", i + 1))
|
||||
.collect::<Vec<_>>()
|
||||
.join("\n"),
|
||||
);
|
||||
let _ = fs::write(&ctx_path, pack);
|
||||
|
||||
let status = if create_allowed {
|
||||
"ready_create"
|
||||
} else if decision == "REUSE" || decision == "ADAPT" {
|
||||
"ready"
|
||||
} else if decision == "CREATE" && !create_allowed {
|
||||
"blocked"
|
||||
} else {
|
||||
"ready"
|
||||
};
|
||||
|
||||
Ok(json!({
|
||||
"status": status,
|
||||
"intent": { "capability": best.map(|c| c.name.clone()).unwrap_or_else(|| task.to_string()) },
|
||||
"existing": existing,
|
||||
"constraints": constraints,
|
||||
"decisions": decisions,
|
||||
"failures": failures,
|
||||
"recommended": recommended,
|
||||
"create_allowed": create_allowed,
|
||||
"decision": decision,
|
||||
"confidence": best_score,
|
||||
"reason": reason,
|
||||
"context_path": ctx_path.display().to_string(),
|
||||
"candidates": ranked.iter().take(8).map(|c| json!({
|
||||
"id": c.id,
|
||||
"name": c.name,
|
||||
"score": c.score,
|
||||
"implemented_by": c.implemented_by,
|
||||
})).collect::<Vec<_>>(),
|
||||
}))
|
||||
}
|
||||
|
||||
/// `agent_eng_reuse` — alias of preflight focused on decision payload.
|
||||
pub fn eng_reuse(args: &Value) -> Result<Value, String> {
|
||||
let mut pf = eng_preflight(args)?;
|
||||
if let Some(obj) = pf.as_object_mut() {
|
||||
obj.insert(
|
||||
"protocol".into(),
|
||||
json!("REUSE|ADAPT|CREATE"),
|
||||
);
|
||||
}
|
||||
Ok(pf)
|
||||
}
|
||||
|
||||
/// `agent_eng_record` — append reuse metrics under `.marqdo/agent/episodes`.
|
||||
pub fn eng_record(args: &Value) -> Result<Value, String> {
|
||||
let decision = arg_text(args, "decision")?;
|
||||
let root = args
|
||||
.get("root")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or(".");
|
||||
let marqdo = args
|
||||
.get("marqdo_dir")
|
||||
.and_then(|v| v.as_str())
|
||||
.map(PathBuf::from)
|
||||
.unwrap_or_else(|| resolve_marqdo_root(root));
|
||||
let duplicated = matches!(args.get("duplicated"), Some(Value::Bool(true)));
|
||||
|
||||
let path = marqdo.join("agent/episodes/reuse_metrics.json");
|
||||
if let Some(p) = path.parent() {
|
||||
let _ = fs::create_dir_all(p);
|
||||
}
|
||||
let mut m = if let Ok(raw) = fs::read_to_string(&path) {
|
||||
serde_json::from_str::<Value>(&raw).unwrap_or(json!({}))
|
||||
} else {
|
||||
json!({})
|
||||
};
|
||||
let get_u = |v: &Value, k: &str| v.get(k).and_then(|x| x.as_u64()).unwrap_or(0);
|
||||
let mut reused = get_u(&m, "reused");
|
||||
let mut adapted = get_u(&m, "adapted");
|
||||
let mut created = get_u(&m, "created");
|
||||
let mut dups = get_u(&m, "duplicated");
|
||||
match decision.to_ascii_uppercase().as_str() {
|
||||
"REUSE" => reused += 1,
|
||||
"ADAPT" => adapted += 1,
|
||||
"CREATE" => {
|
||||
created += 1;
|
||||
if duplicated {
|
||||
dups += 1;
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
m = json!({
|
||||
"generated_functions": get_u(&m, "generated_functions") + 1,
|
||||
"reused": reused,
|
||||
"adapted": adapted,
|
||||
"created": created,
|
||||
"duplicated": dups,
|
||||
});
|
||||
let gen = m["generated_functions"].as_u64().unwrap_or(1) as f64;
|
||||
m["reuse_ratio"] = json!(reused as f64 / gen);
|
||||
m["adapt_ratio"] = json!(adapted as f64 / gen);
|
||||
m["novel_ratio"] = json!(created as f64 / gen);
|
||||
m["duplication_rate"] = json!(if created == 0 {
|
||||
0.0
|
||||
} else {
|
||||
dups as f64 / created as f64
|
||||
});
|
||||
fs::write(&path, serde_json::to_string_pretty(&m).unwrap_or_default())
|
||||
.map_err(|e| e.to_string())?;
|
||||
Ok(m)
|
||||
}
|
||||
@@ -1269,7 +1269,7 @@ pub fn workbook_solidify(args: &Value) -> Result<Value, String> {
|
||||
};
|
||||
let lit = marqdo_quoted_text(&result);
|
||||
let body = format!(
|
||||
"---\ntitle: agent workbook\n---\n\n# main\n\n*`result` = {lit} *\n**`result`**\n"
|
||||
"---\ntitle: agent workbook\n---\n\n# main\n\n**`result` = {lit}**\n*`result`*\n"
|
||||
);
|
||||
fs::write(&p, body).map_err(|e| format!("solidify write: {e}"))?;
|
||||
Ok(json!({
|
||||
|
||||
+159
-1
@@ -1,8 +1,13 @@
|
||||
//! Agent plugin (C ABI v2): layout helpers + session bag + context via host_query + agent-kb.
|
||||
//! Agent plugin (C ABI v2): layout helpers + session bag + context via host_query + agent-kb + v2 memory + adaptive route.
|
||||
|
||||
mod corpus;
|
||||
mod eng;
|
||||
mod kb;
|
||||
mod mcp_client;
|
||||
mod mcp_server;
|
||||
mod memory;
|
||||
mod resume;
|
||||
mod route;
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::ffi::{CStr, CString};
|
||||
@@ -902,6 +907,33 @@ kb_ffi!(
|
||||
"agent_workbook_solidify"
|
||||
);
|
||||
kb_ffi!(agent_kb_task_files, kb::kb_task_files, "agent_kb_task_files");
|
||||
kb_ffi!(agent_memory_ensure, memory::memory_ensure, "agent_memory_ensure");
|
||||
kb_ffi!(agent_record_episode, memory::record_episode, "agent_record_episode");
|
||||
kb_ffi!(agent_list_episodes, memory::list_episodes, "agent_list_episodes");
|
||||
kb_ffi!(agent_get_episode, memory::get_episode, "agent_get_episode");
|
||||
kb_ffi!(agent_cluster_episodes, memory::cluster_episodes, "agent_cluster_episodes");
|
||||
kb_ffi!(agent_extract_pattern, memory::extract_pattern, "agent_extract_pattern");
|
||||
kb_ffi!(agent_generalize, memory::generalize, "agent_generalize");
|
||||
kb_ffi!(agent_synthesize_skill, memory::synthesize_skill, "agent_synthesize_skill");
|
||||
kb_ffi!(agent_compile_skill, memory::compile_skill, "agent_compile_skill");
|
||||
kb_ffi!(agent_maybe_learn, memory::maybe_learn, "agent_maybe_learn");
|
||||
kb_ffi!(agent_metrics, memory::metrics, "agent_metrics");
|
||||
kb_ffi!(agent_resolve, memory::resolve_task, "agent_resolve");
|
||||
kb_ffi!(agent_route, route::route, "agent_route");
|
||||
kb_ffi!(agent_route_apply, route::route_apply, "agent_route_apply");
|
||||
kb_ffi!(agent_compile_policy, route::compile_policy, "agent_compile_policy");
|
||||
kb_ffi!(
|
||||
agent_maybe_compile_policy,
|
||||
route::maybe_compile_policy,
|
||||
"agent_maybe_compile_policy"
|
||||
);
|
||||
kb_ffi!(agent_resume_save, resume::resume_save, "agent_resume_save");
|
||||
kb_ffi!(agent_resume_load, resume::resume_load, "agent_resume_load");
|
||||
kb_ffi!(agent_resume_clear, resume::resume_clear, "agent_resume_clear");
|
||||
kb_ffi!(agent_resume_list, resume::resume_list, "agent_resume_list");
|
||||
kb_ffi!(agent_eng_preflight, eng::eng_preflight, "agent_eng_preflight");
|
||||
kb_ffi!(agent_eng_reuse, eng::eng_reuse, "agent_eng_reuse");
|
||||
kb_ffi!(agent_eng_record, eng::eng_record, "agent_eng_record");
|
||||
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn marqdo_plugin_abi_version() -> u32 {
|
||||
@@ -1001,6 +1033,27 @@ pub unsafe extern "C" fn marqdo_plugin_init(host: *const MarqdoHostApi) -> c_int
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
if register(
|
||||
host,
|
||||
"agent_mcp_client",
|
||||
"action,command",
|
||||
mcp_client::agent_mcp_client,
|
||||
) != 0
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_resume_save", "id,goal,status,round", agent_resume_save) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_resume_load", "id,resume_dir", agent_resume_load) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_resume_clear", "id,resume_dir", agent_resume_clear) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_resume_list", "resume_dir", agent_resume_list) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_kb_add_alias", "kb_dir,slug,alias", agent_kb_add_alias) != 0 {
|
||||
return 1;
|
||||
}
|
||||
@@ -1022,6 +1075,111 @@ pub unsafe extern "C" fn marqdo_plugin_init(host: *const MarqdoHostApi) -> c_int
|
||||
if register(host, "agent_kb_task_files", "kb_dir,goal", agent_kb_task_files) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_memory_ensure", "memory_dir", agent_memory_ensure) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(
|
||||
host,
|
||||
"agent_record_episode",
|
||||
"task,status,result,mode",
|
||||
agent_record_episode,
|
||||
) != 0
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_list_episodes", "memory_dir,task", agent_list_episodes) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_get_episode", "id", agent_get_episode) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(
|
||||
host,
|
||||
"agent_cluster_episodes",
|
||||
"memory_dir,task,threshold",
|
||||
agent_cluster_episodes,
|
||||
) != 0
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_extract_pattern", "task", agent_extract_pattern) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_generalize", "task", agent_generalize) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_synthesize_skill", "task", agent_synthesize_skill) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(
|
||||
host,
|
||||
"agent_compile_skill",
|
||||
"task,kb_dir,min_evidence",
|
||||
agent_compile_skill,
|
||||
) != 0
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
if register(
|
||||
host,
|
||||
"agent_maybe_learn",
|
||||
"task,kb_dir,improve_every",
|
||||
agent_maybe_learn,
|
||||
) != 0
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_metrics", "memory_dir", agent_metrics) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_resolve", "task,kb_dir", agent_resolve) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_route", "task,backend,kb_dir,memory_dir", agent_route) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_route_apply", "task,reply,kb_dir", agent_route_apply) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(
|
||||
host,
|
||||
"agent_compile_policy",
|
||||
"memory_dir,min_evidence",
|
||||
agent_compile_policy,
|
||||
) != 0
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
if register(
|
||||
host,
|
||||
"agent_maybe_compile_policy",
|
||||
"memory_dir,min_evidence",
|
||||
agent_maybe_compile_policy,
|
||||
) != 0
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
if register(
|
||||
host,
|
||||
"agent_eng_preflight",
|
||||
"task",
|
||||
agent_eng_preflight,
|
||||
) != 0
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
if register(host, "agent_eng_reuse", "task", agent_eng_reuse) != 0 {
|
||||
return 1;
|
||||
}
|
||||
if register(
|
||||
host,
|
||||
"agent_eng_record",
|
||||
"decision",
|
||||
agent_eng_record,
|
||||
) != 0
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
0
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,327 @@
|
||||
//! MCP Client (stdio JSON-RPC) — spawn a server process, initialize, list/call tools.
|
||||
|
||||
use serde_json::{json, Value};
|
||||
use std::ffi::CStr;
|
||||
use std::io::{BufRead, BufReader, Write};
|
||||
use std::os::raw::{c_char, c_int};
|
||||
use std::process::{Child, Command, Stdio};
|
||||
use std::sync::Mutex;
|
||||
use std::time::Duration;
|
||||
|
||||
use crate::{parse_args, set_err, set_out};
|
||||
|
||||
struct Session {
|
||||
child: Child,
|
||||
stdin: std::process::ChildStdin,
|
||||
reader: BufReader<std::process::ChildStdout>,
|
||||
next_id: i64,
|
||||
server_name: String,
|
||||
}
|
||||
|
||||
static SESSIONS: Mutex<Vec<(String, Session)>> = Mutex::new(Vec::new());
|
||||
|
||||
fn opt_text<'a>(args: &'a Value, key: &str) -> Option<&'a str> {
|
||||
args.get(key).and_then(|v| v.as_str()).filter(|s| !s.is_empty())
|
||||
}
|
||||
|
||||
fn arg_text<'a>(args: &'a Value, key: &str) -> Result<&'a str, String> {
|
||||
opt_text(args, key).ok_or_else(|| format!("missing text `{key}`"))
|
||||
}
|
||||
|
||||
fn write_rpc(stdin: &mut impl Write, msg: &Value) -> Result<(), String> {
|
||||
let s = serde_json::to_string(msg).map_err(|e| e.to_string())?;
|
||||
writeln!(stdin, "{s}").map_err(|e| format!("mcp client write: {e}"))?;
|
||||
stdin.flush().map_err(|e| format!("mcp client flush: {e}"))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn read_rpc(reader: &mut impl BufRead) -> Result<Value, String> {
|
||||
let mut line = String::new();
|
||||
loop {
|
||||
line.clear();
|
||||
let n = reader
|
||||
.read_line(&mut line)
|
||||
.map_err(|e| format!("mcp client read: {e}"))?;
|
||||
if n == 0 {
|
||||
return Err("mcp client: server closed stdout".into());
|
||||
}
|
||||
let t = line.trim();
|
||||
if t.is_empty() {
|
||||
continue;
|
||||
}
|
||||
return serde_json::from_str(t).map_err(|e| format!("mcp client json: {e}"));
|
||||
}
|
||||
}
|
||||
|
||||
fn rpc_call(session: &mut Session, method: &str, params: Value) -> Result<Value, String> {
|
||||
let id = session.next_id;
|
||||
session.next_id += 1;
|
||||
let req = json!({
|
||||
"jsonrpc": "2.0",
|
||||
"id": id,
|
||||
"method": method,
|
||||
"params": params,
|
||||
});
|
||||
write_rpc(&mut session.stdin, &req)?;
|
||||
// Read until matching id (skip notifications).
|
||||
for _ in 0..64 {
|
||||
let resp = read_rpc(&mut session.reader)?;
|
||||
if resp.get("id") == Some(&json!(id)) {
|
||||
if let Some(err) = resp.get("error") {
|
||||
let msg = err
|
||||
.get("message")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("rpc error");
|
||||
return Err(msg.to_string());
|
||||
}
|
||||
return Ok(resp.get("result").cloned().unwrap_or(Value::Null));
|
||||
}
|
||||
}
|
||||
Err("mcp client: no matching response".into())
|
||||
}
|
||||
|
||||
fn spawn_session(command: &str, args: &[String], name: &str) -> Result<Session, String> {
|
||||
let mut cmd = Command::new(command);
|
||||
for a in args {
|
||||
cmd.arg(a);
|
||||
}
|
||||
cmd.stdin(Stdio::piped())
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::inherit());
|
||||
let mut child = cmd
|
||||
.spawn()
|
||||
.map_err(|e| format!("mcp client spawn `{command}`: {e}"))?;
|
||||
let stdin = child
|
||||
.stdin
|
||||
.take()
|
||||
.ok_or_else(|| "mcp client: no stdin".to_string())?;
|
||||
let stdout = child
|
||||
.stdout
|
||||
.take()
|
||||
.ok_or_else(|| "mcp client: no stdout".to_string())?;
|
||||
let mut session = Session {
|
||||
child,
|
||||
stdin,
|
||||
reader: BufReader::new(stdout),
|
||||
next_id: 1,
|
||||
server_name: name.to_string(),
|
||||
};
|
||||
// initialize
|
||||
let _ = rpc_call(
|
||||
&mut session,
|
||||
"initialize",
|
||||
json!({
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": {},
|
||||
"clientInfo": { "name": "marqdo-agent", "version": "0.1.0" }
|
||||
}),
|
||||
)?;
|
||||
// optional initialized notification
|
||||
let _ = write_rpc(
|
||||
&mut session.stdin,
|
||||
&json!({
|
||||
"jsonrpc": "2.0",
|
||||
"method": "notifications/initialized",
|
||||
"params": {}
|
||||
}),
|
||||
);
|
||||
let _ = session.server_name;
|
||||
// Give slow servers a beat (ignore).
|
||||
let _ = Duration::from_millis(1);
|
||||
Ok(session)
|
||||
}
|
||||
|
||||
fn command_args(args: &Value) -> Result<(String, Vec<String>), String> {
|
||||
let command = arg_text(args, "command")?;
|
||||
let mut argv = Vec::new();
|
||||
if let Some(arr) = args.get("args").and_then(|v| v.as_array()) {
|
||||
for a in arr {
|
||||
if let Some(s) = a.as_str() {
|
||||
argv.push(s.to_string());
|
||||
} else {
|
||||
argv.push(a.to_string());
|
||||
}
|
||||
}
|
||||
} else if let Some(s) = opt_text(args, "args") {
|
||||
// space-split simple args
|
||||
for part in s.split_whitespace() {
|
||||
argv.push(part.to_string());
|
||||
}
|
||||
}
|
||||
Ok((command.to_string(), argv))
|
||||
}
|
||||
|
||||
fn session_id_of(args: &Value) -> String {
|
||||
opt_text(args, "session")
|
||||
.or_else(|| opt_text(args, "id"))
|
||||
.unwrap_or("default")
|
||||
.to_string()
|
||||
}
|
||||
|
||||
fn with_session<F>(args: &Value, f: F) -> Result<Value, String>
|
||||
where
|
||||
F: FnOnce(&mut Session) -> Result<Value, String>,
|
||||
{
|
||||
let sid = session_id_of(args);
|
||||
let mut guard = SESSIONS
|
||||
.lock()
|
||||
.map_err(|_| "mcp client: session lock poisoned".to_string())?;
|
||||
let idx = guard
|
||||
.iter()
|
||||
.position(|(id, _)| id == &sid)
|
||||
.ok_or_else(|| format!("mcp client: no session `{sid}` (call connect first)"))?;
|
||||
f(&mut guard[idx].1)
|
||||
}
|
||||
|
||||
fn action_connect(args: &Value) -> Result<Value, String> {
|
||||
let (command, argv) = command_args(args)?;
|
||||
let name = opt_text(args, "name").unwrap_or("mcp");
|
||||
let sid = session_id_of(args);
|
||||
let session = spawn_session(&command, &argv, name)?;
|
||||
let mut guard = SESSIONS
|
||||
.lock()
|
||||
.map_err(|_| "mcp client: session lock poisoned".to_string())?;
|
||||
// Replace existing
|
||||
guard.retain(|(id, _)| id != &sid);
|
||||
guard.push((sid.clone(), session));
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"session": sid,
|
||||
"transport": "stdio",
|
||||
"command": command,
|
||||
}))
|
||||
}
|
||||
|
||||
fn action_list(args: &Value) -> Result<Value, String> {
|
||||
with_session(args, |s| {
|
||||
let result = rpc_call(s, "tools/list", json!({}))?;
|
||||
let mut out = result;
|
||||
if let Some(obj) = out.as_object_mut() {
|
||||
obj.insert("authority".into(), json!("workbook"));
|
||||
obj.insert("session".into(), json!(session_id_of(args)));
|
||||
}
|
||||
Ok(out)
|
||||
})
|
||||
}
|
||||
|
||||
fn action_call(args: &Value) -> Result<Value, String> {
|
||||
let name = arg_text(args, "name")?;
|
||||
let arguments = args.get("arguments").cloned().unwrap_or(json!({}));
|
||||
with_session(args, |s| {
|
||||
let result = rpc_call(
|
||||
s,
|
||||
"tools/call",
|
||||
json!({ "name": name, "arguments": arguments }),
|
||||
)?;
|
||||
let mut out = if result.is_object() {
|
||||
result
|
||||
} else {
|
||||
json!({ "result": result })
|
||||
};
|
||||
if let Some(obj) = out.as_object_mut() {
|
||||
obj.insert("authority".into(), json!("workbook"));
|
||||
obj.insert("session".into(), json!(session_id_of(args)));
|
||||
}
|
||||
Ok(out)
|
||||
})
|
||||
}
|
||||
|
||||
fn action_close(args: &Value) -> Result<Value, String> {
|
||||
let sid = session_id_of(args);
|
||||
let mut guard = SESSIONS
|
||||
.lock()
|
||||
.map_err(|_| "mcp client: session lock poisoned".to_string())?;
|
||||
if let Some(idx) = guard.iter().position(|(id, _)| id == &sid) {
|
||||
let (_, mut session) = guard.remove(idx);
|
||||
let _ = session.child.kill();
|
||||
let _ = session.child.wait();
|
||||
Ok(json!({ "ok": true, "session": sid, "closed": true }))
|
||||
} else {
|
||||
Ok(json!({ "ok": true, "session": sid, "closed": false }))
|
||||
}
|
||||
}
|
||||
|
||||
/// One-shot: connect → list or call → close (no persistent session).
|
||||
fn action_oneshot(args: &Value, kind: &str) -> Result<Value, String> {
|
||||
let (command, argv) = command_args(args)?;
|
||||
let name = opt_text(args, "name").unwrap_or("mcp");
|
||||
let mut session = spawn_session(&command, &argv, name)?;
|
||||
let result = match kind {
|
||||
"list" => {
|
||||
let r = rpc_call(&mut session, "tools/list", json!({}))?;
|
||||
Ok(r)
|
||||
}
|
||||
"call" => {
|
||||
let tname = arg_text(args, "tool").or_else(|_| arg_text(args, "name"))?;
|
||||
// When oneshot call, `name` may be server name — prefer `tool=`.
|
||||
let tname = opt_text(args, "tool").unwrap_or(tname);
|
||||
let arguments = args.get("arguments").cloned().unwrap_or(json!({}));
|
||||
rpc_call(
|
||||
&mut session,
|
||||
"tools/call",
|
||||
json!({ "name": tname, "arguments": arguments }),
|
||||
)
|
||||
}
|
||||
_ => Err(format!("unknown oneshot kind `{kind}`")),
|
||||
};
|
||||
let _ = session.child.kill();
|
||||
let _ = session.child.wait();
|
||||
let mut out = result?;
|
||||
if let Some(obj) = out.as_object_mut() {
|
||||
obj.insert("authority".into(), json!("workbook"));
|
||||
obj.insert("transport".into(), json!("stdio"));
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
pub fn mcp_client(args: &Value) -> Result<Value, String> {
|
||||
let action = opt_text(args, "action").unwrap_or("list");
|
||||
match action {
|
||||
"connect" => action_connect(args),
|
||||
"list" => {
|
||||
if opt_text(args, "command").is_some() && opt_text(args, "session").is_none() {
|
||||
action_oneshot(args, "list")
|
||||
} else {
|
||||
action_list(args)
|
||||
}
|
||||
}
|
||||
"call" => {
|
||||
if opt_text(args, "command").is_some() && opt_text(args, "session").is_none() {
|
||||
action_oneshot(args, "call")
|
||||
} else {
|
||||
action_call(args)
|
||||
}
|
||||
}
|
||||
"close" | "disconnect" => action_close(args),
|
||||
other => Err(format!(
|
||||
"mcp client: unknown action `{other}` (connect|list|call|close)"
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
pub unsafe extern "C" fn agent_mcp_client(
|
||||
args_json: *const c_char,
|
||||
out_json: *mut *mut c_char,
|
||||
err_msg: *mut *mut c_char,
|
||||
) -> c_int {
|
||||
let args = match parse_args(args_json) {
|
||||
Ok(v) => v,
|
||||
Err(e) => {
|
||||
set_err(err_msg, &e);
|
||||
return 1;
|
||||
}
|
||||
};
|
||||
match mcp_client(&args) {
|
||||
Ok(v) => {
|
||||
set_out(out_json, &v.to_string());
|
||||
0
|
||||
}
|
||||
Err(e) => {
|
||||
set_err(err_msg, &e);
|
||||
1
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
fn _unused(_: &CStr) {}
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,181 @@
|
||||
//! Agent plan/run resume checkpoints under `.marqdo/agent-resume/`.
|
||||
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use serde_json::{json, Value};
|
||||
|
||||
const DEFAULT_RESUME_DIR: &str = ".marqdo/agent-resume";
|
||||
|
||||
fn resolve_path(rel: &str) -> PathBuf {
|
||||
let p = PathBuf::from(rel);
|
||||
if p.is_absolute() {
|
||||
return p;
|
||||
}
|
||||
if p.exists() {
|
||||
return p;
|
||||
}
|
||||
let base = (|| {
|
||||
let q = crate::host_query_json("cwd")?;
|
||||
q.as_str()
|
||||
.map(PathBuf::from)
|
||||
.ok_or_else(|| "cwd not text".to_string())
|
||||
})()
|
||||
.or_else(|_| std::env::current_dir().map_err(|e| e.to_string()))
|
||||
.unwrap_or_else(|_| PathBuf::from("."));
|
||||
base.join(p)
|
||||
}
|
||||
|
||||
fn opt_text<'a>(args: &'a Value, key: &str) -> Option<&'a str> {
|
||||
args.get(key).and_then(|v| v.as_str()).filter(|s| !s.is_empty())
|
||||
}
|
||||
|
||||
fn arg_text<'a>(args: &'a Value, key: &str) -> Result<&'a str, String> {
|
||||
opt_text(args, key).ok_or_else(|| format!("missing text `{key}`"))
|
||||
}
|
||||
|
||||
fn opt_i64(args: &Value, key: &str, default: i64) -> i64 {
|
||||
match args.get(key) {
|
||||
None | Some(Value::Null) => default,
|
||||
Some(Value::Number(n)) => n.as_i64().unwrap_or(default),
|
||||
Some(Value::String(s)) => s.parse().unwrap_or(default),
|
||||
_ => default,
|
||||
}
|
||||
}
|
||||
|
||||
fn resume_root(args: &Value) -> PathBuf {
|
||||
let rel = opt_text(args, "resume_dir").unwrap_or(DEFAULT_RESUME_DIR);
|
||||
resolve_path(rel)
|
||||
}
|
||||
|
||||
fn checkpoint_path(root: &Path, id: &str) -> PathBuf {
|
||||
let safe: String = id
|
||||
.chars()
|
||||
.map(|c| {
|
||||
if c.is_ascii_alphanumeric() || c == '-' || c == '_' {
|
||||
c
|
||||
} else {
|
||||
'_'
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
root.join(format!("{safe}.json"))
|
||||
}
|
||||
|
||||
fn now_rfc3339() -> String {
|
||||
chrono::Utc::now().to_rfc3339_opts(chrono::SecondsFormat::Secs, true)
|
||||
}
|
||||
|
||||
fn ensure_dir(root: &Path) -> Result<(), String> {
|
||||
fs::create_dir_all(root).map_err(|e| format!("resume mkdir: {e}"))
|
||||
}
|
||||
|
||||
pub fn resume_save(args: &Value) -> Result<Value, String> {
|
||||
let id = arg_text(args, "id")?;
|
||||
let root = resume_root(args);
|
||||
ensure_dir(&root)?;
|
||||
let path = checkpoint_path(&root, id);
|
||||
let mut doc = json!({
|
||||
"id": id,
|
||||
"updated_at": now_rfc3339(),
|
||||
});
|
||||
if let Some(obj) = doc.as_object_mut() {
|
||||
for key in [
|
||||
"goal", "task", "status", "workbook", "events", "route", "result", "round", "meta",
|
||||
] {
|
||||
if let Some(v) = args.get(key) {
|
||||
if !v.is_null() {
|
||||
obj.insert(key.to_string(), v.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
if !obj.contains_key("round") {
|
||||
obj.insert("round".into(), json!(opt_i64(args, "round", 0)));
|
||||
}
|
||||
if !obj.contains_key("status") {
|
||||
obj.insert("status".into(), json!("running"));
|
||||
}
|
||||
}
|
||||
let text = serde_json::to_string_pretty(&doc).map_err(|e| e.to_string())?;
|
||||
fs::write(&path, text).map_err(|e| format!("resume save: {e}"))?;
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"id": id,
|
||||
"path": path.to_string_lossy(),
|
||||
"updated_at": doc.get("updated_at").cloned().unwrap_or(Value::Null),
|
||||
}))
|
||||
}
|
||||
|
||||
pub fn resume_load(args: &Value) -> Result<Value, String> {
|
||||
let id = arg_text(args, "id")?;
|
||||
let root = resume_root(args);
|
||||
let path = checkpoint_path(&root, id);
|
||||
if !path.is_file() {
|
||||
return Ok(json!({
|
||||
"ok": false,
|
||||
"found": false,
|
||||
"id": id,
|
||||
"path": path.to_string_lossy(),
|
||||
}));
|
||||
}
|
||||
let text = fs::read_to_string(&path).map_err(|e| format!("resume load: {e}"))?;
|
||||
let mut doc: Value = serde_json::from_str(&text).map_err(|e| format!("resume json: {e}"))?;
|
||||
if let Some(obj) = doc.as_object_mut() {
|
||||
obj.insert("ok".into(), json!(true));
|
||||
obj.insert("found".into(), json!(true));
|
||||
obj.insert("path".into(), json!(path.to_string_lossy()));
|
||||
}
|
||||
Ok(doc)
|
||||
}
|
||||
|
||||
pub fn resume_clear(args: &Value) -> Result<Value, String> {
|
||||
let id = arg_text(args, "id")?;
|
||||
let root = resume_root(args);
|
||||
let path = checkpoint_path(&root, id);
|
||||
let existed = path.is_file();
|
||||
if existed {
|
||||
fs::remove_file(&path).map_err(|e| format!("resume clear: {e}"))?;
|
||||
}
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"id": id,
|
||||
"cleared": existed,
|
||||
"path": path.to_string_lossy(),
|
||||
}))
|
||||
}
|
||||
|
||||
pub fn resume_list(args: &Value) -> Result<Value, String> {
|
||||
let root = resume_root(args);
|
||||
if !root.is_dir() {
|
||||
return Ok(json!({ "ok": true, "items": [], "resume_dir": root.to_string_lossy() }));
|
||||
}
|
||||
let mut items = Vec::new();
|
||||
let rd = fs::read_dir(&root).map_err(|e| format!("resume list: {e}"))?;
|
||||
for ent in rd.flatten() {
|
||||
let p = ent.path();
|
||||
if p.extension().and_then(|e| e.to_str()) != Some("json") {
|
||||
continue;
|
||||
}
|
||||
let id = p
|
||||
.file_stem()
|
||||
.and_then(|s| s.to_str())
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
let meta = fs::metadata(&p).ok();
|
||||
items.push(json!({
|
||||
"id": id,
|
||||
"path": p.to_string_lossy(),
|
||||
"len": meta.map(|m| m.len()).unwrap_or(0),
|
||||
}));
|
||||
}
|
||||
items.sort_by(|a, b| {
|
||||
let ai = a.get("id").and_then(|v| v.as_str()).unwrap_or("");
|
||||
let bi = b.get("id").and_then(|v| v.as_str()).unwrap_or("");
|
||||
ai.cmp(bi)
|
||||
});
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"items": items,
|
||||
"resume_dir": root.to_string_lossy(),
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,719 @@
|
||||
//! Adaptive Routing / Policy Compilation (Phase 4).
|
||||
//! Router is abstract: marqdo | small-llm | llm | auto.
|
||||
//! `jev` is accepted as an alias of `small-llm` (compat); not a first-class backend.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use serde_json::{json, Value};
|
||||
|
||||
use crate::kb::{goal_sig_hex, goal_slug_str, normalize_goal};
|
||||
use crate::memory::{self};
|
||||
|
||||
const DEFAULT_MEMORY_DIR: &str = ".marqdo/agent-memory";
|
||||
const DEFAULT_KB_DIR: &str = ".marqdo/agent-kb";
|
||||
const POLICY_NAME: &str = "router.mq.md";
|
||||
|
||||
fn arg_text<'a>(args: &'a Value, key: &str) -> Result<&'a str, String> {
|
||||
args.get(key)
|
||||
.and_then(|v| v.as_str())
|
||||
.ok_or_else(|| format!("missing text `{key}`"))
|
||||
}
|
||||
|
||||
fn opt_text<'a>(args: &'a Value, key: &str) -> Option<&'a str> {
|
||||
args.get(key).and_then(|v| v.as_str()).filter(|s| !s.is_empty())
|
||||
}
|
||||
|
||||
fn opt_i64(args: &Value, key: &str, default: i64) -> i64 {
|
||||
match args.get(key) {
|
||||
None | Some(Value::Null) => default,
|
||||
Some(Value::Number(n)) => n.as_i64().unwrap_or(default),
|
||||
Some(Value::String(s)) => s.parse().unwrap_or(default),
|
||||
_ => default,
|
||||
}
|
||||
}
|
||||
|
||||
fn opt_f64(args: &Value, key: &str, default: f64) -> f64 {
|
||||
match args.get(key) {
|
||||
None | Some(Value::Null) => default,
|
||||
Some(Value::Number(n)) => n.as_f64().unwrap_or(default),
|
||||
Some(Value::String(s)) => s.parse().unwrap_or(default),
|
||||
_ => default,
|
||||
}
|
||||
}
|
||||
|
||||
fn opt_bool(args: &Value, key: &str, default: bool) -> bool {
|
||||
match args.get(key) {
|
||||
None | Some(Value::Null) => default,
|
||||
Some(Value::Bool(b)) => *b,
|
||||
Some(Value::String(s)) => matches!(s.as_str(), "true" | "True" | "1" | "yes"),
|
||||
Some(Value::Number(n)) => n.as_i64().unwrap_or(0) != 0,
|
||||
_ => default,
|
||||
}
|
||||
}
|
||||
|
||||
fn now_rfc3339() -> String {
|
||||
chrono::Utc::now().to_rfc3339_opts(chrono::SecondsFormat::Secs, true)
|
||||
}
|
||||
|
||||
fn yaml_escape(s: &str) -> String {
|
||||
if s.is_empty()
|
||||
|| s.contains(':')
|
||||
|| s.contains('#')
|
||||
|| s.contains('|')
|
||||
|| s.contains('"')
|
||||
|| s.contains('\n')
|
||||
{
|
||||
format!("\"{}\"", s.replace('\\', "\\\\").replace('"', "\\\""))
|
||||
} else {
|
||||
s.to_string()
|
||||
}
|
||||
}
|
||||
|
||||
fn resolve_path(rel: &str) -> PathBuf {
|
||||
let p = PathBuf::from(rel);
|
||||
if p.is_absolute() {
|
||||
return p;
|
||||
}
|
||||
if p.exists() {
|
||||
return p;
|
||||
}
|
||||
let base = (|| {
|
||||
let q = crate::host_query_json("cwd")?;
|
||||
q.as_str()
|
||||
.map(PathBuf::from)
|
||||
.ok_or_else(|| "cwd not text".to_string())
|
||||
})()
|
||||
.or_else(|_| std::env::current_dir().map_err(|e| e.to_string()))
|
||||
.unwrap_or_else(|_| PathBuf::from("."));
|
||||
base.join(p)
|
||||
}
|
||||
|
||||
fn memory_root(args: &Value) -> PathBuf {
|
||||
let rel = opt_text(args, "memory_dir").unwrap_or(DEFAULT_MEMORY_DIR);
|
||||
resolve_path(rel)
|
||||
}
|
||||
|
||||
fn policy_path(root: &Path) -> PathBuf {
|
||||
root.join("policies").join(POLICY_NAME)
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
struct PolicyRule {
|
||||
task_hash: String,
|
||||
slug: String,
|
||||
skill: String,
|
||||
task: String,
|
||||
count: i64,
|
||||
confidence: f64,
|
||||
}
|
||||
|
||||
fn parse_policy(src: &str) -> Vec<PolicyRule> {
|
||||
let mut rules = Vec::new();
|
||||
let mut in_table = false;
|
||||
let mut header_ok = false;
|
||||
for line in src.lines() {
|
||||
let t = line.trim();
|
||||
if t.starts_with("|") && t.contains("task_hash") {
|
||||
in_table = true;
|
||||
header_ok = true;
|
||||
continue;
|
||||
}
|
||||
if in_table && t.starts_with("|") && t.contains("---") {
|
||||
continue;
|
||||
}
|
||||
if in_table && t.starts_with("|") {
|
||||
let cols: Vec<&str> = t
|
||||
.trim_matches('|')
|
||||
.split('|')
|
||||
.map(|c| c.trim().trim_matches('"'))
|
||||
.collect();
|
||||
if cols.len() >= 5 {
|
||||
let count = cols[4].parse().unwrap_or(1);
|
||||
let confidence = if cols.len() >= 6 {
|
||||
cols[5].parse().unwrap_or(0.9)
|
||||
} else {
|
||||
0.9
|
||||
};
|
||||
let task = if cols.len() >= 7 {
|
||||
cols[6].to_string()
|
||||
} else {
|
||||
String::new()
|
||||
};
|
||||
rules.push(PolicyRule {
|
||||
task_hash: cols[0].to_string(),
|
||||
slug: cols[1].to_string(),
|
||||
skill: cols[2].to_string(),
|
||||
count,
|
||||
confidence,
|
||||
task,
|
||||
});
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if in_table && header_ok && !t.starts_with('|') && !t.is_empty() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
rules
|
||||
}
|
||||
|
||||
fn write_policy(root: &Path, rules: &[PolicyRule]) -> Result<PathBuf, String> {
|
||||
fs::create_dir_all(root.join("policies")).map_err(|e| format!("mkdir policies: {e}"))?;
|
||||
let path = policy_path(root);
|
||||
let ts = now_rfc3339();
|
||||
let mut body = format!(
|
||||
"---\ntype: policy\nstatus: compiled\nbackend: marqdo\ngenerated_at: {ts}\nrules: {}\n---\n\n# Adaptive router (compiled Marqdo policy)\n\nDeterministic task → skill routing. No LLM required.\n\n# Rules\n\n| task_hash | slug | skill | resource | count | confidence | task |\n|-----------|------|-------|----------|------:|-----------:|------|\n",
|
||||
rules.len()
|
||||
);
|
||||
for r in rules {
|
||||
body.push_str(&format!(
|
||||
"| {} | {} | {} | {} | {} | {:.2} | {} |\n",
|
||||
r.task_hash,
|
||||
yaml_escape(&r.slug),
|
||||
yaml_escape(&r.skill),
|
||||
yaml_escape(&r.skill), // resource often == skill path for OKF
|
||||
r.count,
|
||||
r.confidence,
|
||||
yaml_escape(&r.task),
|
||||
));
|
||||
}
|
||||
body.push_str("\n# Fallback\n\nllm\n");
|
||||
fs::write(&path, body).map_err(|e| format!("write policy: {e}"))?;
|
||||
Ok(path)
|
||||
}
|
||||
|
||||
fn load_rules(root: &Path) -> Vec<PolicyRule> {
|
||||
let path = policy_path(root);
|
||||
let Ok(src) = fs::read_to_string(&path) else {
|
||||
return Vec::new();
|
||||
};
|
||||
parse_policy(&src)
|
||||
}
|
||||
|
||||
fn match_policy(rules: &[PolicyRule], task: &str, threshold: f64) -> Option<PolicyRule> {
|
||||
let th = goal_sig_hex(&normalize_goal(task));
|
||||
if let Some(r) = rules.iter().find(|r| r.task_hash == th) {
|
||||
return Some(r.clone());
|
||||
}
|
||||
// Near-match on stored task text / slug.
|
||||
let mut best: Option<(f64, PolicyRule)> = None;
|
||||
for r in rules {
|
||||
let score = memory::task_near_score(task, &r.task);
|
||||
if score >= threshold {
|
||||
if best.as_ref().map(|(s, _)| score > *s).unwrap_or(true) {
|
||||
best = Some((score, r.clone()));
|
||||
}
|
||||
}
|
||||
let slug_score = memory::task_near_score(task, &r.slug);
|
||||
if slug_score >= threshold {
|
||||
if best.as_ref().map(|(s, _)| slug_score > *s).unwrap_or(true) {
|
||||
let mut rr = r.clone();
|
||||
rr.confidence = slug_score;
|
||||
best = Some((slug_score, rr));
|
||||
}
|
||||
}
|
||||
}
|
||||
best.map(|(s, mut r)| {
|
||||
r.confidence = s;
|
||||
r
|
||||
})
|
||||
}
|
||||
|
||||
fn build_route_prompt(task: &str, candidates: &[Value]) -> String {
|
||||
let mut s = String::from(
|
||||
"You are a Marqdo Agent router. Pick the best skill slug for the task, or say EXPLORE.\n\nReply with exactly one line:\nSKILL:<slug>\nor\nEXPLORE\n\nTask:\n",
|
||||
);
|
||||
s.push_str(task);
|
||||
s.push_str("\n\nCandidates:\n");
|
||||
for (i, c) in candidates.iter().enumerate() {
|
||||
let slug = c
|
||||
.get("slug")
|
||||
.or_else(|| c.get("title"))
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("?");
|
||||
let title = c
|
||||
.get("title")
|
||||
.or_else(|| c.get("goal"))
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("");
|
||||
s.push_str(&format!("{}. {slug} — {title}\n", i + 1));
|
||||
}
|
||||
s
|
||||
}
|
||||
|
||||
fn candidate_list(args: &Value) -> Result<Vec<Value>, String> {
|
||||
if let Some(Value::Array(items)) = args.get("candidates") {
|
||||
return Ok(items.clone());
|
||||
}
|
||||
let kb = opt_text(args, "kb_dir").unwrap_or(DEFAULT_KB_DIR);
|
||||
let listed = crate::kb::kb_list_tasks(&json!({ "kb_dir": kb }))?;
|
||||
let tasks = listed
|
||||
.get("tasks")
|
||||
.and_then(|v| v.as_array())
|
||||
.cloned()
|
||||
.unwrap_or_default();
|
||||
if !tasks.is_empty() {
|
||||
return Ok(tasks);
|
||||
}
|
||||
let goal = arg_text(args, "task").or_else(|_| arg_text(args, "goal"))?;
|
||||
let near = crate::kb::kb_near_match(&json!({
|
||||
"kb_dir": kb,
|
||||
"goal": goal,
|
||||
}))?;
|
||||
Ok(near
|
||||
.get("candidates")
|
||||
.and_then(|v| v.as_array())
|
||||
.cloned()
|
||||
.unwrap_or_default())
|
||||
}
|
||||
|
||||
fn enrich_with_kb(rule: &PolicyRule, kb: &str, task: &str) -> Result<Value, String> {
|
||||
let hit = crate::kb::kb_lookup(&json!({
|
||||
"kb_dir": kb,
|
||||
"goal": task,
|
||||
"slug": rule.slug,
|
||||
"near_match": true,
|
||||
"near_threshold": 0.5,
|
||||
}))
|
||||
.unwrap_or(Value::Null);
|
||||
let resource = hit
|
||||
.get("resource")
|
||||
.cloned()
|
||||
.unwrap_or_else(|| Value::String(rule.skill.clone()));
|
||||
let skill = hit
|
||||
.get("skill")
|
||||
.cloned()
|
||||
.unwrap_or_else(|| Value::String(rule.skill.clone()));
|
||||
let llm_free = hit
|
||||
.get("llm_free")
|
||||
.and_then(|v| v.as_bool())
|
||||
.unwrap_or(true);
|
||||
Ok(json!({
|
||||
"backend": "marqdo",
|
||||
"mode": "policy",
|
||||
"matched": true,
|
||||
"needs_llm": false,
|
||||
"slug": rule.slug,
|
||||
"skill": skill,
|
||||
"resource": resource,
|
||||
"task_hash": rule.task_hash,
|
||||
"confidence": rule.confidence,
|
||||
"llm_free": llm_free,
|
||||
"count": rule.count,
|
||||
}))
|
||||
}
|
||||
|
||||
fn route_marqdo(args: &Value) -> Result<Value, String> {
|
||||
let root = memory_root(args);
|
||||
let _ = memory::memory_ensure(&json!({
|
||||
"memory_dir": opt_text(args, "memory_dir").unwrap_or(DEFAULT_MEMORY_DIR),
|
||||
}));
|
||||
let task = arg_text(args, "task").or_else(|_| arg_text(args, "goal"))?;
|
||||
let threshold = opt_f64(args, "threshold", 0.78);
|
||||
let kb = opt_text(args, "kb_dir").unwrap_or(DEFAULT_KB_DIR);
|
||||
let rules = load_rules(&root);
|
||||
if let Some(rule) = match_policy(&rules, task, threshold) {
|
||||
if rule.confidence >= threshold {
|
||||
return enrich_with_kb(&rule, kb, task);
|
||||
}
|
||||
}
|
||||
Ok(json!({
|
||||
"backend": "marqdo",
|
||||
"mode": "miss",
|
||||
"matched": false,
|
||||
"needs_llm": false,
|
||||
"confidence": 0.0,
|
||||
}))
|
||||
}
|
||||
|
||||
fn route_resolve_fallback(args: &Value) -> Result<Value, String> {
|
||||
let resolved = memory::resolve_task(args)?;
|
||||
let conf = resolved
|
||||
.get("confidence")
|
||||
.and_then(|v| v.as_f64())
|
||||
.unwrap_or(0.0);
|
||||
let llm_free = resolved
|
||||
.get("llm_free")
|
||||
.and_then(|v| v.as_bool())
|
||||
.unwrap_or(false);
|
||||
if conf >= opt_f64(args, "threshold", 0.78) || llm_free {
|
||||
return Ok(json!({
|
||||
"backend": "kb",
|
||||
"mode": resolved.get("mode").cloned().unwrap_or(Value::String("near".into())),
|
||||
"matched": true,
|
||||
"needs_llm": false,
|
||||
"slug": resolved.get("slug").cloned().unwrap_or(Value::Null),
|
||||
"skill": resolved.get("skill").cloned().unwrap_or(Value::Null),
|
||||
"resource": resolved.get("resource").cloned().unwrap_or(Value::Null),
|
||||
"confidence": conf,
|
||||
"llm_free": llm_free,
|
||||
}));
|
||||
}
|
||||
Ok(json!({
|
||||
"backend": "kb",
|
||||
"mode": "miss",
|
||||
"matched": false,
|
||||
"needs_llm": false,
|
||||
"confidence": conf,
|
||||
"resolve": resolved,
|
||||
}))
|
||||
}
|
||||
|
||||
fn route_model_request(args: &Value, level: &str) -> Result<Value, String> {
|
||||
let task = arg_text(args, "task").or_else(|_| arg_text(args, "goal"))?;
|
||||
let cands = candidate_list(args)?;
|
||||
let prompt = build_route_prompt(task, &cands);
|
||||
Ok(json!({
|
||||
"backend": level,
|
||||
"mode": "needs_llm",
|
||||
"matched": false,
|
||||
"needs_llm": true,
|
||||
"level": level,
|
||||
"prompt": prompt,
|
||||
"candidates": cands,
|
||||
"confidence": 0.0,
|
||||
}))
|
||||
}
|
||||
|
||||
/// Cascade Adaptive Router (Phase 4).
|
||||
///
|
||||
/// `backend`: `auto` | `marqdo` | `small-llm` | `llm` (`jev` → alias of `small-llm`).
|
||||
///
|
||||
/// `auto` order: marqdo policy → kb resolve → small-llm request.
|
||||
pub fn route(args: &Value) -> Result<Value, String> {
|
||||
let backend = opt_text(args, "backend").unwrap_or("auto");
|
||||
let threshold = opt_f64(args, "threshold", 0.78);
|
||||
|
||||
match backend {
|
||||
"marqdo" => {
|
||||
let hit = route_marqdo(args)?;
|
||||
if hit.get("matched").and_then(|v| v.as_bool()) == Some(true) {
|
||||
return Ok(hit);
|
||||
}
|
||||
route_resolve_fallback(args)
|
||||
}
|
||||
"small-llm" | "small_llm" | "small" | "jev" => route_model_request(args, "small-llm"),
|
||||
"llm" | "large" | "large-llm" => route_model_request(args, "llm"),
|
||||
"auto" | _ => {
|
||||
let mut trace: Vec<Value> = Vec::new();
|
||||
let m = route_marqdo(args)?;
|
||||
trace.push(json!({"try": "marqdo", "matched": m.get("matched")}));
|
||||
if m.get("matched").and_then(|v| v.as_bool()) == Some(true)
|
||||
&& m.get("confidence").and_then(|v| v.as_f64()).unwrap_or(0.0) >= threshold
|
||||
{
|
||||
let mut out = m;
|
||||
if let Some(obj) = out.as_object_mut() {
|
||||
obj.insert("trace".into(), Value::Array(trace));
|
||||
obj.insert("backend".into(), Value::String("auto/marqdo".into()));
|
||||
}
|
||||
return Ok(out);
|
||||
}
|
||||
let kb = route_resolve_fallback(args)?;
|
||||
trace.push(json!({"try": "kb", "matched": kb.get("matched")}));
|
||||
if kb.get("matched").and_then(|v| v.as_bool()) == Some(true) {
|
||||
let mut out = kb;
|
||||
if let Some(obj) = out.as_object_mut() {
|
||||
obj.insert("trace".into(), Value::Array(trace));
|
||||
obj.insert("backend".into(), Value::String("auto/kb".into()));
|
||||
}
|
||||
return Ok(out);
|
||||
}
|
||||
// Prefer small-llm request in auto when nothing matched (caller may use large).
|
||||
let mut req = route_model_request(args, "small-llm")?;
|
||||
if let Some(obj) = req.as_object_mut() {
|
||||
obj.insert("trace".into(), Value::Array(trace));
|
||||
obj.insert("backend".into(), Value::String("auto/small-llm".into()));
|
||||
}
|
||||
Ok(req)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Apply a router model reply (`SKILL:<slug>` / `EXPLORE`) to a concrete skill hit.
|
||||
pub fn route_apply(args: &Value) -> Result<Value, String> {
|
||||
let reply = arg_text(args, "reply")?;
|
||||
let kb = opt_text(args, "kb_dir").unwrap_or(DEFAULT_KB_DIR);
|
||||
let task = arg_text(args, "task").or_else(|_| arg_text(args, "goal"))?;
|
||||
let line = reply
|
||||
.lines()
|
||||
.map(|l| l.trim())
|
||||
.find(|l| !l.is_empty())
|
||||
.unwrap_or("");
|
||||
let upper = line.to_ascii_uppercase();
|
||||
if upper.starts_with("EXPLORE") || upper == "LLM" || upper == "NEW" {
|
||||
return Ok(json!({
|
||||
"matched": false,
|
||||
"mode": "explore",
|
||||
"decision": "EXPLORE",
|
||||
"needs_llm": false,
|
||||
}));
|
||||
}
|
||||
let slug = if let Some(rest) = line.strip_prefix("SKILL:") {
|
||||
rest.trim()
|
||||
} else if let Some(rest) = line.strip_prefix("skill:") {
|
||||
rest.trim()
|
||||
} else if let Some(rest) = line.strip_prefix("调用:") {
|
||||
rest.trim()
|
||||
} else {
|
||||
line.trim()
|
||||
};
|
||||
if slug.is_empty() {
|
||||
return Ok(json!({
|
||||
"matched": false,
|
||||
"mode": "explore",
|
||||
"decision": "EXPLORE",
|
||||
"reason": "empty_slug",
|
||||
}));
|
||||
}
|
||||
let hit = crate::kb::kb_lookup(&json!({
|
||||
"kb_dir": kb,
|
||||
"goal": task,
|
||||
"slug": slug,
|
||||
"near_match": true,
|
||||
"near_threshold": 0.5,
|
||||
}))?;
|
||||
if hit.is_null() {
|
||||
return Ok(json!({
|
||||
"matched": false,
|
||||
"mode": "miss",
|
||||
"decision": "SKILL",
|
||||
"slug": slug,
|
||||
"reason": "slug_not_in_kb",
|
||||
}));
|
||||
}
|
||||
Ok(json!({
|
||||
"matched": true,
|
||||
"mode": "routed",
|
||||
"decision": "SKILL",
|
||||
"backend": opt_text(args, "backend").unwrap_or("llm"),
|
||||
"slug": slug,
|
||||
"skill": hit.get("skill").cloned().unwrap_or(Value::Null),
|
||||
"resource": hit.get("resource").cloned().unwrap_or(Value::Null),
|
||||
"llm_free": hit.get("llm_free").cloned().unwrap_or(Value::Bool(false)),
|
||||
"confidence": 0.85,
|
||||
"needs_llm": false,
|
||||
}))
|
||||
}
|
||||
|
||||
/// Compile policy from successful episode skill selections (Policy Compilation).
|
||||
pub fn compile_policy(args: &Value) -> Result<Value, String> {
|
||||
let root = memory_root(args);
|
||||
let mem_arg = opt_text(args, "memory_dir").unwrap_or(DEFAULT_MEMORY_DIR);
|
||||
let _ = memory::memory_ensure(&json!({ "memory_dir": mem_arg }));
|
||||
let min_evidence = opt_i64(args, "min_evidence", opt_i64(args, "improve_every", 3));
|
||||
let listed = memory::list_episodes(&json!({
|
||||
"memory_dir": mem_arg,
|
||||
"status": "success",
|
||||
"limit": 500,
|
||||
}))?;
|
||||
let eps = listed
|
||||
.get("episodes")
|
||||
.and_then(|v| v.as_array())
|
||||
.cloned()
|
||||
.unwrap_or_default();
|
||||
|
||||
// task_hash → (slug → count, sample task, skill path)
|
||||
let mut by_hash: HashMap<String, HashMap<String, (i64, String, String)>> = HashMap::new();
|
||||
for e in &eps {
|
||||
let id = e.get("id").and_then(|v| v.as_str()).unwrap_or("");
|
||||
if id.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let full = memory::get_episode(&json!({
|
||||
"memory_dir": mem_arg,
|
||||
"id": id,
|
||||
}))?;
|
||||
if full.is_null() {
|
||||
continue;
|
||||
}
|
||||
let src = full.get("source").and_then(|v| v.as_str()).unwrap_or("");
|
||||
let skill = extract_fm(src, "skill").unwrap_or_default();
|
||||
if skill.is_empty() || skill == "none" || skill == "null" {
|
||||
continue;
|
||||
}
|
||||
let task_hash = extract_fm(src, "task_hash")
|
||||
.or_else(|| e.get("task_hash").and_then(|v| v.as_str()).map(|s| s.to_string()))
|
||||
.unwrap_or_default();
|
||||
if task_hash.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let task = full
|
||||
.get("task")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
let slug = extract_fm(src, "slug")
|
||||
.filter(|s| !s.is_empty())
|
||||
.unwrap_or_else(|| goal_slug_str(&task));
|
||||
// Prefer skill path as key when it looks like a path; else slug.
|
||||
let key = if skill.contains('/') || skill.ends_with(".md") {
|
||||
skill.clone()
|
||||
} else {
|
||||
slug.clone()
|
||||
};
|
||||
let entry = by_hash.entry(task_hash).or_default();
|
||||
let cell = entry.entry(key).or_insert((0, task, skill));
|
||||
cell.0 += 1;
|
||||
}
|
||||
|
||||
let mut rules: Vec<PolicyRule> = Vec::new();
|
||||
for (task_hash, skills) in by_hash {
|
||||
let Some((skill_key, (count, task, skill_path))) =
|
||||
skills.into_iter().max_by_key(|(_, (c, _, _))| *c)
|
||||
else {
|
||||
continue;
|
||||
};
|
||||
if count < min_evidence {
|
||||
continue;
|
||||
}
|
||||
let slug = if skill_key.contains('/') {
|
||||
Path::new(&skill_key)
|
||||
.file_stem()
|
||||
.and_then(|s| s.to_str())
|
||||
.unwrap_or(&skill_key)
|
||||
.trim_end_matches(".mq")
|
||||
.to_string()
|
||||
} else {
|
||||
skill_key
|
||||
};
|
||||
let confidence = (count as f64 / (count as f64 + 1.0)).min(0.99);
|
||||
rules.push(PolicyRule {
|
||||
task_hash,
|
||||
slug,
|
||||
skill: skill_path,
|
||||
task,
|
||||
count,
|
||||
confidence,
|
||||
});
|
||||
}
|
||||
rules.sort_by(|a, b| b.count.cmp(&a.count));
|
||||
|
||||
if rules.is_empty() && !opt_bool(args, "force", false) {
|
||||
return Ok(json!({
|
||||
"ok": false,
|
||||
"compiled": false,
|
||||
"reason": "insufficient_stable_mappings",
|
||||
"min_evidence": min_evidence,
|
||||
}));
|
||||
}
|
||||
|
||||
let path = write_policy(&root, &rules)?;
|
||||
Ok(json!({
|
||||
"ok": true,
|
||||
"compiled": true,
|
||||
"path": path.to_string_lossy(),
|
||||
"rules": rules.len() as i64,
|
||||
"min_evidence": min_evidence,
|
||||
}))
|
||||
}
|
||||
|
||||
fn extract_fm(source: &str, key: &str) -> Option<String> {
|
||||
let source = source.strip_prefix('\u{feff}').unwrap_or(source);
|
||||
let lines: Vec<&str> = source.lines().collect();
|
||||
if lines.first().map(|l| l.trim()) != Some("---") {
|
||||
return None;
|
||||
}
|
||||
let prefix = format!("{key}:");
|
||||
for line in lines.iter().skip(1) {
|
||||
let t = line.trim();
|
||||
if t == "---" {
|
||||
break;
|
||||
}
|
||||
if let Some(rest) = t.strip_prefix(&prefix) {
|
||||
return Some(rest.trim().trim_matches('"').to_string());
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Learn gate for policy: compile when enough skill-tagged episodes exist.
|
||||
pub fn maybe_compile_policy(args: &Value) -> Result<Value, String> {
|
||||
let compiled = compile_policy(args)?;
|
||||
Ok(json!({
|
||||
"policy_learned": compiled.get("ok").and_then(|v| v.as_bool()).unwrap_or(false),
|
||||
"compile": compiled,
|
||||
}))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn compile_and_route_marqdo() {
|
||||
let dir = std::env::temp_dir().join(format!("marqdo-route-{}", std::process::id()));
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
fs::create_dir_all(&dir).unwrap();
|
||||
let mem = dir.join("agent-memory");
|
||||
let kb = dir.join("agent-kb");
|
||||
let mem_s = mem.to_string_lossy().into_owned();
|
||||
let kb_s = kb.to_string_lossy().into_owned();
|
||||
let task = "Reply with exactly the word pong and nothing else.";
|
||||
|
||||
// Seed a solidified skill via memory compile path.
|
||||
for _ in 0..3 {
|
||||
memory::record_episode(&json!({
|
||||
"memory_dir": mem_s,
|
||||
"task": task,
|
||||
"status": "ok",
|
||||
"mode": "explore",
|
||||
"result": "pong",
|
||||
"skill": "concepts/skills/reply-with-exactly-the-word-pong-and-nothing-els.md",
|
||||
"llm_calls": 1,
|
||||
}))
|
||||
.unwrap();
|
||||
}
|
||||
let learned = memory::maybe_learn(&json!({
|
||||
"memory_dir": mem_s,
|
||||
"kb_dir": kb_s,
|
||||
"task": task,
|
||||
"improve_every": 3,
|
||||
"promote": true,
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(learned.get("learned").and_then(|v| v.as_bool()), Some(true));
|
||||
|
||||
// Tag episodes with skill path for policy mining (re-record with skill).
|
||||
for _ in 0..3 {
|
||||
memory::record_episode(&json!({
|
||||
"memory_dir": mem_s,
|
||||
"task": task,
|
||||
"status": "ok",
|
||||
"mode": "compiled",
|
||||
"result": "pong",
|
||||
"skill": "concepts/skills/reply-with-exactly-the-word-pong-and-nothing-els.md",
|
||||
"llm_calls": 0,
|
||||
}))
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
let pol = compile_policy(&json!({
|
||||
"memory_dir": mem_s,
|
||||
"min_evidence": 3,
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(pol.get("ok").and_then(|v| v.as_bool()), Some(true), "{pol}");
|
||||
|
||||
let routed = route(&json!({
|
||||
"memory_dir": mem_s,
|
||||
"kb_dir": kb_s,
|
||||
"task": task,
|
||||
"backend": "auto",
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
routed.get("matched").and_then(|v| v.as_bool()),
|
||||
Some(true),
|
||||
"{routed}"
|
||||
);
|
||||
assert_eq!(
|
||||
routed.get("needs_llm").and_then(|v| v.as_bool()),
|
||||
Some(false),
|
||||
"{routed}"
|
||||
);
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
[package]
|
||||
name = "marqdo_plugin_llm"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
license = "Apache-2.0"
|
||||
description = "Marqdo LLM transport plugin (ABI v2) — OpenAI-compatible chat completions"
|
||||
publish = false
|
||||
|
||||
[lib]
|
||||
name = "llm"
|
||||
crate-type = ["cdylib"]
|
||||
|
||||
[dependencies]
|
||||
serde_json = "1"
|
||||
# Pin with host (edition2024-safe for rustc 1.81).
|
||||
ureq = { version = "=2.9.7", default-features = false, features = ["tls", "gzip"] }
|
||||
idna_adapter = { version = "=1.1.0" }
|
||||
zeroize = { version = "=1.8.1" }
|
||||
@@ -0,0 +1,369 @@
|
||||
//! Marqdo LLM transport plugin (C ABI v2): OpenAI-compatible chat completions.
|
||||
|
||||
use std::ffi::{CStr, CString};
|
||||
use std::io::{BufRead, BufReader, Read};
|
||||
use std::os::raw::{c_char, c_int, c_void};
|
||||
use std::ptr;
|
||||
use std::time::Duration;
|
||||
|
||||
use serde_json::{json, Map, Value};
|
||||
|
||||
const ABI_VERSION: u32 = 2;
|
||||
const MAX_BODY: usize = 32 * 1024 * 1024;
|
||||
|
||||
type PluginFn = unsafe extern "C" fn(
|
||||
args_json: *const c_char,
|
||||
out_json: *mut *mut c_char,
|
||||
err_msg: *mut *mut c_char,
|
||||
) -> c_int;
|
||||
|
||||
type HostQueryFn = unsafe extern "C" fn(
|
||||
userdata: *mut c_void,
|
||||
name: *const c_char,
|
||||
args_json: *const c_char,
|
||||
out_json: *mut *mut c_char,
|
||||
err_msg: *mut *mut c_char,
|
||||
) -> c_int;
|
||||
|
||||
#[repr(C)]
|
||||
pub struct MarqdoHostApi {
|
||||
pub userdata: *mut c_void,
|
||||
pub register_fn: Option<
|
||||
unsafe extern "C" fn(
|
||||
userdata: *mut c_void,
|
||||
name: *const c_char,
|
||||
params: *const c_char,
|
||||
fn_ptr: PluginFn,
|
||||
) -> c_int,
|
||||
>,
|
||||
pub alloc: Option<unsafe extern "C" fn(n: usize) -> *mut c_void>,
|
||||
pub free: Option<unsafe extern "C" fn(p: *mut c_void)>,
|
||||
pub host_query: Option<HostQueryFn>,
|
||||
}
|
||||
|
||||
static mut HOST_FREE: Option<unsafe extern "C" fn(*mut c_void)> = None;
|
||||
static mut HOST_ALLOC: Option<unsafe extern "C" fn(usize) -> *mut c_void> = None;
|
||||
|
||||
unsafe fn host_strdup(s: &str) -> *mut c_char {
|
||||
let alloc = HOST_ALLOC.expect("host alloc");
|
||||
let bytes = s.as_bytes();
|
||||
let p = alloc(bytes.len() + 1) as *mut u8;
|
||||
if p.is_null() {
|
||||
return ptr::null_mut();
|
||||
}
|
||||
std::ptr::copy_nonoverlapping(bytes.as_ptr(), p, bytes.len());
|
||||
*p.add(bytes.len()) = 0;
|
||||
p as *mut c_char
|
||||
}
|
||||
|
||||
fn set_out(out: *mut *mut c_char, s: &str) {
|
||||
if out.is_null() {
|
||||
return;
|
||||
}
|
||||
unsafe {
|
||||
*out = host_strdup(s);
|
||||
}
|
||||
}
|
||||
|
||||
fn set_err(err: *mut *mut c_char, s: &str) {
|
||||
if err.is_null() {
|
||||
return;
|
||||
}
|
||||
unsafe {
|
||||
*err = host_strdup(s);
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_args(args_json: *const c_char) -> Result<Value, String> {
|
||||
if args_json.is_null() {
|
||||
return Ok(json!({}));
|
||||
}
|
||||
let s = unsafe { CStr::from_ptr(args_json) }
|
||||
.to_str()
|
||||
.map_err(|_| "args not utf-8".to_string())?;
|
||||
if s.trim().is_empty() {
|
||||
return Ok(json!({}));
|
||||
}
|
||||
serde_json::from_str(s).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn reply(out_json: *mut *mut c_char, err_msg: *mut *mut c_char, r: Result<Value, String>) -> c_int {
|
||||
match r {
|
||||
Ok(v) => {
|
||||
set_out(out_json, &v.to_string());
|
||||
0
|
||||
}
|
||||
Err(e) => {
|
||||
set_err(err_msg, &e);
|
||||
1
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn opt_text<'a>(args: &'a Value, key: &str) -> Option<&'a str> {
|
||||
args.get(key).and_then(|v| v.as_str()).filter(|s| !s.is_empty())
|
||||
}
|
||||
|
||||
fn arg_text<'a>(args: &'a Value, key: &str) -> Result<&'a str, String> {
|
||||
opt_text(args, key).ok_or_else(|| format!("missing text `{key}`"))
|
||||
}
|
||||
|
||||
fn opt_bool(args: &Value, key: &str, default: bool) -> bool {
|
||||
match args.get(key) {
|
||||
None | Some(Value::Null) => default,
|
||||
Some(Value::Bool(b)) => *b,
|
||||
Some(Value::String(s)) => matches!(s.as_str(), "true" | "True" | "1" | "yes"),
|
||||
Some(Value::Number(n)) => n.as_i64().unwrap_or(0) != 0,
|
||||
_ => default,
|
||||
}
|
||||
}
|
||||
|
||||
fn read_limited(mut r: impl Read) -> Result<String, String> {
|
||||
let mut buf = Vec::new();
|
||||
let mut chunk = [0u8; 8192];
|
||||
loop {
|
||||
let n = r.read(&mut chunk).map_err(|e| e.to_string())?;
|
||||
if n == 0 {
|
||||
break;
|
||||
}
|
||||
if buf.len() + n > MAX_BODY {
|
||||
return Err("response body too large".into());
|
||||
}
|
||||
buf.extend_from_slice(&chunk[..n]);
|
||||
}
|
||||
String::from_utf8(buf).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
fn usage_from(data: &Value) -> Value {
|
||||
let u = data.get("usage").cloned().unwrap_or(Value::Null);
|
||||
let pt = u.get("prompt_tokens").and_then(|v| v.as_i64()).unwrap_or(0);
|
||||
let ct = u
|
||||
.get("completion_tokens")
|
||||
.and_then(|v| v.as_i64())
|
||||
.unwrap_or(0);
|
||||
let tt = u
|
||||
.get("total_tokens")
|
||||
.and_then(|v| v.as_i64())
|
||||
.unwrap_or(pt + ct);
|
||||
json!({
|
||||
"prompt_tokens": pt,
|
||||
"completion_tokens": ct,
|
||||
"total_tokens": tt,
|
||||
})
|
||||
}
|
||||
|
||||
fn tool_calls_from(data: &Value) -> Value {
|
||||
data.get("choices")
|
||||
.and_then(|c| c.as_array())
|
||||
.and_then(|a| a.first())
|
||||
.and_then(|c0| c0.get("message"))
|
||||
.and_then(|m| m.get("tool_calls"))
|
||||
.cloned()
|
||||
.unwrap_or_else(|| json!([]))
|
||||
}
|
||||
|
||||
fn parse_sse_events(reader: impl BufRead, echo: bool) -> Result<Vec<Value>, String> {
|
||||
let mut events = Vec::new();
|
||||
let mut data_lines: Vec<String> = Vec::new();
|
||||
for line in reader.lines() {
|
||||
let line = line.map_err(|e| e.to_string())?;
|
||||
if echo {
|
||||
eprint!("{line}\n");
|
||||
}
|
||||
if line.is_empty() {
|
||||
if !data_lines.is_empty() {
|
||||
let data = data_lines.join("\n");
|
||||
data_lines.clear();
|
||||
if data.trim() == "[DONE]" {
|
||||
events.push(json!({"type": "done", "data": "[DONE]"}));
|
||||
break;
|
||||
}
|
||||
events.push(json!({"type": "data", "data": data}));
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if let Some(rest) = line.strip_prefix("data:") {
|
||||
data_lines.push(rest.trim_start().to_string());
|
||||
}
|
||||
}
|
||||
if !data_lines.is_empty() {
|
||||
let data = data_lines.join("\n");
|
||||
if data.trim() == "[DONE]" {
|
||||
events.push(json!({"type": "done", "data": "[DONE]"}));
|
||||
} else {
|
||||
events.push(json!({"type": "data", "data": data}));
|
||||
}
|
||||
}
|
||||
Ok(events)
|
||||
}
|
||||
|
||||
fn chat_completions(args: &Value) -> Result<Value, String> {
|
||||
let base_url = arg_text(args, "base_url")?;
|
||||
let api_key = arg_text(args, "api_key").unwrap_or("");
|
||||
let model = arg_text(args, "model")?;
|
||||
let prompt = arg_text(args, "prompt")?;
|
||||
let stream = opt_bool(args, "stream", false);
|
||||
let echo = opt_bool(args, "echo", false);
|
||||
let suffix = opt_text(args, "suffix").unwrap_or("/chat/completions");
|
||||
let bearer = opt_text(args, "bearer").unwrap_or("Bearer ");
|
||||
|
||||
let url = format!("{}{}", base_url.trim_end_matches('/'), suffix);
|
||||
if !(url.starts_with("http://") || url.starts_with("https://")) {
|
||||
return Err("url must start with http:// or https://".into());
|
||||
}
|
||||
|
||||
let mut req_body = Map::new();
|
||||
req_body.insert("model".into(), Value::String(model.to_string()));
|
||||
req_body.insert(
|
||||
"messages".into(),
|
||||
json!([{"role": "user", "content": prompt}]),
|
||||
);
|
||||
if stream {
|
||||
req_body.insert("stream".into(), Value::Bool(true));
|
||||
}
|
||||
let body = Value::Object(req_body).to_string();
|
||||
let auth = format!("{bearer}{api_key}");
|
||||
|
||||
let agent = ureq::AgentBuilder::new()
|
||||
.timeout_connect(Duration::from_secs(15))
|
||||
.timeout_read(Duration::from_secs(120))
|
||||
.build();
|
||||
let mut req = agent
|
||||
.post(&url)
|
||||
.set("User-Agent", "marqdo-plugin-llm")
|
||||
.set("Content-Type", "application/json; charset=utf-8");
|
||||
if stream {
|
||||
req = req.set("Accept", "text/event-stream");
|
||||
}
|
||||
if !api_key.is_empty() {
|
||||
req = req.set("Authorization", &auth);
|
||||
}
|
||||
|
||||
if stream {
|
||||
match req.send_string(&body) {
|
||||
Ok(r) => {
|
||||
let status = r.status();
|
||||
if status != 200 {
|
||||
let text = read_limited(r.into_reader())?;
|
||||
return Err(format!("HTTP {status}: {text}"));
|
||||
}
|
||||
let events = parse_sse_events(BufReader::new(r.into_reader()), echo)?;
|
||||
Ok(json!({
|
||||
"events": events,
|
||||
"finish": "stop",
|
||||
"transport": "plugin",
|
||||
}))
|
||||
}
|
||||
Err(ureq::Error::Status(code, r)) => {
|
||||
let text = read_limited(r.into_reader()).unwrap_or_default();
|
||||
Err(format!("HTTP {code}: {text}"))
|
||||
}
|
||||
Err(e) => Err(format!("http POST sse {url}: {e}")),
|
||||
}
|
||||
} else {
|
||||
match req.send_string(&body) {
|
||||
Ok(r) => {
|
||||
let status = r.status();
|
||||
let text = read_limited(r.into_reader())?;
|
||||
if status != 200 {
|
||||
return Err(format!("HTTP {status}: {text}"));
|
||||
}
|
||||
let data: Value = serde_json::from_str(&text).map_err(|e| e.to_string())?;
|
||||
let msg = data
|
||||
.get("choices")
|
||||
.and_then(|c| c.as_array())
|
||||
.and_then(|a| a.first())
|
||||
.and_then(|c0| c0.get("message"))
|
||||
.cloned()
|
||||
.unwrap_or(json!({}));
|
||||
let content = msg
|
||||
.get("content")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
let fr = data
|
||||
.get("choices")
|
||||
.and_then(|c| c.as_array())
|
||||
.and_then(|a| a.first())
|
||||
.and_then(|c0| c0.get("finish_reason"))
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("stop")
|
||||
.to_string();
|
||||
Ok(json!({
|
||||
"text": content,
|
||||
"usage": usage_from(&data),
|
||||
"finish": fr,
|
||||
"tool_calls": tool_calls_from(&data),
|
||||
"transport": "plugin",
|
||||
}))
|
||||
}
|
||||
Err(ureq::Error::Status(code, r)) => {
|
||||
let text = read_limited(r.into_reader()).unwrap_or_default();
|
||||
Err(format!("HTTP {code}: {text}"))
|
||||
}
|
||||
Err(e) => Err(format!("http POST {url}: {e}")),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
unsafe extern "C" fn llm_chat_completions(
|
||||
args_json: *const c_char,
|
||||
out_json: *mut *mut c_char,
|
||||
err_msg: *mut *mut c_char,
|
||||
) -> c_int {
|
||||
let r = (|| {
|
||||
let args = parse_args(args_json)?;
|
||||
chat_completions(&args)
|
||||
})();
|
||||
reply(out_json, err_msg, r)
|
||||
}
|
||||
|
||||
unsafe extern "C" fn llm_ping(
|
||||
_args_json: *const c_char,
|
||||
out_json: *mut *mut c_char,
|
||||
err_msg: *mut *mut c_char,
|
||||
) -> c_int {
|
||||
reply(out_json, err_msg, Ok(json!({"ok": true, "plugin": "llm"})))
|
||||
}
|
||||
|
||||
fn register(host: &MarqdoHostApi, name: &str, params: &str, fn_ptr: PluginFn) -> c_int {
|
||||
let register = match host.register_fn {
|
||||
Some(f) => f,
|
||||
None => return 1,
|
||||
};
|
||||
let n = CString::new(name).unwrap();
|
||||
let p = CString::new(params).unwrap();
|
||||
unsafe { register(host.userdata, n.as_ptr(), p.as_ptr(), fn_ptr) }
|
||||
}
|
||||
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn marqdo_plugin_abi_version() -> u32 {
|
||||
ABI_VERSION
|
||||
}
|
||||
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn marqdo_plugin_shutdown() {}
|
||||
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn marqdo_plugin_init(host: *const MarqdoHostApi) -> c_int {
|
||||
if host.is_null() {
|
||||
return 1;
|
||||
}
|
||||
let host = &*host;
|
||||
HOST_ALLOC = host.alloc;
|
||||
HOST_FREE = host.free;
|
||||
let regs = [
|
||||
("llm_ping", "", llm_ping as PluginFn),
|
||||
(
|
||||
"llm_chat_completions",
|
||||
"base_url,api_key,model,prompt,stream,echo,suffix,bearer",
|
||||
llm_chat_completions as PluginFn,
|
||||
),
|
||||
];
|
||||
for (name, params, f) in regs {
|
||||
if register(host, name, params, f) != 0 {
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
0
|
||||
}
|
||||
@@ -0,0 +1,326 @@
|
||||
package main
|
||||
|
||||
/*
|
||||
#cgo CFLAGS: -I${SRCDIR}/abi
|
||||
#include "marqdo_abi_cgo.h"
|
||||
|
||||
extern int web_render_nodes(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_render_document(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_dom_markdown(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_serve_root(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_inspect(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_app_use(char *args_json, char **out_json, char **err_msg);
|
||||
extern int web_route_use(char *args_json, char **out_json, char **err_msg);
|
||||
*/
|
||||
import "C"
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/marqdo/marqdo/plugins/web/internal/app"
|
||||
"github.com/marqdo/marqdo/plugins/web/internal/artifact"
|
||||
"github.com/marqdo/marqdo/plugins/web/internal/httpx"
|
||||
"github.com/marqdo/marqdo/plugins/web/internal/middleware"
|
||||
"github.com/marqdo/marqdo/plugins/web/internal/page"
|
||||
)
|
||||
|
||||
//export web_render_nodes
|
||||
func web_render_nodes(argsJSON *C.char, outJSON **C.char, errMsg **C.char) C.int {
|
||||
args, err := parseArgs(argsJSON)
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
frag, err := artifact.RenderNodes(args["nodes"])
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
pg := asPageMap(args["page"])
|
||||
title := ""
|
||||
if pg != nil {
|
||||
if t, ok := pg["title"].(string); ok {
|
||||
title = t
|
||||
}
|
||||
}
|
||||
if t, ok := argStr(args, "title"); ok && t != "" {
|
||||
title = t
|
||||
}
|
||||
return replyJSON(outJSON, errMsg, artifact.RenderHTMLPage(title, frag), nil)
|
||||
}
|
||||
|
||||
//export web_render_document
|
||||
func web_render_document(argsJSON *C.char, outJSON **C.char, errMsg **C.char) C.int {
|
||||
args, err := parseArgs(argsJSON)
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
pg := asPageMap(args["page"])
|
||||
title := ""
|
||||
body := ""
|
||||
if pg != nil {
|
||||
if t, ok := pg["title"].(string); ok {
|
||||
title = t
|
||||
}
|
||||
if b, ok := pg["body"].(string); ok {
|
||||
body = b
|
||||
}
|
||||
}
|
||||
if b, ok := argStr(args, "body"); ok && b != "" {
|
||||
body = b
|
||||
}
|
||||
if t, ok := argStr(args, "title"); ok && t != "" {
|
||||
title = t
|
||||
}
|
||||
return replyJSON(outJSON, errMsg, artifact.RenderDocument(title, body), nil)
|
||||
}
|
||||
|
||||
//export web_dom_markdown
|
||||
func web_dom_markdown(argsJSON *C.char, outJSON **C.char, errMsg **C.char) C.int {
|
||||
args, err := parseArgs(argsJSON)
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
text, _ := argStr(args, "text")
|
||||
return replyJSON(outJSON, errMsg, artifact.MarkdownHTML(text), nil)
|
||||
}
|
||||
|
||||
//export web_serve_root
|
||||
func web_serve_root(argsJSON *C.char, outJSON **C.char, errMsg **C.char) C.int {
|
||||
args, err := parseArgs(argsJSON)
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
root, _ := argStr(args, "root")
|
||||
if root == "" {
|
||||
root = "."
|
||||
}
|
||||
abs, err := filepath.Abs(root)
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
if st, err := os.Stat(abs); err != nil || !st.IsDir() {
|
||||
return replyJSON(outJSON, errMsg, nil, fmt.Errorf("web.serve: root not a directory: %s", abs))
|
||||
}
|
||||
files, err := artifact.ScanRoot(abs)
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
host, _ := argStr(args, "host")
|
||||
if host == "" {
|
||||
host = "127.0.0.1"
|
||||
}
|
||||
port := float64(18081)
|
||||
if v, ok := args["port"]; ok && v != nil {
|
||||
switch t := v.(type) {
|
||||
case float64:
|
||||
port = t
|
||||
case string:
|
||||
if n, e := strconv.Atoi(t); e == nil {
|
||||
port = float64(n)
|
||||
}
|
||||
}
|
||||
}
|
||||
homeArgs := map[string]any{"title": "Marqdo"}
|
||||
var homeMeta map[string]string
|
||||
for _, f := range files {
|
||||
if f.Kind == "web" && (f.Route == "/" || f.Rel == "index.mq.md") {
|
||||
title := f.Meta["title"]
|
||||
if title == "" {
|
||||
title = f.Meta["标题"]
|
||||
}
|
||||
if title == "" {
|
||||
title = "Marqdo"
|
||||
}
|
||||
homeArgs = map[string]any{
|
||||
"title": title,
|
||||
"intro": artifact.MarkdownHTML(f.Body),
|
||||
}
|
||||
homeMeta = f.Meta
|
||||
break
|
||||
}
|
||||
}
|
||||
home := page.New(homeArgs)
|
||||
stampArtifactData(home, homeMeta)
|
||||
a := app.New(map[string]any{
|
||||
"page": home,
|
||||
"db": args["db"],
|
||||
"host": host,
|
||||
"port": port,
|
||||
})
|
||||
artifactRoutes := map[string]any{}
|
||||
for _, f := range files {
|
||||
if f.Kind == "web" {
|
||||
if f.Route == "/" {
|
||||
continue
|
||||
}
|
||||
title := f.Meta["title"]
|
||||
if title == "" {
|
||||
title = f.Meta["标题"]
|
||||
}
|
||||
pg := page.New(map[string]any{
|
||||
"title": title,
|
||||
"intro": artifact.MarkdownHTML(f.Body),
|
||||
})
|
||||
stampArtifactData(pg, f.Meta)
|
||||
out, err := app.Route(a, f.Route, pg)
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
a = out
|
||||
continue
|
||||
}
|
||||
// endpoint
|
||||
req := f.Meta["request"]
|
||||
if req == "" {
|
||||
req = f.Meta["请求"]
|
||||
}
|
||||
if req == "" {
|
||||
req = "json"
|
||||
}
|
||||
resp := f.Meta["response"]
|
||||
if resp == "" {
|
||||
resp = f.Meta["响应"]
|
||||
}
|
||||
if resp == "" {
|
||||
resp = "json"
|
||||
}
|
||||
auth := f.Meta["auth"]
|
||||
if auth == "" {
|
||||
auth = f.Meta["鉴权"]
|
||||
}
|
||||
key := f.Method + " " + f.Route
|
||||
artifactRoutes[key] = map[string]any{
|
||||
"method": f.Method,
|
||||
"path": f.Route,
|
||||
"file": f.Path,
|
||||
"request": req,
|
||||
"response": resp,
|
||||
"auth": auth,
|
||||
"kind": "endpoint",
|
||||
}
|
||||
}
|
||||
if len(artifactRoutes) > 0 {
|
||||
a["artifact_routes"] = artifactRoutes
|
||||
}
|
||||
if mw := args["middleware"]; mw != nil {
|
||||
out, err := middleware.Configure(a, map[string]any{"security": mw})
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
a = out
|
||||
}
|
||||
if sd, ok := argStr(args, "static_dir"); ok && sd != "" {
|
||||
out, err := app.Static(a, sd, "/static")
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
a = out
|
||||
}
|
||||
a["inspect"] = artifact.InspectBag(a, files)
|
||||
_, err = httpx.Listen(a, abs)
|
||||
return replyJSON(outJSON, errMsg, map[string]any{"ok": true, "value": a["inspect"], "error": nil}, err)
|
||||
}
|
||||
|
||||
//export web_inspect
|
||||
func web_inspect(argsJSON *C.char, outJSON **C.char, errMsg **C.char) C.int {
|
||||
args, err := parseArgs(argsJSON)
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
a := asPageMap(args["app"])
|
||||
if a != nil {
|
||||
if insp, ok := a["inspect"].(map[string]any); ok {
|
||||
return replyJSON(outJSON, errMsg, insp, nil)
|
||||
}
|
||||
}
|
||||
return replyJSON(outJSON, errMsg, artifact.InspectBag(a, nil), nil)
|
||||
}
|
||||
|
||||
//export web_app_use
|
||||
func web_app_use(argsJSON *C.char, outJSON **C.char, errMsg **C.char) C.int {
|
||||
args, err := parseArgs(argsJSON)
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
a := asPageMap(args["app"])
|
||||
out, err := middleware.Configure(a, map[string]any{"security": args["middleware"]})
|
||||
return replyJSON(outJSON, errMsg, out, err)
|
||||
}
|
||||
|
||||
//export web_route_use
|
||||
func web_route_use(argsJSON *C.char, outJSON **C.char, errMsg **C.char) C.int {
|
||||
args, err := parseArgs(argsJSON)
|
||||
if err != nil {
|
||||
return replyJSON(outJSON, errMsg, nil, err)
|
||||
}
|
||||
rt := asPageMap(args["route"])
|
||||
if rt == nil {
|
||||
rt = map[string]any{}
|
||||
}
|
||||
rt["middleware"] = args["middleware"]
|
||||
return replyJSON(outJSON, errMsg, rt, nil)
|
||||
}
|
||||
|
||||
// stampArtifactData applies flat Document metadata (ADR 0007) onto a page bag.
|
||||
func stampArtifactData(pg map[string]any, meta map[string]string) {
|
||||
if pg == nil || meta == nil {
|
||||
return
|
||||
}
|
||||
src := meta["data_source"]
|
||||
if src == "" {
|
||||
src = meta["数据源"]
|
||||
}
|
||||
if src != "" {
|
||||
pg["data_source"] = src
|
||||
}
|
||||
order := meta["data_order"]
|
||||
if order == "" {
|
||||
order = meta["排序"]
|
||||
}
|
||||
if order != "" {
|
||||
pg["order"] = order
|
||||
}
|
||||
where := meta["data_where"]
|
||||
if where == "" {
|
||||
where = meta["条件"]
|
||||
}
|
||||
if where != "" {
|
||||
// col=val,col2=val2 → query map (values may contain {param})
|
||||
q := map[string]any{}
|
||||
for _, part := range strings.Split(where, ",") {
|
||||
part = strings.TrimSpace(part)
|
||||
if part == "" {
|
||||
continue
|
||||
}
|
||||
k, v, ok := strings.Cut(part, "=")
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
q[strings.TrimSpace(k)] = strings.TrimSpace(v)
|
||||
}
|
||||
if len(q) > 0 {
|
||||
pg["query"] = q
|
||||
// Filtered Document routes render as article detail (007 /post/{slug}).
|
||||
pg["detail"] = true
|
||||
}
|
||||
}
|
||||
if d := meta["detail"]; d != "" {
|
||||
pg["detail"] = d == "true" || d == "True" || d == "1"
|
||||
}
|
||||
if d := meta["详情"]; d != "" {
|
||||
pg["detail"] = d == "true" || d == "True" || d == "1" || d == "是"
|
||||
}
|
||||
prefix := meta["link_prefix"]
|
||||
if prefix == "" {
|
||||
prefix = meta["链接前缀"]
|
||||
}
|
||||
if prefix != "" {
|
||||
pg["link_prefix"] = prefix
|
||||
} else if src != "" && where == "" {
|
||||
// List pages default card links to /post/{slug}.
|
||||
pg["link_prefix"] = "/post/"
|
||||
}
|
||||
}
|
||||
@@ -256,6 +256,9 @@ func OIDC(appBag map[string]any, opts map[string]any) (map[string]any, error) {
|
||||
"scopes": cfg.ScopesCSV(),
|
||||
"callback_path": cfg.Callback(),
|
||||
}
|
||||
if cfg.RedirectOrigins != "" {
|
||||
bag["redirect_origins"] = cfg.RedirectOrigins
|
||||
}
|
||||
if cfg.AuthorizeURL != "" {
|
||||
bag["authorize_url"] = cfg.AuthorizeURL
|
||||
}
|
||||
|
||||
@@ -0,0 +1,298 @@
|
||||
// Package artifact implements Document/Endpoint View render, serve scan, and inspect (ADR 0007).
|
||||
package artifact
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"html"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"github.com/marqdo/marqdo/plugins/web/internal/render"
|
||||
"github.com/marqdo/marqdo/plugins/web/internal/table"
|
||||
)
|
||||
|
||||
// MetaType returns canonical kind: web | endpoint | "" from flat frontmatter map.
|
||||
func MetaType(meta map[string]string) string {
|
||||
if v := strings.TrimSpace(meta["type"]); v != "" {
|
||||
return normalizeType(v)
|
||||
}
|
||||
if v := strings.TrimSpace(meta["类型"]); v != "" {
|
||||
return normalizeType(v)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func normalizeType(v string) string {
|
||||
switch v {
|
||||
case "web", "网页":
|
||||
return "web"
|
||||
case "endpoint", "端点":
|
||||
return "endpoint"
|
||||
default:
|
||||
return v
|
||||
}
|
||||
}
|
||||
|
||||
// ParseFrontmatter reads flat key: value pairs from a .mq.md file head.
|
||||
func ParseFrontmatter(src string) (meta map[string]string, body string) {
|
||||
meta = map[string]string{}
|
||||
s := strings.TrimPrefix(src, "\ufeff")
|
||||
if !strings.HasPrefix(s, "---\n") && !strings.HasPrefix(s, "---\r\n") {
|
||||
return meta, src
|
||||
}
|
||||
rest := s[4:]
|
||||
if strings.HasPrefix(rest, "\r\n") {
|
||||
rest = rest[2:]
|
||||
} else if strings.HasPrefix(rest, "\n") {
|
||||
rest = rest[1:]
|
||||
}
|
||||
end := strings.Index(rest, "\n---\n")
|
||||
if end < 0 {
|
||||
end = strings.Index(rest, "\r\n---\r\n")
|
||||
}
|
||||
if end < 0 {
|
||||
end = strings.Index(rest, "\n---\r\n")
|
||||
}
|
||||
if end < 0 {
|
||||
return meta, src
|
||||
}
|
||||
fm := rest[:end]
|
||||
body = rest[end:]
|
||||
if i := strings.Index(body, "---"); i >= 0 {
|
||||
body = body[i+3:]
|
||||
body = strings.TrimPrefix(body, "\r")
|
||||
body = strings.TrimPrefix(body, "\n")
|
||||
}
|
||||
for _, line := range strings.Split(fm, "\n") {
|
||||
line = strings.TrimSpace(strings.TrimSuffix(line, "\r"))
|
||||
if line == "" || strings.HasPrefix(line, "#") || strings.HasPrefix(line, "import ") || strings.HasPrefix(line, "导入 ") {
|
||||
continue
|
||||
}
|
||||
k, v, ok := strings.Cut(line, ":")
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
k = strings.TrimSpace(k)
|
||||
v = strings.TrimSpace(v)
|
||||
if len(v) >= 2 {
|
||||
if (v[0] == '"' && v[len(v)-1] == '"') || (v[0] == '\'' && v[len(v)-1] == '\'') {
|
||||
v = v[1 : len(v)-1]
|
||||
}
|
||||
}
|
||||
if k != "" {
|
||||
meta[k] = v
|
||||
}
|
||||
}
|
||||
return meta, body
|
||||
}
|
||||
|
||||
// RenderNodes turns a unified View table into an HTML fragment.
|
||||
// Columns: type|slot|value|attrs|style (ZH: 类型|槽|值|属性|样式).
|
||||
func RenderNodes(nodes any) (string, error) {
|
||||
raw := table.AsRows(nodes)
|
||||
list, _ := raw.([]any)
|
||||
var b strings.Builder
|
||||
b.WriteString(`<div class="mq-view">`)
|
||||
for _, item := range list {
|
||||
row, ok := item.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
ty := first(row, "type", "类型")
|
||||
slot := first(row, "slot", "槽")
|
||||
val := first(row, "value", "值")
|
||||
attrs := first(row, "attrs", "属性")
|
||||
style := first(row, "style", "样式")
|
||||
class := "mq-node"
|
||||
if style != "" {
|
||||
class += " " + html.EscapeString(style)
|
||||
}
|
||||
if slot != "" {
|
||||
class += " slot-" + html.EscapeString(slot)
|
||||
}
|
||||
switch strings.ToLower(ty) {
|
||||
case "title", "标题":
|
||||
b.WriteString(`<h1 class="` + class + `">` + inline(val) + `</h1>`)
|
||||
case "link", "链接":
|
||||
href := attrs
|
||||
if href == "" {
|
||||
href = "#"
|
||||
}
|
||||
b.WriteString(`<a class="` + class + `" href="` + html.EscapeString(href) + `">` + inline(val) + `</a>`)
|
||||
case "html", "原文":
|
||||
b.WriteString(`<div class="` + class + `">` + val + `</div>`)
|
||||
default: // text
|
||||
b.WriteString(`<p class="` + class + `">` + inline(val) + `</p>`)
|
||||
}
|
||||
}
|
||||
b.WriteString(`</div>`)
|
||||
return b.String(), nil
|
||||
}
|
||||
|
||||
func inline(s string) string {
|
||||
return html.EscapeString(s)
|
||||
}
|
||||
|
||||
func first(row map[string]any, keys ...string) string {
|
||||
for _, k := range keys {
|
||||
if v, ok := row[k]; ok && v != nil {
|
||||
return fmt.Sprint(v)
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// RenderDocument builds a minimal HTML page from title + markdown body.
|
||||
func RenderDocument(title, body string) string {
|
||||
return RenderHTMLPage(title, render.MarkdownToHTML(body))
|
||||
}
|
||||
|
||||
// RenderHTMLPage wraps an HTML fragment in a minimal document shell.
|
||||
func RenderHTMLPage(title, innerHTML string) string {
|
||||
t := html.EscapeString(title)
|
||||
if t == "" {
|
||||
t = "Marqdo"
|
||||
}
|
||||
return "<!DOCTYPE html><html><head><meta charset=\"utf-8\"><title>" + t +
|
||||
"</title></head><body class=\"mq-document\"><main>" + innerHTML + "</main></body></html>"
|
||||
}
|
||||
|
||||
// MarkdownHTML exposes render helper.
|
||||
func MarkdownHTML(md string) string {
|
||||
return render.MarkdownToHTML(md)
|
||||
}
|
||||
|
||||
// ScanRoot finds Artifact .mq.md files under root.
|
||||
func ScanRoot(root string) ([]ArtifactFile, error) {
|
||||
var out []ArtifactFile
|
||||
err := filepath.WalkDir(root, func(path string, d os.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
name := d.Name()
|
||||
if d.IsDir() {
|
||||
if name == "data" || name == ".git" || name == "node_modules" || strings.HasPrefix(name, ".") {
|
||||
return filepath.SkipDir
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if !strings.HasSuffix(name, ".mq.md") {
|
||||
return nil
|
||||
}
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
meta, body := ParseFrontmatter(string(raw))
|
||||
kind := MetaType(meta)
|
||||
if kind != "web" && kind != "endpoint" {
|
||||
return nil
|
||||
}
|
||||
rel, _ := filepath.Rel(root, path)
|
||||
af := ArtifactFile{Path: path, Rel: rel, Kind: kind, Meta: meta, Body: body}
|
||||
if kind == "web" {
|
||||
af.Route = metaString(meta, "route", "路由")
|
||||
if af.Route == "" {
|
||||
af.Route = routeFromRel(rel)
|
||||
}
|
||||
af.Method = metaString(meta, "method", "方法")
|
||||
if af.Method == "" {
|
||||
af.Method = "GET"
|
||||
}
|
||||
} else {
|
||||
af.Route = metaString(meta, "path", "路径")
|
||||
af.Method = metaString(meta, "method", "方法")
|
||||
if af.Method == "" {
|
||||
af.Method = "GET"
|
||||
}
|
||||
if af.Route == "" {
|
||||
return fmt.Errorf("endpoint %s: missing path/路径", rel)
|
||||
}
|
||||
}
|
||||
out = append(out, af)
|
||||
return nil
|
||||
})
|
||||
return out, err
|
||||
}
|
||||
|
||||
// ArtifactFile is one discovered Document or Endpoint.
|
||||
type ArtifactFile struct {
|
||||
Path string
|
||||
Rel string
|
||||
Kind string
|
||||
Route string
|
||||
Method string
|
||||
Meta map[string]string
|
||||
Body string
|
||||
}
|
||||
|
||||
func metaString(meta map[string]string, keys ...string) string {
|
||||
for _, k := range keys {
|
||||
if v := strings.TrimSpace(meta[k]); v != "" {
|
||||
return v
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func routeFromRel(rel string) string {
|
||||
rel = filepath.ToSlash(rel)
|
||||
if rel == "index.mq.md" {
|
||||
return "/"
|
||||
}
|
||||
rel = strings.TrimSuffix(rel, ".mq.md")
|
||||
rel = strings.TrimSuffix(rel, "/index")
|
||||
if !strings.HasPrefix(rel, "/") {
|
||||
rel = "/" + rel
|
||||
}
|
||||
return rel
|
||||
}
|
||||
|
||||
// SystemRoutes lists runtime-owned routes for inspect.
|
||||
func SystemRoutes() []map[string]any {
|
||||
return []map[string]any{
|
||||
{"method": "GET", "path": "/_part/*", "kind": "system"},
|
||||
{"method": "GET|POST", "path": "/_form/*", "kind": "system"},
|
||||
}
|
||||
}
|
||||
|
||||
// InspectBag builds {declared, system} from an app bag and/or scanned files.
|
||||
func InspectBag(app map[string]any, declared []ArtifactFile) map[string]any {
|
||||
var dec []map[string]any
|
||||
for _, a := range declared {
|
||||
dec = append(dec, map[string]any{
|
||||
"method": a.Method,
|
||||
"path": a.Route,
|
||||
"kind": a.Kind,
|
||||
"file": a.Rel,
|
||||
})
|
||||
}
|
||||
if app != nil {
|
||||
if routes, ok := app["routes"].(map[string]any); ok {
|
||||
for path := range routes {
|
||||
dec = append(dec, map[string]any{"method": "GET", "path": path, "kind": "declared"})
|
||||
}
|
||||
}
|
||||
if inv, ok := app["invoke_routes"].(map[string]any); ok {
|
||||
for key := range inv {
|
||||
dec = append(dec, map[string]any{"path": key, "kind": "endpoint"})
|
||||
}
|
||||
}
|
||||
if art, ok := app["artifact_routes"].(map[string]any); ok {
|
||||
for key, v := range art {
|
||||
m, _ := v.(map[string]any)
|
||||
dec = append(dec, map[string]any{
|
||||
"path": key,
|
||||
"kind": "endpoint",
|
||||
"method": fmt.Sprint(m["method"]),
|
||||
"file": fmt.Sprint(m["file"]),
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
return map[string]any{
|
||||
"declared": dec,
|
||||
"system": SystemRoutes(),
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
package artifact
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestMetaType(t *testing.T) {
|
||||
if MetaType(map[string]string{"type": "web"}) != "web" {
|
||||
t.Fatal("en web")
|
||||
}
|
||||
if MetaType(map[string]string{"类型": "网页"}) != "web" {
|
||||
t.Fatal("zh web")
|
||||
}
|
||||
if MetaType(map[string]string{"type": "endpoint"}) != "endpoint" {
|
||||
t.Fatal("endpoint")
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseFrontmatter(t *testing.T) {
|
||||
src := "---\ntype: web\nroute: /x\n---\n\n# Hi\n"
|
||||
meta, body := ParseFrontmatter(src)
|
||||
if meta["type"] != "web" || meta["route"] != "/x" {
|
||||
t.Fatalf("meta=%v", meta)
|
||||
}
|
||||
if body == "" || body[0] != '#' {
|
||||
t.Fatalf("body=%q", body)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRenderNodes(t *testing.T) {
|
||||
nodes := map[string]any{
|
||||
"type": []any{"title", "text"},
|
||||
"slot": []any{"title", "lede"},
|
||||
"value": []any{"Marqdo", "Hello"},
|
||||
"attrs": []any{"", ""},
|
||||
"style": []any{"title", "lede"},
|
||||
}
|
||||
html, err := RenderNodes(nodes)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !contains(html, "Marqdo") || !contains(html, "mq-view") {
|
||||
t.Fatalf("html=%s", html)
|
||||
}
|
||||
}
|
||||
|
||||
func contains(s, sub string) bool {
|
||||
return len(s) >= len(sub) && (s == sub || len(sub) == 0 ||
|
||||
(func() bool {
|
||||
for i := 0; i+len(sub) <= len(s); i++ {
|
||||
if s[i:i+len(sub)] == sub {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
})())
|
||||
}
|
||||
@@ -117,11 +117,17 @@ func (st *state) handleOIDCStart(w http.ResponseWriter, r *http.Request) {
|
||||
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
redirectURI, err := st.oidc.ResolveRedirectURI(r)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
session.SetValue(sid, "oidc_state", state)
|
||||
session.SetValue(sid, "oidc_verifier", verifier)
|
||||
session.SetValue(sid, "oidc_redirect_uri", redirectURI)
|
||||
next := loginDest(r, st.oidcFallbackNext(r))
|
||||
session.SetValue(sid, "oidc_next", next)
|
||||
loc, err := st.oidc.AuthorizeRedirect(state, oidc.PKCEChallengeS256(verifier))
|
||||
loc, err := st.oidc.AuthorizeRedirect(state, oidc.PKCEChallengeS256(verifier), redirectURI)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||
return
|
||||
@@ -164,6 +170,15 @@ func (st *state) handleOIDCCallback(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
verifier, _ := session.GetValue(sid, "oidc_verifier")
|
||||
verifierS, _ := verifier.(string)
|
||||
redirVal, _ := session.GetValue(sid, "oidc_redirect_uri")
|
||||
redirectURI, _ := redirVal.(string)
|
||||
if redirectURI == "" && !st.oidc.IsPathRedirect() {
|
||||
redirectURI = strings.TrimSpace(st.oidc.RedirectURI)
|
||||
}
|
||||
if redirectURI == "" {
|
||||
http.Error(w, "oidc: missing redirect_uri in session", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
nextVal, _ := session.GetValue(sid, "oidc_next")
|
||||
next, _ := nextVal.(string)
|
||||
if next == "" {
|
||||
@@ -171,9 +186,10 @@ func (st *state) handleOIDCCallback(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
session.DelKey(sid, "oidc_state")
|
||||
session.DelKey(sid, "oidc_verifier")
|
||||
session.DelKey(sid, "oidc_redirect_uri")
|
||||
session.DelKey(sid, "oidc_next")
|
||||
|
||||
tok, err := st.oidc.ExchangeCode(oidcClient(), code, verifierS)
|
||||
tok, err := st.oidc.ExchangeCode(oidcClient(), code, verifierS, redirectURI)
|
||||
if err != nil {
|
||||
http.Error(w, "oidc token exchange: "+err.Error(), http.StatusBadGateway)
|
||||
return
|
||||
|
||||
@@ -122,6 +122,7 @@ func NewHandler(appBag map[string]any, entryDir string) (http.Handler, error) {
|
||||
galleryRoutes: routeMapOf(appBag, "gallery_routes"),
|
||||
proxyRoutes: routeMapOf(appBag, "proxy_routes"),
|
||||
invokeRoutes: routeMapOf(appBag, "invoke_routes"),
|
||||
artifactRoutes: routeMapOf(appBag, "artifact_routes"),
|
||||
auth: authCfg,
|
||||
oidc: oidc.ParseConfig(appBag),
|
||||
tenant: tenant.FromBag(appBag),
|
||||
@@ -238,6 +239,7 @@ func NewHandler(appBag map[string]any, entryDir string) (http.Handler, error) {
|
||||
st.mountGalleryRoutes(mux)
|
||||
st.mountProxyRoutes(mux)
|
||||
st.mountInvokeRoutes(mux)
|
||||
st.mountArtifactRoutes(mux)
|
||||
st.mountAuthRoutes(mux)
|
||||
|
||||
mux.HandleFunc("/{path...}", st.handleNotFound)
|
||||
@@ -245,6 +247,7 @@ func NewHandler(appBag map[string]any, entryDir string) (http.Handler, error) {
|
||||
handler := withMiddleware(mux, mw)
|
||||
handler = withRBAC(handler, authCfg.gates, authCfg.loginPath)
|
||||
handler = withTenant(handler, st.tenant)
|
||||
handler = withStripTrailingSlash(handler)
|
||||
return handler, nil
|
||||
}
|
||||
|
||||
@@ -307,6 +310,7 @@ type state struct {
|
||||
galleryRoutes map[string]any
|
||||
proxyRoutes map[string]any
|
||||
invokeRoutes map[string]any
|
||||
artifactRoutes map[string]any
|
||||
auth authConfig
|
||||
oidc oidc.Config
|
||||
tenant tenant.Config
|
||||
@@ -638,6 +642,21 @@ func withTenant(next http.Handler, cfg tenant.Config) http.Handler {
|
||||
})
|
||||
}
|
||||
|
||||
// withStripTrailingSlash rewrites request URLs so that paths like /post/foo/
|
||||
// are routed as /post/foo. The root path "/" is left unchanged.
|
||||
func withStripTrailingSlash(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
path := r.URL.Path
|
||||
if len(path) > 1 && path[len(path)-1] == '/' {
|
||||
r.URL.Path = strings.TrimRight(path, "/")
|
||||
if r.URL.Path == "" {
|
||||
r.URL.Path = "/"
|
||||
}
|
||||
}
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
func withMiddleware(next http.Handler, mw middleware.Config) http.Handler {
|
||||
h := next
|
||||
if mw.BodyLimit != nil && *mw.BodyLimit > 0 {
|
||||
|
||||
@@ -152,3 +152,65 @@ func TestNewHandlerIconsRedirectSitemap404(t *testing.T) {
|
||||
t.Fatalf("home missing icon link: %s", rec.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestTrailingSlashNormalization(t *testing.T) {
|
||||
page := map[string]any{"title": "Home", "intro": "<p>home</p>"}
|
||||
a := app.New(map[string]any{"page": page})
|
||||
about := map[string]any{"title": "About", "intro": "<h1>About</h1>", "_route": "/about"}
|
||||
var err error
|
||||
a, err = app.Route(a, "/about", about)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Dynamic route: /post/{slug}
|
||||
post := map[string]any{"title": "Post", "intro": "<h1>Post</h1>", "_route": "/post/{slug}"}
|
||||
a, err = app.Route(a, "/post/{slug}", post)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
h, err := httpx.NewHandler(a, "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Static route: /about should work with and without trailing slash.
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/about", nil))
|
||||
if rec.Code != 200 || !strings.Contains(rec.Body.String(), "About") {
|
||||
t.Fatalf("about (no slash) code=%d body=%q", rec.Code, rec.Body.String())
|
||||
}
|
||||
|
||||
rec = httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/about/", nil))
|
||||
if rec.Code != 200 || !strings.Contains(rec.Body.String(), "About") {
|
||||
t.Fatalf("about (trailing slash) code=%d body=%q", rec.Code, rec.Body.String())
|
||||
}
|
||||
|
||||
// Dynamic route: /post/hello should work with and without trailing slash.
|
||||
rec = httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/post/hello", nil))
|
||||
if rec.Code != 200 {
|
||||
t.Fatalf("post/hello (no slash) code=%d body=%q", rec.Code, rec.Body.String())
|
||||
}
|
||||
|
||||
rec = httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/post/hello/", nil))
|
||||
if rec.Code != 200 {
|
||||
t.Fatalf("post/hello (trailing slash) code=%d body=%q", rec.Code, rec.Body.String())
|
||||
}
|
||||
|
||||
// Root path "/" should still work.
|
||||
rec = httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/", nil))
|
||||
if rec.Code != 200 || !strings.Contains(rec.Body.String(), "Home") {
|
||||
t.Fatalf("root code=%d body=%q", rec.Code, rec.Body.String())
|
||||
}
|
||||
|
||||
// Query string should be preserved after trailing slash strip.
|
||||
rec = httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/about/?foo=bar", nil))
|
||||
if rec.Code != 200 || !strings.Contains(rec.Body.String(), "About") {
|
||||
t.Fatalf("about (trailing slash + query) code=%d body=%q", rec.Code, rec.Body.String())
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,142 @@
|
||||
package httpx
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/marqdo/marqdo/plugins/web/internal/plugin"
|
||||
"github.com/marqdo/marqdo/plugins/web/internal/session"
|
||||
)
|
||||
|
||||
// artifactRoute is one Endpoint file registered by web.serve (ADR 0007).
|
||||
type artifactRoute struct {
|
||||
Method string
|
||||
Path string
|
||||
File string
|
||||
Request string // json | form | none | query
|
||||
Response string // json | text | html
|
||||
Auth string // role name or "" / "none"
|
||||
}
|
||||
|
||||
func parseArtifactRoute(v any) artifactRoute {
|
||||
m, _ := v.(map[string]any)
|
||||
return artifactRoute{
|
||||
Method: strings.ToUpper(strOpt(m, "method", "GET")),
|
||||
Path: strOpt(m, "path", ""),
|
||||
File: strOpt(m, "file", ""),
|
||||
Request: strings.ToLower(strOpt(m, "request", "json")),
|
||||
Response: strings.ToLower(strOpt(m, "response", "json")),
|
||||
Auth: strOpt(m, "auth", ""),
|
||||
}
|
||||
}
|
||||
|
||||
func (st *state) mountArtifactRoutes(mux *http.ServeMux) {
|
||||
for _, key := range sortedKeys(st.artifactRoutes) {
|
||||
cfg := parseArtifactRoute(st.artifactRoutes[key])
|
||||
if cfg.File == "" || cfg.Path == "" {
|
||||
continue
|
||||
}
|
||||
method := cfg.Method
|
||||
if method == "" {
|
||||
method = http.MethodGet
|
||||
}
|
||||
pattern := method + " " + goMuxPattern(cfg.Path)
|
||||
routeCfg := cfg
|
||||
mux.HandleFunc(pattern, func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != method {
|
||||
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
|
||||
return
|
||||
}
|
||||
st.handleArtifact(w, r, routeCfg)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func (st *state) handleArtifact(w http.ResponseWriter, r *http.Request, route artifactRoute) {
|
||||
if route.Auth != "" && !strings.EqualFold(route.Auth, "none") {
|
||||
cookie := r.Header.Get("Cookie")
|
||||
user := session.UsernameFromCookie(cookie)
|
||||
if user == "" {
|
||||
jsonErr(w, http.StatusUnauthorized, "auth required")
|
||||
return
|
||||
}
|
||||
role := session.RoleFromCookie(cookie)
|
||||
if role != "" && role != route.Auth && role != "admin" && role != "superadmin" {
|
||||
jsonErr(w, http.StatusForbidden, "forbidden")
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
payload := map[string]any{}
|
||||
switch route.Request {
|
||||
case "none":
|
||||
payload = queryToMap(r.URL.RawQuery)
|
||||
case "query":
|
||||
payload = queryToMap(r.URL.RawQuery)
|
||||
case "form":
|
||||
body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 8<<20))
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusBadRequest, fmt.Sprintf("body: %v", err))
|
||||
return
|
||||
}
|
||||
payload = formToMap(body)
|
||||
default: // json
|
||||
body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 8<<20))
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusBadRequest, fmt.Sprintf("body: %v", err))
|
||||
return
|
||||
}
|
||||
q := queryToMap(r.URL.RawQuery)
|
||||
if len(body) == 0 {
|
||||
payload = q
|
||||
} else {
|
||||
var raw map[string]any
|
||||
if err := json.Unmarshal(body, &raw); err != nil {
|
||||
jsonErr(w, http.StatusBadRequest, fmt.Sprintf("json: %v", err))
|
||||
return
|
||||
}
|
||||
if raw == nil {
|
||||
payload = q
|
||||
} else {
|
||||
for k, v := range q {
|
||||
if _, exists := raw[k]; !exists {
|
||||
raw[k] = v
|
||||
}
|
||||
}
|
||||
payload = raw
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, name := range pathParamNames(route.Path) {
|
||||
if v := r.PathValue(name); v != "" {
|
||||
payload[name] = v
|
||||
}
|
||||
}
|
||||
|
||||
value, err := plugin.RunArtifact(route.File, payload)
|
||||
if err != nil {
|
||||
jsonErr(w, http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
switch route.Response {
|
||||
case "text":
|
||||
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = io.WriteString(w, fmt.Sprint(value))
|
||||
case "html", "text/html":
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = io.WriteString(w, fmt.Sprint(value))
|
||||
default:
|
||||
code, body, err := jsonWrapInvokeValue(value)
|
||||
if err != nil {
|
||||
jsonErr(w, code, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, code, body)
|
||||
}
|
||||
}
|
||||
@@ -20,8 +20,11 @@ type Config struct {
|
||||
ClientID string
|
||||
ClientSecret string
|
||||
RedirectURI string
|
||||
Scopes string
|
||||
CallbackPath string
|
||||
// RedirectOrigins is an optional comma-separated allowlist of origins
|
||||
// (e.g. "https://qdqc.com,https://www.qdqc.com") used when RedirectURI is a path.
|
||||
RedirectOrigins string
|
||||
Scopes string
|
||||
CallbackPath string
|
||||
// AuthorizeURL / TokenURL / UserInfoURL override discovery when set.
|
||||
AuthorizeURL string
|
||||
TokenURL string
|
||||
@@ -36,11 +39,35 @@ func (c Config) Enabled() bool {
|
||||
(strings.TrimSpace(c.Issuer) != "" || strings.TrimSpace(c.AuthorizeURL) != "")
|
||||
}
|
||||
|
||||
// IsPathRedirect reports whether RedirectURI is a path (resolved per request Host).
|
||||
func (c Config) IsPathRedirect() bool {
|
||||
s := strings.TrimSpace(c.RedirectURI)
|
||||
if s == "" {
|
||||
return false
|
||||
}
|
||||
if strings.HasPrefix(s, "/") {
|
||||
return true
|
||||
}
|
||||
u, err := url.Parse(s)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
// scheme-relative or bare path without host
|
||||
return u.Scheme == "" && u.Host == "" && u.Path != ""
|
||||
}
|
||||
|
||||
// Callback reports the local path (default /oidc/callback).
|
||||
func (c Config) Callback() string {
|
||||
p := strings.TrimSpace(c.CallbackPath)
|
||||
if p == "" {
|
||||
if u, err := url.Parse(c.RedirectURI); err == nil && u.Path != "" {
|
||||
ru := strings.TrimSpace(c.RedirectURI)
|
||||
if c.IsPathRedirect() {
|
||||
if !strings.HasPrefix(ru, "/") {
|
||||
ru = "/" + ru
|
||||
}
|
||||
return ru
|
||||
}
|
||||
if u, err := url.Parse(ru); err == nil && u.Path != "" {
|
||||
return u.Path
|
||||
}
|
||||
return "/oidc/callback"
|
||||
@@ -51,6 +78,91 @@ func (c Config) Callback() string {
|
||||
return p
|
||||
}
|
||||
|
||||
// PublicOrigin derives scheme://host from a login/callback request.
|
||||
// Prefers X-Forwarded-Proto + X-Forwarded-Host (first value), else Host + TLS.
|
||||
func PublicOrigin(r *http.Request) (string, error) {
|
||||
if r == nil {
|
||||
return "", fmt.Errorf("oidc: nil request")
|
||||
}
|
||||
scheme := strings.TrimSpace(r.Header.Get("X-Forwarded-Proto"))
|
||||
if i := strings.IndexByte(scheme, ','); i >= 0 {
|
||||
scheme = strings.TrimSpace(scheme[:i])
|
||||
}
|
||||
if scheme == "" {
|
||||
if r.TLS != nil {
|
||||
scheme = "https"
|
||||
} else {
|
||||
scheme = "http"
|
||||
}
|
||||
}
|
||||
host := strings.TrimSpace(r.Header.Get("X-Forwarded-Host"))
|
||||
if i := strings.IndexByte(host, ','); i >= 0 {
|
||||
host = strings.TrimSpace(host[:i])
|
||||
}
|
||||
if host == "" {
|
||||
host = strings.TrimSpace(r.Host)
|
||||
}
|
||||
if host == "" {
|
||||
return "", fmt.Errorf("oidc: empty Host")
|
||||
}
|
||||
return scheme + "://" + host, nil
|
||||
}
|
||||
|
||||
// ResolveRedirectURI returns the absolute redirect_uri for this request.
|
||||
// Absolute RedirectURI is returned unchanged; path RedirectURI is joined to the request origin.
|
||||
func (c Config) ResolveRedirectURI(r *http.Request) (string, error) {
|
||||
ru := strings.TrimSpace(c.RedirectURI)
|
||||
if ru == "" {
|
||||
return "", fmt.Errorf("oidc: empty redirect_uri")
|
||||
}
|
||||
if !c.IsPathRedirect() {
|
||||
return ru, nil
|
||||
}
|
||||
origin, err := PublicOrigin(r)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
path := ru
|
||||
if !strings.HasPrefix(path, "/") {
|
||||
path = "/" + path
|
||||
}
|
||||
abs := origin + path
|
||||
if err := c.checkOriginAllowlist(origin); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return abs, nil
|
||||
}
|
||||
|
||||
// AllowedOrigins returns trimmed non-empty origins from RedirectOrigins.
|
||||
func (c Config) AllowedOrigins() []string {
|
||||
raw := strings.TrimSpace(c.RedirectOrigins)
|
||||
if raw == "" {
|
||||
return nil
|
||||
}
|
||||
var out []string
|
||||
for _, p := range strings.Split(raw, ",") {
|
||||
p = strings.TrimRight(strings.TrimSpace(p), "/")
|
||||
if p != "" {
|
||||
out = append(out, p)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (c Config) checkOriginAllowlist(origin string) error {
|
||||
allowed := c.AllowedOrigins()
|
||||
if len(allowed) == 0 {
|
||||
return nil
|
||||
}
|
||||
origin = strings.TrimRight(strings.TrimSpace(origin), "/")
|
||||
for _, a := range allowed {
|
||||
if strings.EqualFold(a, origin) {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("oidc: origin %q not in redirect_origins", origin)
|
||||
}
|
||||
|
||||
type discoveryDoc struct {
|
||||
AuthorizationEndpoint string `json:"authorization_endpoint"`
|
||||
TokenEndpoint string `json:"token_endpoint"`
|
||||
@@ -126,14 +238,19 @@ func PKCEChallengeS256(verifier string) string {
|
||||
}
|
||||
|
||||
// AuthorizeRedirect builds the browser redirect to the IdP authorize endpoint.
|
||||
func (c Config) AuthorizeRedirect(state, codeChallenge string) (string, error) {
|
||||
// redirectURI must be the absolute URI used for this login (same value later for token exchange).
|
||||
func (c Config) AuthorizeRedirect(state, codeChallenge, redirectURI string) (string, error) {
|
||||
if c.AuthorizeURL == "" {
|
||||
return "", fmt.Errorf("oidc: authorize URL not resolved")
|
||||
}
|
||||
redirectURI = strings.TrimSpace(redirectURI)
|
||||
if redirectURI == "" {
|
||||
return "", fmt.Errorf("oidc: empty redirect_uri")
|
||||
}
|
||||
q := url.Values{}
|
||||
q.Set("response_type", "code")
|
||||
q.Set("client_id", c.ClientID)
|
||||
q.Set("redirect_uri", c.RedirectURI)
|
||||
q.Set("redirect_uri", redirectURI)
|
||||
q.Set("scope", c.ScopesCSV())
|
||||
q.Set("state", state)
|
||||
if codeChallenge != "" {
|
||||
@@ -160,17 +277,22 @@ type TokenResponse struct {
|
||||
}
|
||||
|
||||
// ExchangeCode swaps authorization code for tokens (client_secret_post).
|
||||
func (c Config) ExchangeCode(client *http.Client, code, codeVerifier string) (*TokenResponse, error) {
|
||||
// redirectURI must match the absolute URI sent in AuthorizeRedirect.
|
||||
func (c Config) ExchangeCode(client *http.Client, code, codeVerifier, redirectURI string) (*TokenResponse, error) {
|
||||
if c.TokenURL == "" {
|
||||
return nil, fmt.Errorf("oidc: token URL not resolved")
|
||||
}
|
||||
redirectURI = strings.TrimSpace(redirectURI)
|
||||
if redirectURI == "" {
|
||||
return nil, fmt.Errorf("oidc: empty redirect_uri")
|
||||
}
|
||||
if client == nil {
|
||||
client = &http.Client{Timeout: 15 * time.Second}
|
||||
}
|
||||
form := url.Values{}
|
||||
form.Set("grant_type", "authorization_code")
|
||||
form.Set("code", code)
|
||||
form.Set("redirect_uri", c.RedirectURI)
|
||||
form.Set("redirect_uri", redirectURI)
|
||||
form.Set("client_id", c.ClientID)
|
||||
form.Set("client_secret", c.ClientSecret)
|
||||
if codeVerifier != "" {
|
||||
@@ -313,15 +435,16 @@ func ParseConfig(m map[string]any) Config {
|
||||
m = nested
|
||||
}
|
||||
c := Config{
|
||||
Issuer: firstStr(m, "issuer", "发行方", "issuer_url"),
|
||||
ClientID: firstStr(m, "client_id", "客户端编号", "客户端id", "clientId"),
|
||||
ClientSecret: firstStr(m, "client_secret", "客户端密钥", "clientSecret"),
|
||||
RedirectURI: firstStr(m, "redirect_uri", "回调", "回调地址", "redirect"),
|
||||
Scopes: firstStr(m, "scopes", "范围", "scope"),
|
||||
CallbackPath: firstStr(m, "callback_path", "回调路径"),
|
||||
AuthorizeURL: firstStr(m, "authorize_url", "授权地址"),
|
||||
TokenURL: firstStr(m, "token_url", "令牌地址"),
|
||||
UserInfoURL: firstStr(m, "userinfo_url", "用户信息地址"),
|
||||
Issuer: firstStr(m, "issuer", "发行方", "issuer_url"),
|
||||
ClientID: firstStr(m, "client_id", "客户端编号", "客户端id", "clientId"),
|
||||
ClientSecret: firstStr(m, "client_secret", "客户端密钥", "clientSecret"),
|
||||
RedirectURI: firstStr(m, "redirect_uri", "回调", "回调地址", "redirect"),
|
||||
RedirectOrigins: firstStr(m, "redirect_origins", "回调来源", "redirect_origin"),
|
||||
Scopes: firstStr(m, "scopes", "范围", "scope"),
|
||||
CallbackPath: firstStr(m, "callback_path", "回调路径"),
|
||||
AuthorizeURL: firstStr(m, "authorize_url", "授权地址"),
|
||||
TokenURL: firstStr(m, "token_url", "令牌地址"),
|
||||
UserInfoURL: firstStr(m, "userinfo_url", "用户信息地址"),
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
package oidc_test
|
||||
|
||||
import (
|
||||
"crypto/tls"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/marqdo/marqdo/plugins/web/internal/oidc"
|
||||
@@ -46,4 +50,138 @@ func TestPKCEAndParse(t *testing.T) {
|
||||
if c.Callback() != "/oidc/callback" {
|
||||
t.Fatalf("callback=%s", c.Callback())
|
||||
}
|
||||
if c.IsPathRedirect() {
|
||||
t.Fatal("absolute redirect should not be path mode")
|
||||
}
|
||||
}
|
||||
|
||||
func TestPathRedirectParseAndCallback(t *testing.T) {
|
||||
c := oidc.ParseConfig(map[string]any{
|
||||
"issuer": "https://id.example.com",
|
||||
"client_id": "app_x",
|
||||
"client_secret": "secret",
|
||||
"redirect_uri": "/oidc/callback",
|
||||
"redirect_origins": "https://qdqc.com,https://www.qdqc.com",
|
||||
})
|
||||
if !c.Enabled() {
|
||||
t.Fatal("expected enabled")
|
||||
}
|
||||
if !c.IsPathRedirect() {
|
||||
t.Fatal("expected path mode")
|
||||
}
|
||||
if c.Callback() != "/oidc/callback" {
|
||||
t.Fatalf("callback=%s", c.Callback())
|
||||
}
|
||||
origins := c.AllowedOrigins()
|
||||
if len(origins) != 2 || origins[0] != "https://qdqc.com" {
|
||||
t.Fatalf("origins=%v", origins)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveRedirectURIAbsolute(t *testing.T) {
|
||||
c := oidc.Config{RedirectURI: "https://qdqc.com/oidc/callback"}
|
||||
r, _ := http.NewRequest(http.MethodGet, "http://www.qdqc.com/oidc/login", nil)
|
||||
r.Host = "www.qdqc.com"
|
||||
got, err := c.ResolveRedirectURI(r)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != "https://qdqc.com/oidc/callback" {
|
||||
t.Fatalf("got=%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveRedirectURIPathFromHost(t *testing.T) {
|
||||
c := oidc.Config{RedirectURI: "/oidc/callback"}
|
||||
r, _ := http.NewRequest(http.MethodGet, "http://qdqc.com/oidc/login", nil)
|
||||
r.Host = "qdqc.com"
|
||||
got, err := c.ResolveRedirectURI(r)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != "http://qdqc.com/oidc/callback" {
|
||||
t.Fatalf("got=%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveRedirectURIPathTLS(t *testing.T) {
|
||||
c := oidc.Config{RedirectURI: "/oidc/callback"}
|
||||
r, _ := http.NewRequest(http.MethodGet, "https://qdqc.com/oidc/login", nil)
|
||||
r.Host = "qdqc.com"
|
||||
r.TLS = &tls.ConnectionState{}
|
||||
got, err := c.ResolveRedirectURI(r)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != "https://qdqc.com/oidc/callback" {
|
||||
t.Fatalf("got=%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveRedirectURIXForwarded(t *testing.T) {
|
||||
c := oidc.Config{RedirectURI: "/oidc/callback"}
|
||||
r, _ := http.NewRequest(http.MethodGet, "http://127.0.0.1:18085/oidc/login", nil)
|
||||
r.Host = "127.0.0.1:18085"
|
||||
r.Header.Set("X-Forwarded-Proto", "https")
|
||||
r.Header.Set("X-Forwarded-Host", "www.qdqc.com, other.example")
|
||||
got, err := c.ResolveRedirectURI(r)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != "https://www.qdqc.com/oidc/callback" {
|
||||
t.Fatalf("got=%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveRedirectURIAllowlistReject(t *testing.T) {
|
||||
c := oidc.Config{
|
||||
RedirectURI: "/oidc/callback",
|
||||
RedirectOrigins: "https://qdqc.com,https://www.qdqc.com",
|
||||
}
|
||||
r, _ := http.NewRequest(http.MethodGet, "https://evil.example/oidc/login", nil)
|
||||
r.Host = "evil.example"
|
||||
r.Header.Set("X-Forwarded-Proto", "https")
|
||||
_, err := c.ResolveRedirectURI(r)
|
||||
if err == nil || !strings.Contains(err.Error(), "redirect_origins") {
|
||||
t.Fatalf("expected allowlist error, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveRedirectURIAllowlistOK(t *testing.T) {
|
||||
c := oidc.Config{
|
||||
RedirectURI: "/oidc/callback",
|
||||
RedirectOrigins: "https://qdqc.com,https://www.qdqc.com",
|
||||
}
|
||||
r, _ := http.NewRequest(http.MethodGet, "https://www.qdqc.com/oidc/login", nil)
|
||||
r.Host = "www.qdqc.com"
|
||||
r.Header.Set("X-Forwarded-Proto", "https")
|
||||
got, err := c.ResolveRedirectURI(r)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != "https://www.qdqc.com/oidc/callback" {
|
||||
t.Fatalf("got=%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAuthorizeRedirectUsesExplicitURI(t *testing.T) {
|
||||
c := oidc.Config{
|
||||
ClientID: "app_x",
|
||||
AuthorizeURL: "https://id.example.com/oauth2/authorize",
|
||||
RedirectURI: "/oidc/callback",
|
||||
}
|
||||
loc, err := c.AuthorizeRedirect("st", "ch", "https://www.qdqc.com/oidc/callback")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
u, err := url.Parse(loc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := u.Query().Get("redirect_uri"); got != "https://www.qdqc.com/oidc/callback" {
|
||||
t.Fatalf("redirect_uri=%s", got)
|
||||
}
|
||||
if u.Query().Get("state") != "st" {
|
||||
t.Fatal("state")
|
||||
}
|
||||
}
|
||||
@@ -49,3 +49,25 @@ func CallLibPath(path string, args map[string]any) (any, error) {
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
|
||||
// RunArtifact loads and runs a Document/Endpoint `.mq.md` (ADR 0007).
|
||||
// Request fields are passed as Metadata Binding overlays (`arg` / `--bind`).
|
||||
func RunArtifact(path string, args map[string]any) (any, error) {
|
||||
payload := map[string]any{"path": path}
|
||||
if len(args) > 0 {
|
||||
payload["args"] = args
|
||||
}
|
||||
raw, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out, err := Query("run_artifact", string(raw))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var v any
|
||||
if err := json.Unmarshal([]byte(out), &v); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
Loaded 100 of 238 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user