feat(llm): semantic ask/stream/collect surface (005)
Lift ext/ai/llm from OpenAI HTTP client to an intelligence primitive; keep complete as compat. Add unified ai-stack roadmap linking agent + llm. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
1 parent
a9ee858f53
commit
1f8e61f3bf
11 files changed
+1581
-115
No files matched your search
@@ -201,7 +201,7 @@ import clock:lib/time.mq.md
|
||||
- Instance methods stay `` > `obj`.method `` (backticks on the receiver only).
|
||||
- `lib/…` resolves via `MARQDO_LIB`, cwd `lib/`, or `lib/` next to the `marqdo` binary.
|
||||
- Design: [module-namespace.md](../../doc/design/module-namespace.md).
|
||||
- Official optional extensions (`ext/`, not stdlib): install with `marqdo ext add llm|agent|web|quantum|linalg` (see `doc/design/ext-cli.md`). `ext/llm` — chat; **`ext/agent`** — document-driven agents (see **ext/agent** below); **`ext/web`** — dynamic sites (see below); **`ext/quantum`** — circuits + Q7/Q8 (see below); **`ext/linalg`** — formula-first linear algebra (see below).
|
||||
- Official optional extensions (`ext/`, not stdlib): install with `marqdo ext add llm|agent|web|quantum|linalg` (see `doc/design/ext-cli.md`). **`ext/llm`** — intelligence primitive (`ask` / `stream` / `collect`; `complete` is compat); **`ext/agent`** — document-driven agents (see **ext/agent** below); **`ext/web`** — dynamic sites (see below); **`ext/quantum`** — circuits + Q7/Q8 (see below); **`ext/linalg`** — formula-first linear algebra (see below).
|
||||
- Native plugins: `lib/plugin` — `## load` / `unload` / `list`. C ABI: `include/marqdo_abi.h`.
|
||||
- Prefer `table.put` / `表.改` for list/map element updates; keep `json` for parse/stringify/quote (see `doc/design/stdlib-table.md`).
|
||||
- Builtins (no import): `print`/`打印`, `input`/`输入`, `len`/`长度`, `str`/`文本`, `int`/`整数`; literals `True`/`真`, `False`/`假`, `None`/`空`; logic `and`/`且`, `or`/`或`, `not`/`非`.
|
||||
|
||||
+54
-52
@@ -2,13 +2,20 @@
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Status | Accepted (v0.1.x) |
|
||||
| Date | 2026-08-06 |
|
||||
| Related | [objects.md](objects.md) · [ext-abi.md](ext-abi.md) · [ext-agent.md](ext-agent.md) · [ext-cli.md](ext-cli.md) |
|
||||
| Status | **Accepted · v0.2 semantic**(005:Intelligence Primitive) |
|
||||
| Date | 2026-09-24 |
|
||||
| Related | [objects.md](objects.md) · [ext-agent.md](ext-agent.md) · [ext-cli.md](ext-cli.md) · [ai-stack.md](../roadmap/ai-stack.md) · [doc/next/005.md](../next/005.md) |
|
||||
|
||||
## Scope
|
||||
|
||||
`ext/ai/` holds **official LLM + agent** extensions. Import paths:
|
||||
`ext/ai/llm` is Marqdo’s **intelligence primitive**, not an OpenAI SDK wrapper.
|
||||
|
||||
```text
|
||||
Author surface: ask / stream / collect / create
|
||||
Internal: complete (compat) + openai-compatible HTTP
|
||||
```
|
||||
|
||||
Import:
|
||||
|
||||
```markdown
|
||||
---
|
||||
@@ -16,55 +23,48 @@ import llm:ext/ai/llm.mq.md
|
||||
---
|
||||
```
|
||||
|
||||
Resolution: `MARQDO_EXT/ai/…`, repo `ext/ai/…`, cwd `ext/ai/…` (see [ext-cli.md](ext-cli.md)).
|
||||
|
||||
Other official ext: [ext-agent.md](ext-agent.md) / [ext-agent-plan.md](ext-agent-plan.md) (**document-driven agent** — step with default writeback / plan workbook; thin `TOOL:` loop superseded). Installer: [ext-cli.md](ext-cli.md). Native plugins: [ext-abi.md](ext-abi.md).
|
||||
|
||||
Streaming: `complete stream=True` returns an event list (`reasoning` / `delta` / `done` / `error`); `echo=True` prints thinking (`reasoning_content`) and answer tokens as they arrive. See [agent-streaming.md](../roadmap/agent-streaming.md). Default (`stream=False`) is unchanged (answer string).
|
||||
|
||||
**Note:** Prefer `marqdo ext add llm`. Import `import llm:ext/ai/llm.mq.md` from repo or install root.
|
||||
|
||||
## Platform prerequisites
|
||||
|
||||
| Capability | Surface |
|
||||
|------------|---------|
|
||||
| HTTPS + headers | `lib/net` → `http_post` / `headers=` map |
|
||||
| Dotenv | `## load_env` in ext (or `load_dotenv` / `lib/sys`) |
|
||||
| JSON quote | `lib/json` → `quote` |
|
||||
|
||||
## `ext/ai/llm` / `ext/ai/大模型` (object handles)
|
||||
## Public API
|
||||
|
||||
| English | Chinese | Role |
|
||||
|---------|---------|------|
|
||||
| `## load_env` | `## 加载环境` | Free function: load `.env` |
|
||||
| `## stream_result` | `## 流式结果` | Collapse event list → final text |
|
||||
| `# llm` | `# 大模型` | Object ctor → handle map |
|
||||
| `## complete` / `## chat` | `## 运行` / `## 聊天` | Methods; optional `stream` / `流式`, `echo` / `打印增量` |
|
||||
| `## load_env` | `## 加载环境` | Load `.env` |
|
||||
| `## create` | `## 创建` | Factory → LLM handle |
|
||||
| `## ask` (module) | `## 提问` | Convenience → **text** |
|
||||
| `## collect` | `## 收集` | Events → text |
|
||||
| `## stream_result` | `## 流式结果` | Alias of `collect` |
|
||||
| `# llm` | `# 大模型` | Handle ctor |
|
||||
| `## ask` (method) | `## 提问` | → **LLMResult** map |
|
||||
| `## stream` | `## 流式` | → event list |
|
||||
| `## complete` / `## chat` | `## 运行` / `## 聊天` | Compat / transport primitive |
|
||||
|
||||
### Preferred
|
||||
|
||||
```markdown
|
||||
---
|
||||
import llm:ext/ai/llm.mq.md
|
||||
---
|
||||
**answer = > llm.ask prompt="What is Marqdo?"**
|
||||
*answer*
|
||||
|
||||
# main
|
||||
**model = > llm.create**
|
||||
**result = > `model`.ask prompt=`prompt`**
|
||||
*[text](result)*
|
||||
|
||||
> load_env
|
||||
|
||||
*`model` = > llm *
|
||||
*`reply` = > `model`.complete prompt=Say hi in one word *
|
||||
> print text=`reply`
|
||||
|
||||
*`events` = > `model`.complete prompt=Say hi stream=True *
|
||||
- [`ev`](`events`)
|
||||
*`t` = > json.get value=`ev` key=type *
|
||||
1. `t` == "delta"
|
||||
*`chunk` = > json.get value=`ev` key=text *
|
||||
> print text=`chunk`
|
||||
2. *
|
||||
*`_` = 1*
|
||||
**events = > `model`.stream prompt=`prompt`**
|
||||
**text = > llm.collect events=`events`**
|
||||
```
|
||||
|
||||
Environment (create a **project-local** `.env`; do not commit secrets):
|
||||
### LLMResult (minimal)
|
||||
|
||||
| Field | Meaning |
|
||||
|-------|---------|
|
||||
| `text` | Answer string |
|
||||
| `model` | Model id |
|
||||
| `finish` | Finish reason (`stop`, …) |
|
||||
| `backend` | e.g. `openai-compatible` |
|
||||
|
||||
### Compat
|
||||
|
||||
`complete` / `chat` still return a bare string (or event list when `stream=True`) so `ext/ai/agent` and older runbooks keep working. **New code should use `ask` / `stream` / `collect`.**
|
||||
|
||||
## Environment
|
||||
|
||||
```env
|
||||
OPENAI_API_KEY=sk-...
|
||||
@@ -72,14 +72,16 @@ OPENAI_BASE_URL=https://api.openai.com/v1
|
||||
OPENAI_MODEL=gpt-4o-mini
|
||||
```
|
||||
|
||||
Fallbacks: `MARQDO_LLM_API_KEY`, `MARQDO_LLM_BASE_URL`, `MARQDO_LLM_MODEL`.
|
||||
Fallbacks: `MARQDO_LLM_API_KEY`, `MARQDO_LLM_BASE_URL`, `MARQDO_LLM_MODEL`. Ctor also accepts `api_key=` / `base_url=` / `model=` / `backend=`.
|
||||
|
||||
## Roadmap (see ai-stack.md)
|
||||
|
||||
Prompt-as-document, named models (`fast` / `reasoning`), multi-backend split, richer usage/tool_calls — deferred; keep transport behind the semantic API.
|
||||
|
||||
## Tests
|
||||
|
||||
- Import smoke: `tests/ext/llm-import.mq.md`
|
||||
- Offline stream: `tests/ext/llm-stream-offline.mq.md` + `tests/lib/net-openai-sse.mq.md`
|
||||
- Live complete (local): `tests/ext/llm-complete.mq.md` + `tests/ext/.env`
|
||||
- Live agent `## step`: `tests/ext/agent-run-live.mq.md` (DeepSeek; gold: `ext_agent_run_live`)
|
||||
- Agent smoke (real `llm` handle): `tests/ext/agent-smoke.mq.md` (gold: `ext_agent_framework_smoke`)
|
||||
- Live complete: `tests/ext/llm-complete.mq.md` (gold: `ext_llm_complete_live`)
|
||||
|
||||
- `tests/ext/llm-import.mq.md`
|
||||
- `tests/ext/llm-ctor-offline.mq.md`
|
||||
- `tests/ext/llm-ask-offline.mq.md` (create + collect)
|
||||
- `tests/ext/llm-stream-offline.mq.md`
|
||||
- Live: `llm-complete.mq.md` · `llm-stream-live.mq.md`
|
||||
+1173
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,70 @@
|
||||
# Marqdo AI 栈统一开发方案(llm + agent)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Status | **Active** |
|
||||
| Date | 2026-09-24 |
|
||||
| Source | [doc/next/002–005](../next/) · [ext-llm.md](../design/ext-llm.md) · [ext-agent.md](../design/ext-agent.md) |
|
||||
|
||||
## 1. 一句话
|
||||
|
||||
> **Document → Prompt → LLM → Agent → Skill → Policy → Executable Knowledge**
|
||||
|
||||
大模型负责探索,小模型/可选 Jev 负责决策,Marqdo 负责执行;经验编译进 `.mq.md` 后推理成本下降。
|
||||
|
||||
## 2. 分层(勿混)
|
||||
|
||||
| 层 | 职责 | 落点 |
|
||||
|----|------|------|
|
||||
| **Intelligence primitive** | `ask` / `stream` / `collect`;结果对象 | `ext/ai/llm` |
|
||||
| **Agent runtime** | `run` / `step` / `plan` / `route` | `ext/ai/agent` |
|
||||
| **Memory / compile** | episode → skill → policy | `plugins/agent`(ABI;**不进** `src/host`) |
|
||||
| **Transport** | HTTP / SSE / provider | llm 内部后端;用户不可见 |
|
||||
|
||||
硬规则:`ext/**` 不调 `host_*`;agent/OKF/路由热路径只进 `plugins/agent`。
|
||||
|
||||
## 3. LLM(005)— 已/待
|
||||
|
||||
| 项 | 状态 |
|
||||
|----|------|
|
||||
| 语义 API:`ask` / `stream` / `collect`(`complete` 降为兼容原语) | **done** |
|
||||
| `create` 工厂;handle 带 `backend` | **done** |
|
||||
| LLMResult:`text` / `model` / `finish` / `backend` | **done**(最小字段) |
|
||||
| Prompt = Document / `llm.ask` 无参吃正文 | 暂缓 |
|
||||
| Named models(`fast` / `reasoning`) | 暂缓 |
|
||||
| 拆 `ext/ai/llm/openai.mq.md` 等多后端 | 暂缓 |
|
||||
| usage / tool_calls 完整 Result | 暂缓 |
|
||||
| Transport → 可选 ABI 插件 | 暂缓 |
|
||||
|
||||
作者面优先:
|
||||
|
||||
```marqdo
|
||||
**answer = > llm.ask prompt="What is Marqdo?"**
|
||||
**events = > model.stream prompt=`p`**
|
||||
**text = > llm.collect events=`events`**
|
||||
```
|
||||
|
||||
## 4. Agent(002–004)— 已/待
|
||||
|
||||
| 项 | 状态 |
|
||||
|----|------|
|
||||
| Skill Compilation:episode → maybe_learn → `llm_free` | **done** |
|
||||
| Adaptive Routing:`marqdo` \| `small-llm` \| `jev?` \| `llm` \| `auto` | **done** |
|
||||
| Policy Compilation → `policies/router.mq.md` | **done** |
|
||||
| `run` 串联 route → execute → record → learn | **done** |
|
||||
| Agent 默认走 `model.ask` + Result.usage 记成本 | 下一步 |
|
||||
| Prompt artifact / document-as-prompt | 与 LLM Prompt 层一起做 |
|
||||
| 真 MCP client / resume / Jev 接线 / 小模型 live 路由 | 暂缓 |
|
||||
|
||||
## 5. 联调顺序(继续开发)
|
||||
|
||||
1. **LLM Result → Agent metrics** — `step`/`plan` 记 `llm_calls`/`tokens` 从 Result 取。
|
||||
2. **Named handles** — `llm.create model=…` 命名 fast/reasoning,接到 `run router_model=`。
|
||||
3. **Prompt document** — `type: prompt` 文件 / 当前文档切片作 ask 输入。
|
||||
4. **Backend split** — openai-compatible 抽到子模块;ollama 第二后端。
|
||||
5. **Execution compilation** — policy+skill 全路径零 LLM 金样与 harness 指标。
|
||||
|
||||
## 6. 验收金样
|
||||
|
||||
- Agent:`tests/ext/agent-v2-learn.mq.md` · `agent-p4-route.mq.md` · `scripts/agent-harness.sh`
|
||||
- LLM:`tests/ext/llm-ask-offline.mq.md` · `llm-stream-offline.mq.md` · `llm-ctor-offline.mq.md`
|
||||
@@ -41,6 +41,7 @@
|
||||
| **B1** | 评测 harness 锁差异化指标(constitution / dump / writeback 遮蔽) | **Active** |
|
||||
| **B1.5** | **Skill Compilation v2(Phase 1–3)**:episode → maybe_learn → compile → `llm_free`;`agent.run` / `resolve` / `metrics`;ABI 在 `plugins/agent`(`memory.rs`);金样 `agent-v2-learn` | **done** |
|
||||
| **B1.6 / P4** | **Adaptive Routing**(原 Jev Router 更名):`route` backends = `marqdo` \| `small-llm` \| `jev?` \| `llm` \| `auto`;Policy Compilation → `policies/router.mq.md`;金样 `agent-p4-route`;设计 [004.md](../next/004.md) | **done** |
|
||||
| **B2–B5** | 真 MCP / resume+HITL / stats / 并行;小模型 live 路由;Jev 可选接线 | 暂缓 |
|
||||
| **L0** | **LLM semantic surface(005)**:`ask` / `stream` / `collect` / `create`;`complete` 降为兼容;统一方案 [ai-stack.md](ai-stack.md) | **done** |
|
||||
| **B2–B5** | 真 MCP / resume+HITL / stats / 并行;小模型 live 路由;Jev 可选接线;LLM named models / prompt-document | 暂缓 |
|
||||
|
||||
示例:`examples/agent-pong/` · `examples/agent-okf-flywheel/`。v2 设计:[doc/next/002.md](../next/002.md) · [doc/next/003.md](../next/003.md) · P4:[doc/next/004.md](../next/004.md)。
|
||||
示例:`examples/agent-pong/` · `examples/agent-okf-flywheel/`。v2 设计:[doc/next/002.md](../next/002.md) · [doc/next/003.md](../next/003.md) · P4:[doc/next/004.md](../next/004.md) · LLM:[doc/next/005.md](../next/005.md)。
|
||||
+112
-30
@@ -1,6 +1,8 @@
|
||||
---
|
||||
title: ext/ai/llm
|
||||
description: OpenAI-compatible LLM object (English). Import ext/ai/llm.mq.md.
|
||||
description: >-
|
||||
Marqdo LLM intelligence primitive — ask / stream / collect.
|
||||
OpenAI-compatible HTTP is an internal backend, not the authoring surface.
|
||||
import sys:lib/sys.mq.md
|
||||
import net:lib/net.mq.md
|
||||
import json:lib/json.mq.md
|
||||
@@ -10,14 +12,14 @@ import table:lib/table.mq.md
|
||||
## load_env
|
||||
+ `path`=None
|
||||
|
||||
Load `.env` from cwd (optional named arg `path=`). Does not override existing variables.
|
||||
Load `.env` from cwd (optional `path=`). Does not override existing variables.
|
||||
|
||||
*> sys.load_dotenv path=`path`*
|
||||
|
||||
## stream_result
|
||||
## collect
|
||||
+ `events`
|
||||
|
||||
Reduce a `complete stream=True` event list to the final `done.result` text (or `error.message`). Unknown types (e.g. `reasoning`) are ignored.
|
||||
Collapse a stream event list to final answer text (`done.result` or `error.message`). Prefer this over raw SSE fields. Alias: `stream_result`.
|
||||
|
||||
**answer = ""**
|
||||
|
||||
@@ -28,51 +30,133 @@ Reduce a `complete stream=True` event list to the final `done.result` text (or `
|
||||
2. `t` == "error"
|
||||
**answer = [message](ev)**
|
||||
3. `t` == "delta"
|
||||
4. *
|
||||
4. `t` == "text"
|
||||
**chunk = [text](ev)**
|
||||
1. `chunk`
|
||||
**answer = answer + chunk**
|
||||
2. *
|
||||
**_ = 1**
|
||||
5. *
|
||||
**_ = 1**
|
||||
|
||||
*answer*
|
||||
|
||||
## stream_result
|
||||
+ `events`
|
||||
|
||||
Compatibility alias for `collect`.
|
||||
|
||||
*> collect events=`events`*
|
||||
|
||||
## create
|
||||
+ `model`=None
|
||||
+ `base_url`=None
|
||||
+ `api_key`=None
|
||||
+ `backend`=openai-compatible
|
||||
|
||||
Factory for an LLM handle (same as constructing `# llm`). Env fallbacks apply inside the constructor.
|
||||
|
||||
**h = > llm model=`model` base_url=`base_url` api_key=`api_key` backend=`backend`**
|
||||
*h*
|
||||
|
||||
## ask
|
||||
+ `prompt`
|
||||
+ `model`=None
|
||||
+ `base_url`=None
|
||||
+ `api_key`=None
|
||||
|
||||
Module convenience: create a default handle and return **answer text** (document-native one-liner).
|
||||
|
||||
**m = > create model=`model` base_url=`base_url` api_key=`api_key`**
|
||||
**r = > m.ask prompt=`prompt`**
|
||||
*[text](r)*
|
||||
|
||||
# llm
|
||||
+ `model`=None
|
||||
+ `base_url`=None
|
||||
+ `api_key`=None
|
||||
+ `backend`=openai-compatible
|
||||
|
||||
Construct an LLM handle from `OPENAI_*` / `MARQDO_LLM_*`.
|
||||
LLM handle — intelligence primitive, not an HTTP client. Transport fields stay on the handle for the openai-compatible backend only.
|
||||
|
||||
**api_key = > sys.env_get name="OPENAI_API_KEY"**
|
||||
1. not `api_key`
|
||||
**api_key = > sys.env_get name="MARQDO_LLM_API_KEY"**
|
||||
1. `api_key`
|
||||
**key = api_key**
|
||||
2. *
|
||||
**key = > sys.env_get name="OPENAI_API_KEY"**
|
||||
1. not `key`
|
||||
**key = > sys.env_get name="MARQDO_LLM_API_KEY"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. not `api_key`
|
||||
> print text=ext/ai/llm: set OPENAI_API_KEY or MARQDO_LLM_API_KEY
|
||||
1. not `key`
|
||||
> print text=ext/ai/llm: set OPENAI_API_KEY or MARQDO_LLM_API_KEY (or pass api_key=)
|
||||
> sys.exit code=1
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
**base_url = > sys.env_get name="OPENAI_BASE_URL"**
|
||||
1. not `base_url`
|
||||
**base_url = > sys.env_get name="MARQDO_LLM_BASE_URL"**
|
||||
1. not `base_url`
|
||||
**base_url = "https://api.openai.com/v1"**
|
||||
1. `base_url`
|
||||
**url = base_url**
|
||||
2. *
|
||||
**url = > sys.env_get name="OPENAI_BASE_URL"**
|
||||
1. not `url`
|
||||
**url = > sys.env_get name="MARQDO_LLM_BASE_URL"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. not `url`
|
||||
**url = "https://api.openai.com/v1"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
**model = > sys.env_get name="OPENAI_MODEL"**
|
||||
1. not `model`
|
||||
**model = > sys.env_get name="MARQDO_LLM_MODEL"**
|
||||
1. not `model`
|
||||
**model = "gpt-4o-mini"**
|
||||
1. `model`
|
||||
**mdl = model**
|
||||
2. *
|
||||
**mdl = > sys.env_get name="OPENAI_MODEL"**
|
||||
1. not `mdl`
|
||||
**mdl = > sys.env_get name="MARQDO_LLM_MODEL"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. not `mdl`
|
||||
**mdl = "gpt-4o-mini"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
`h` =
|
||||
|
||||
| api_key | base_url | model | suffix | bearer |
|
||||
|---------|----------|-------|--------|--------|
|
||||
| `api_key` | `base_url` | `model` | /chat/completions | "Bearer " |
|
||||
| _type | backend | api_key | base_url | model | suffix | bearer |
|
||||
|-------|---------|---------|----------|-------|--------|--------|
|
||||
| llm | `backend` | `key` | `url` | `mdl` | /chat/completions | "Bearer " |
|
||||
|
||||
*h*
|
||||
|
||||
## ask
|
||||
+ `prompt`
|
||||
|
||||
Semantic ask: one-shot answer as an **LLMResult** map (`text`, `model`, `finish`, `backend`). Prefer `[text](result)` or `result` in agent metrics.
|
||||
|
||||
**text = > self.complete prompt=`prompt` stream=False echo=False**
|
||||
|
||||
`result` =
|
||||
|
||||
| text | model | finish | backend |
|
||||
|------|-------|--------|---------|
|
||||
| `text` | [model](self) | stop | [backend](self) |
|
||||
|
||||
*result*
|
||||
|
||||
## stream
|
||||
+ `prompt`
|
||||
+ `echo`=False
|
||||
|
||||
Semantic stream: return the event list (use `llm.collect` to reduce to text).
|
||||
|
||||
*> self.complete prompt=`prompt` stream=True echo=`echo`*
|
||||
|
||||
## complete
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
|
||||
Chat completion using `self` handle fields.
|
||||
|
||||
Wire body is `{model, messages:[{role,content}], stream?}`. Messages are an `@` row table; headers and optional `stream` use `table.put`. `json` is only for stringify/parse.
|
||||
Internal / compatibility primitive (API-client shape). Prefer `ask` / `stream` in new code. OpenAI-compatible transport only when `backend` is openai-compatible.
|
||||
|
||||
**url = [base_url](self) + [suffix](self)**
|
||||
**auth = [bearer](self) + [api_key](self)**
|
||||
@@ -112,13 +196,11 @@ Wire body is `{model, messages:[{role,content}], stream?}`. Messages are an `@`
|
||||
> print text=[body](resp)
|
||||
> sys.exit code=1
|
||||
|
||||
---
|
||||
|
||||
## chat
|
||||
+ `prompt`
|
||||
+ `stream`=False
|
||||
+ `echo`=False
|
||||
|
||||
Alias for `complete`.
|
||||
Alias for `complete` (compat).
|
||||
|
||||
*> self.complete prompt=`prompt` stream=`stream` echo=`echo`*
|
||||
+112
-30
@@ -1,6 +1,8 @@
|
||||
---
|
||||
title: ext/ai/大模型
|
||||
description: OpenAI 兼容大模型对象(中文)。导入 ext/ai/大模型.mq.md。
|
||||
description: >-
|
||||
Marqdo 大模型智能原语 — 提问 / 流式 / 收集。
|
||||
OpenAI 兼容 HTTP 仅为内部后端,不是编排表面。
|
||||
导入 系统:lib/系统.mq.md
|
||||
导入 网络:lib/网络.mq.md
|
||||
import json:lib/json.mq.md
|
||||
@@ -10,14 +12,14 @@ import json:lib/json.mq.md
|
||||
## 加载环境
|
||||
+ `路径`=None
|
||||
|
||||
从工作目录加载 `.env`(可选命名实参 `路径=`)。不覆盖已有变量。
|
||||
从工作目录加载 `.env`(可选 `路径=`)。不覆盖已有变量。
|
||||
|
||||
*> 系统.加载环境 路径=`路径`*
|
||||
|
||||
## 流式结果
|
||||
## 收集
|
||||
+ `事件`
|
||||
|
||||
把 `运行 流式=真` 的事件列表收成最终 `done.result`(或 `error.message`)。未知类型(如 `reasoning`)忽略。
|
||||
把流式事件列表收成最终答复文本。兼容别名:`流式结果`。
|
||||
|
||||
**answer = ""**
|
||||
|
||||
@@ -28,51 +30,133 @@ import json:lib/json.mq.md
|
||||
2. `t` == "error"
|
||||
**answer = [message](ev)**
|
||||
3. `t` == "delta"
|
||||
4. *
|
||||
4. `t` == "text"
|
||||
**chunk = [text](ev)**
|
||||
1. `chunk`
|
||||
**answer = answer + chunk**
|
||||
2. *
|
||||
**_ = 1**
|
||||
5. *
|
||||
**_ = 1**
|
||||
|
||||
*answer*
|
||||
|
||||
## 流式结果
|
||||
+ `事件`
|
||||
|
||||
`收集` 的兼容别名。
|
||||
|
||||
*> 收集 事件=`事件`*
|
||||
|
||||
## 创建
|
||||
+ `模型`=None
|
||||
+ `基址`=None
|
||||
+ `密钥`=None
|
||||
+ `后端`=openai-compatible
|
||||
|
||||
工厂:构造大模型句柄(同 `# 大模型` 构造)。
|
||||
|
||||
**h = > 大模型 模型=`模型` 基址=`基址` 密钥=`密钥` 后端=`后端`**
|
||||
*h*
|
||||
|
||||
## 提问
|
||||
+ `提示`
|
||||
+ `模型`=None
|
||||
+ `基址`=None
|
||||
+ `密钥`=None
|
||||
|
||||
模块便捷入口:创建默认句柄并返回**答复文本**。
|
||||
|
||||
**m = > 创建 模型=`模型` 基址=`基址` 密钥=`密钥`**
|
||||
**r = > m.提问 提示=`提示`**
|
||||
*[text](r)*
|
||||
|
||||
# 大模型
|
||||
+ `模型`=None
|
||||
+ `基址`=None
|
||||
+ `密钥`=None
|
||||
+ `后端`=openai-compatible
|
||||
|
||||
从 `OPENAI_*` / `MARQDO_LLM_*` 构造句柄。
|
||||
大模型句柄 — 智能原语,不是 HTTP 客户端。
|
||||
|
||||
**api_key = > 系统.取环境 名="OPENAI_API_KEY"**
|
||||
1. 非 `api_key`
|
||||
**api_key = > 系统.取环境 名="MARQDO_LLM_API_KEY"**
|
||||
1. `密钥`
|
||||
**key = 密钥**
|
||||
2. *
|
||||
**key = > 系统.取环境 名="OPENAI_API_KEY"**
|
||||
1. 非 `key`
|
||||
**key = > 系统.取环境 名="MARQDO_LLM_API_KEY"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
1. 非 `api_key`
|
||||
> 打印 内容=ext/ai/大模型: 请设置 OPENAI_API_KEY 或 MARQDO_LLM_API_KEY
|
||||
1. 非 `key`
|
||||
> 打印 内容=ext/ai/大模型: 请设置 OPENAI_API_KEY 或 MARQDO_LLM_API_KEY(或传入 密钥=)
|
||||
> 系统.退出 码=1
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
**base_url = > 系统.取环境 名="OPENAI_BASE_URL"**
|
||||
1. 非 `base_url`
|
||||
**base_url = > 系统.取环境 名="MARQDO_LLM_BASE_URL"**
|
||||
1. 非 `base_url`
|
||||
**base_url = "https://api.openai.com/v1"**
|
||||
1. `基址`
|
||||
**url = 基址**
|
||||
2. *
|
||||
**url = > 系统.取环境 名="OPENAI_BASE_URL"**
|
||||
1. 非 `url`
|
||||
**url = > 系统.取环境 名="MARQDO_LLM_BASE_URL"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. 非 `url`
|
||||
**url = "https://api.openai.com/v1"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
**model = > 系统.取环境 名="OPENAI_MODEL"**
|
||||
1. 非 `model`
|
||||
**model = > 系统.取环境 名="MARQDO_LLM_MODEL"**
|
||||
1. 非 `model`
|
||||
**model = "gpt-4o-mini"**
|
||||
1. `模型`
|
||||
**mdl = 模型**
|
||||
2. *
|
||||
**mdl = > 系统.取环境 名="OPENAI_MODEL"**
|
||||
1. 非 `mdl`
|
||||
**mdl = > 系统.取环境 名="MARQDO_LLM_MODEL"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
1. 非 `mdl`
|
||||
**mdl = "gpt-4o-mini"**
|
||||
2. *
|
||||
**_ = 1**
|
||||
|
||||
`h` =
|
||||
|
||||
| api_key | base_url | model | suffix | bearer |
|
||||
|---------|----------|-------|--------|--------|
|
||||
| `api_key` | `base_url` | `model` | /chat/completions | "Bearer " |
|
||||
| _type | backend | api_key | base_url | model | suffix | bearer |
|
||||
|-------|---------|---------|----------|-------|--------|--------|
|
||||
| 大模型 | `后端` | `key` | `url` | `mdl` | /chat/completions | "Bearer " |
|
||||
|
||||
*h*
|
||||
|
||||
## 提问
|
||||
+ `提示`
|
||||
|
||||
语义提问:返回 **结果 map**(`text` / `model` / `finish` / `backend`)。
|
||||
|
||||
**text = > 自.运行 提示=`提示` 流式=False 打印增量=False**
|
||||
|
||||
`result` =
|
||||
|
||||
| text | model | finish | backend |
|
||||
|------|-------|--------|---------|
|
||||
| `text` | [model](自) | stop | [backend](自) |
|
||||
|
||||
*result*
|
||||
|
||||
## 流式
|
||||
+ `提示`
|
||||
+ `打印增量`=False
|
||||
|
||||
语义流式:返回事件列表(再用 `收集` 收成文本)。
|
||||
|
||||
*> 自.运行 提示=`提示` 流式=True 打印增量=`打印增量`*
|
||||
|
||||
## 运行
|
||||
+ `提示`
|
||||
+ `流式`=False
|
||||
+ `打印增量`=False
|
||||
|
||||
使用 `自` 句柄字段做聊天补全。
|
||||
|
||||
线上 body 为 `{model, messages:[{role,content}], stream?}`。消息用 `@` 行向表;headers 与可选 `stream` 用 `表.改`。`json` 只做 stringify/parse。
|
||||
内部 / 兼容原语。新代码优先 `提问` / `流式`。
|
||||
|
||||
**url = [base_url](自) + [suffix](自)**
|
||||
**auth = [bearer](自) + [api_key](自)**
|
||||
@@ -112,13 +196,11 @@ import json:lib/json.mq.md
|
||||
> 打印 内容=[body](resp)
|
||||
> 系统.退出 码=1
|
||||
|
||||
---
|
||||
|
||||
## 聊天
|
||||
+ `提示`
|
||||
+ `流式`=False
|
||||
+ `打印增量`=False
|
||||
|
||||
运行 的别名。
|
||||
`运行` 的别名。
|
||||
|
||||
*> 自.运行 提示=`提示` 流式=`流式` 打印增量=`打印增量`*
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
title: Agent Framework P4 — Adaptive Routing + Policy Compilation (offline)
|
||||
description: Learn skill, compile policy from skill-tagged episodes, route via marqdo backend without Jev/LLM.
|
||||
import fs:lib/fs.mq.md
|
||||
import json:lib/json.mq.md
|
||||
import plugin:lib/plugin.mq.md
|
||||
import sys:lib/sys.mq.md
|
||||
@@ -21,6 +22,8 @@ import sys:lib/sys.mq.md
|
||||
**task = > json.get value=`paths` key="task"**
|
||||
**skill = > json.get value=`paths` key="skill"**
|
||||
|
||||
> fs.remove_tree path=`mem`
|
||||
> fs.remove_tree path=`kb`
|
||||
> agent_memory_ensure memory_dir=`mem`
|
||||
|
||||
**ep1 = > agent_record_episode task=`task` status="ok" result="pong" mode="explore" skill=`skill` llm_calls=2 memory_dir=`mem`**
|
||||
|
||||
@@ -22,6 +22,8 @@ import sys:lib/sys.mq.md
|
||||
**kb = > json.get value=`paths` key="kb"**
|
||||
**task = > json.get value=`paths` key="task"**
|
||||
|
||||
> fs.remove_tree path=`mem`
|
||||
> fs.remove_tree path=`kb`
|
||||
> agent_memory_ensure memory_dir=`mem`
|
||||
|
||||
**ep1 = > agent_record_episode task=`task` status="ok" result="pong" mode="explore" llm_calls=2 memory_dir=`mem`**
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
---
|
||||
title: llm create + collect offline
|
||||
description: Factory create + collect alias; no network.
|
||||
import llm:ext/ai/llm.mq.md
|
||||
import net:lib/net.mq.md
|
||||
---
|
||||
|
||||
# main
|
||||
|
||||
**model = > llm.create**
|
||||
> print text=[model](`model`)
|
||||
> print text=[backend](`model`)
|
||||
|
||||
**fixture = "data: {\"choices\":[{\"delta\":{\"content\":\"Hi\"}}]}\n\ndata: {\"choices\":[{\"delta\":{\"content\":\"!\"}}]}\n\ndata: [DONE]\n"**
|
||||
**events = > net.openai_sse_parse text=`fixture`**
|
||||
**answer = > llm.collect events=`events`**
|
||||
> print text=`answer`
|
||||
**alias = > llm.stream_result events=`events`**
|
||||
> print text=`alias`
|
||||
@@ -162,10 +162,16 @@ fn assert_err(path: &str, line_col: &str, substr: &str) {
|
||||
|
||||
/// Like [`assert_out`], but sets a dummy OpenAI key so offline LLM/agent paths
|
||||
/// that only check for the env var (no live HTTP) can proceed.
|
||||
/// Also points `MARQDO_EXT` at the repo `ext/` so tests do not pick up a stale
|
||||
/// `~/.marqdo/ext` pack ahead of in-tree sources.
|
||||
fn assert_out_llm_offline(path: &str, expect: &str) {
|
||||
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
|
||||
let ext = root.join("ext");
|
||||
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
|
||||
.args(["run", path])
|
||||
.env("OPENAI_API_KEY", "sk-test")
|
||||
.env("MARQDO_EXT", &ext)
|
||||
.current_dir(&root)
|
||||
.output()
|
||||
.expect("failed to run marqdo");
|
||||
let code = output.status.code().unwrap_or(1);
|
||||
@@ -3860,13 +3866,39 @@ fn ext_llm_stream_offline() {
|
||||
assert_out("tests/ext/llm-stream-offline.mq.md", "Hi!");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ext_llm_ask_offline() {
|
||||
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
|
||||
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
|
||||
.args(["run", "tests/ext/llm-ask-offline.mq.md"])
|
||||
.env("OPENAI_API_KEY", "sk-test")
|
||||
.env("MARQDO_EXT", root.join("ext"))
|
||||
.env_remove("OPENAI_MODEL")
|
||||
.env_remove("MARQDO_LLM_MODEL")
|
||||
.current_dir(&root)
|
||||
.output()
|
||||
.expect("run llm-ask-offline");
|
||||
let code = output.status.code().unwrap_or(1);
|
||||
let stdout = String::from_utf8_lossy(&output.stdout);
|
||||
let stderr = String::from_utf8_lossy(&output.stderr);
|
||||
assert_eq!(code, 0, "llm-ask-offline stderr={stderr}");
|
||||
assert_eq!(
|
||||
stdout.trim_end(),
|
||||
"gpt-4o-mini\nopenai-compatible\nHi!\nHi!",
|
||||
"stdout={stdout}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ext_llm_ctor_offline() {
|
||||
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
|
||||
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
|
||||
.args(["run", "tests/ext/llm-ctor-offline.mq.md"])
|
||||
.env("OPENAI_API_KEY", "sk-test")
|
||||
.env("MARQDO_EXT", root.join("ext"))
|
||||
.env_remove("OPENAI_MODEL")
|
||||
.env_remove("MARQDO_LLM_MODEL")
|
||||
.current_dir(&root)
|
||||
.output()
|
||||
.expect("run llm-ctor-offline");
|
||||
let code = output.status.code().unwrap_or(1);
|
||||
|
||||
Reference in new issue
Block a user