Long sse-live chats held INVOKE_LOCK indefinitely and starved short routes (UI freeze). Timed try-lock returns interpreter busy → HTTP 503; ADR 0008 records that HTTP is multi-goroutine while Marqdo evaluation stays single-flight. Also lands plan/delta streaming and soft map guards used by qdagent chat. Co-authored-by: Cursor <cursoragent@cursor.com>
12 KiB
markup, title, description, import
| markup | title | description | import | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| occlusion | ext/ai/llm | Marqdo LLM intelligence primitive — ask / stream / collect / named handles. OpenAI-compatible and Ollama HTTP stay behind the semantic surface. |
|
load_env
+ `path`=None
Load .env from cwd (optional path=). Does not override existing variables.
> sys.load_dotenv path=path
collect
+ `events`
Collapse a stream event list to final answer text.
Accepts semantic events (delta / text / done.result / error.message) and
native-plugin raw SSE frames (type=data with OpenAI chunk JSON in data).
Alias: stream_result.
answer = ""
- [ev](遍历 events 为 ev)
t = > json.get value=
evkey="type"t== "done" done_r = > json.get value=evkey="result"done_ranswer =done_r-
**_ = 1**t== "error" em = > json.get value=evkey="message"emanswer =em-
**_ = 1**t== "delta"t== "text" chunk = > json.get value=evkey="text"chunkanswer = answer + chunk-
**_ = 1**t== "data" raw = > json.get value=evkey="data"raw1.raw== "[DONE]" _ = 1 2. * parsed = > json.parse text=rawparsedchoices = > json.get value=parsedkey="choices"choicesc0 = 1) delta = > json.get value=c0key="delta" 1.deltapiece = > json.get value=deltakey="content"pieceanswer = answer + piece-
c0key="message"msgpiece = > json.get value=msgkey="content"pieceanswer = answer + piece-
**_ = 1**-
**_ = 1**-
**_ = 1**
- not
answeranswer = "" answer
stream_result
+ `events`
Compatibility alias for collect.
> collect events=events
usage_from
+ `data`
Extract {prompt_tokens,completion_tokens,total_tokens} from an OpenAI-shaped response map (or empty zeros).
> openai.usage_from data=data
create
+ `model`=None
+ `base_url`=None
+ `api_key`=None
+ `backend`=openai-compatible
+ `name`=None
Factory for an LLM handle (same as constructing # llm). Env fallbacks apply inside the constructor. Optional name= tags the handle (fast / reasoning).
h = > llm model=model base_url=base_url api_key=api_key backend=backend name=name
h
fast
+ `model`=None
+ `base_url`=None
+ `api_key`=None
Named small/fast handle. Model from arg, else MARQDO_LLM_FAST_MODEL, else default create model.
modelmdl =model-
mdl = > sys.env_get name="MARQDO_LLM_FAST_MODEL"
> create model=mdl base_url=base_url api_key=api_key name="fast" backend="openai-compatible"
reasoning
+ `model`=None
+ `base_url`=None
+ `api_key`=None
Named strong/reasoning handle. Model from arg, else MARQDO_LLM_REASONING_MODEL, else default create model.
modelmdl =model-
mdl = > sys.env_get name="MARQDO_LLM_REASONING_MODEL"
> create model=mdl base_url=base_url api_key=api_key name="reasoning" backend="openai-compatible"
prompt_body
+ `src`
If src starts with YAML frontmatter declaring type: prompt / 类型: 提示, return the body after the closing ---; otherwise return src unchanged.
sw = > text.starts_with text=src prefix="---"
- not
swsrc -
parts = > split value=src sep="\n---\n"
n = > len value=parts
n< 2 src-
**fm = > at value=`parts` index=0**
**is_p = > text.contains text=`fm` sub="type: prompt"**
1. not `is_p`
**is_p = > text.contains text=`fm` sub="类型: 提示"**
2. *
**_ = 1**
1. `is_p`
**body = > at value=`parts` index=1**
*> text.str_trim s=`body`*
2. *
*src*
prompt_load
+ `path`
Load a prompt artifact (.mq.md / .md) from disk for ask.
raw = > fs.read_text path=path
> prompt_body src=raw
prompt
+ `path`=None
+ `text`=None
Prompt artifact helper: path= loads a file; text= strips type: prompt frontmatter from an in-memory string.
path> prompt_load path=pathtext> prompt_body src=text-
print text=ext/ai/llm.prompt: pass path= or text= sys.exit code=1
ask
+ `prompt`=None
+ `path`=None
+ `model`=None
+ `base_url`=None
+ `api_key`=None
Module convenience: create a default handle and return answer text. Pass path= to load a prompt document, prompt= text, or omit both to use the entry .mq.md as a prompt artifact (type: prompt body via sys.module_source).
p = prompt
pathp = > prompt_load path=pathpromptp =prompt-
raw = > sys.module_source
- not
raw> print text=ext/ai/llm.ask: pass prompt= or path= (or run a prompt document as entry) > sys.exit code=1 -
**p = > prompt_body src=`raw`**
m = > create model=model base_url=base_url api_key=api_key
r = > m.ask prompt=p
_ret = > json.get value=r key="text"
_ret
llm
+ `model`=None
+ `base_url`=None
+ `api_key`=None
+ `backend`=openai-compatible
+ `name`=None
LLM handle — intelligence primitive, not an HTTP client.
Resolution: explicit args → entry-scope / Artifact Metadata (sys.meta_get) → sys.env_get → defaults.
Prefer declaring model / api_key / base_url in frontmatter Binding, then use the lifted names.
api_keykey =api_key-
key = > sys.meta_get key="api_key"
- not
keykey = > sys.env_get name="OPENAI_API_KEY" 1. notkeykey = > sys.env_get name="MARQDO_LLM_API_KEY" 2. * _ = 1 -
**_ = 1**
backend== ollama- not
keykey = "ollama" -
**_ = 1**
- not
key
print text=ext/ai/llm: set OPENAI_API_KEY or MARQDO_LLM_API_KEY (or pass api_key= / metadata api_key) sys.exit code=1
_ = 1
base_urlurl =base_url-
url = > sys.meta_get key="base_url"
url_ = 1backend== ollama url = > sys.env_get name="OLLAMA_HOST" 1. noturlurl = "http://127.0.0.1:11434/v1" 2. * _ = 1-
**url = > sys.env_get name="OPENAI_BASE_URL"**
1. not `url`
**url = > sys.env_get name="MARQDO_LLM_BASE_URL"**
2. *
**_ = 1**
1. not `url`
**url = "https://api.openai.com/v1"**
2. *
**_ = 1**
modelmdl =model-
mdl = > sys.meta_get key="model"
mdl_ = 1-
**mdl = > sys.env_get name="OPENAI_MODEL"**
1. not `mdl`
**mdl = > sys.env_get name="MARQDO_LLM_MODEL"**
2. *
**_ = 1**
1. not `mdl`
1. `backend` == ollama
**mdl = "llama3.2"**
2. *
**mdl = "gpt-4o-mini"**
2. *
**_ = 1**
namenm =name-
nm = None
h =
| _type | backend | api_key | base_url | model | suffix | bearer | name |
|---|---|---|---|---|---|---|---|
| llm | backend |
key |
url |
mdl |
/chat/completions | "Bearer " | nm |
h
ask
+ `prompt`=None
+ `path`=None
+ `messages`=None
+ `selection`=None
+ `strict_selection`=False
Semantic ask → LLMResult (text, model, finish, backend, usage, name, tool_calls). Prefer [text](result) for the answer string. Omit prompt= / path= to use the entry document (sys.module_source + prompt_body). Pass messages= for multi-turn history (M01). selection= + strict_selection=True freeze request identity without env fallback.
p = prompt
messagesp = Nonepathp = > prompt_load path=pathpromptp =prompt-
raw = > sys.module_source
- not
rawp = "" -
**p = > prompt_body src=`raw`**
pack = > self.complete_result prompt=p messages=messages selection=selection strict_selection=strict_selection stream=False echo=False
text = > json.get value=pack key="text"
usage = > json.get value=pack key="usage"
finish = > json.get value=pack key="finish"
err = > json.get value=pack key="error"
tc = > json.get value=pack key="tool_calls"
- not
tctc = > table.empty_list -
_ = 1
- not
texttext = "" -
_ = 1
- not
finish errfinish = "error"-
**finish = "stop"**
_ = 1
Hoist receiver fields before the result table — table cells can shadow self /
column names (model, usage), so [model](self) / [total_tokens](usage)
inside the table is unsafe.
mdl = > json.get value=self key="model"
b = > json.get value=self key="backend"
nm = > json.get value=self key="name"
usagett = > json.get value=usagekey="total_tokens" ustate = > json.get value=usagekey="usage_state"-
tt = None ustate = "unknown"
ustate== "unknown" Keep null — never coerce missing usage to zero tokens (M02 / F04). tt = None- not
tttt = None -
_ = 1
errcalls = 0-
calls = 1
result =
| text | model | finish | backend | usage | name | llm_calls | tokens | tool_calls | error | selection |
|---|---|---|---|---|---|---|---|---|---|---|
text |
mdl |
finish |
b |
usage |
nm |
calls |
tt |
tc |
err |
selection |
result
stream
+ `prompt`=None
+ `path`=None
+ `echo`=False
+ `stream_kind`="delta"
Semantic stream: return the event list (use llm.collect to reduce to text). Omit prompt= / path= to use the entry document.
stream_kind=plan tags EventBus frames for the parent plan protocol (process trail).
pathp = > prompt_load path=pathpromptp =prompt-
raw = > sys.module_source
- not
rawp = "" -
**p = > prompt_body src=`raw`**
> self.complete prompt=p stream=True echo=echo stream_kind=stream_kind
complete_result
+ `prompt`=None
+ `messages`=None
+ `selection`=None
+ `strict_selection`=False
+ `stream`=False
+ `echo`=False
+ `stream_kind`="delta"
Transport pack via ext/ai/llm/openai (Ollama uses the same OpenAI-compatible path). Non-stream → {text,usage,finish}; stream → {events}. Prefer messages= for multi-turn; optional selection= (ModelSelectionV1) with strict_selection=True for request-scoped identity (M01).
> openai.chat_completions base_url=base_url) api_key=api_key) model=model) prompt=prompt messages=messages selection=selection strict_selection=strict_selection stream=stream echo=echo stream_kind=stream_kind suffix=suffix) bearer=bearer)
complete
+ `prompt`
+ `stream`=False
+ `echo`=False
+ `stream_kind`="delta"
Internal / compatibility primitive. Prefer ask / stream in new code.
streampack = > self.complete_result prompt=promptstream=True echo=echostream_kind=stream_kind_ret = > json.get value=packkey="events" _ret-
pack = > self.complete_result prompt=prompt stream=False echo=echo stream_kind=stream_kind
_ret = > json.get value=pack key="text"
_ret
chat
+ `prompt`
+ `stream`=False
+ `echo`=False
+ `stream_kind`="delta"
Alias for complete (compat).
> self.complete prompt=prompt stream=stream echo=echo stream_kind=stream_kind