Files
marqdo/ext/ai/llm.mq.md
T
cflmyandCursor 1b11fe063a fix(web): fail-fast when invoke lock is busy; document single-interp limit
Long sse-live chats held INVOKE_LOCK indefinitely and starved short routes
(UI freeze). Timed try-lock returns interpreter busy → HTTP 503; ADR 0008
records that HTTP is multi-goroutine while Marqdo evaluation stays single-flight.
Also lands plan/delta streaming and soft map guards used by qdagent chat.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-26 13:27:53 +08:00

12 KiB

markup, title, description, import
markup title description import
occlusion ext/ai/llm Marqdo LLM intelligence primitive — ask / stream / collect / named handles. OpenAI-compatible and Ollama HTTP stay behind the semantic surface.
sys json table fs text openai
lib/sys.mq.md lib/json.mq.md lib/table.mq.md lib/fs.mq.md lib/text.mq.md ext/ai/llm/openai.mq.md

load_env

+ `path`=None

Load .env from cwd (optional path=). Does not override existing variables.

> sys.load_dotenv path=path

collect

+ `events`

Collapse a stream event list to final answer text. Accepts semantic events (delta / text / done.result / error.message) and native-plugin raw SSE frames (type=data with OpenAI chunk JSON in data). Alias: stream_result.

answer = ""

  • [ev](遍历 events 为 ev) t = > json.get value=ev key="type"
    1. t == "done" done_r = > json.get value=ev key="result"
    2. done_r answer = done_r
    **_ = 1**
    
    1. t == "error" em = > json.get value=ev key="message"
    2. em answer = em
    **_ = 1**
    
    1. t == "delta"
    2. t == "text" chunk = > json.get value=ev key="text"
    3. chunk answer = answer + chunk
    **_ = 1**
    
    1. t == "data" raw = > json.get value=ev key="data"
    2. raw 1. raw == "[DONE]" _ = 1 2. * parsed = > json.parse text=raw
      1. parsed choices = > json.get value=parsed key="choices"
      2. choices c0 = 1) delta = > json.get value=c0 key="delta" 1. delta piece = > json.get value=delta key="content"
        1. piece answer = answer + piece
        _ = 1 2. * msg = > json.get value=c0 key="message"
        1. msg piece = > json.get value=msg key="content"
        2. piece answer = answer + piece
         **_ = 1**
        
        _ = 1
       **_ = 1**
      
      _ = 1
    **_ = 1**
    
    _ = 1
  1. not answer answer = "" answer

stream_result

+ `events`

Compatibility alias for collect.

> collect events=events

usage_from

+ `data`

Extract {prompt_tokens,completion_tokens,total_tokens} from an OpenAI-shaped response map (or empty zeros).

> openai.usage_from data=data

create

+ `model`=None
+ `base_url`=None
+ `api_key`=None
+ `backend`=openai-compatible
+ `name`=None

Factory for an LLM handle (same as constructing # llm). Env fallbacks apply inside the constructor. Optional name= tags the handle (fast / reasoning).

h = > llm model=model base_url=base_url api_key=api_key backend=backend name=name h

fast

+ `model`=None
+ `base_url`=None
+ `api_key`=None

Named small/fast handle. Model from arg, else MARQDO_LLM_FAST_MODEL, else default create model.

  1. model mdl = model

mdl = > sys.env_get name="MARQDO_LLM_FAST_MODEL"

> create model=mdl base_url=base_url api_key=api_key name="fast" backend="openai-compatible"

reasoning

+ `model`=None
+ `base_url`=None
+ `api_key`=None

Named strong/reasoning handle. Model from arg, else MARQDO_LLM_REASONING_MODEL, else default create model.

  1. model mdl = model

mdl = > sys.env_get name="MARQDO_LLM_REASONING_MODEL"

> create model=mdl base_url=base_url api_key=api_key name="reasoning" backend="openai-compatible"

prompt_body

+ `src`

If src starts with YAML frontmatter declaring type: prompt / 类型: 提示, return the body after the closing ---; otherwise return src unchanged.

sw = > text.starts_with text=src prefix="---"

  1. not sw src

parts = > split value=src sep="\n---\n" n = > len value=parts

  1. n < 2 src
**fm = > at value=`parts` index=0**
**is_p = > text.contains text=`fm` sub="type: prompt"**
1. not `is_p`
  **is_p = > text.contains text=`fm` sub="类型: 提示"**
2. *
  **_ = 1**
1. `is_p`
  **body = > at value=`parts` index=1**
  *> text.str_trim s=`body`*
2. *
  *src*

prompt_load

+ `path`

Load a prompt artifact (.mq.md / .md) from disk for ask.

raw = > fs.read_text path=path > prompt_body src=raw

prompt

+ `path`=None
+ `text`=None

Prompt artifact helper: path= loads a file; text= strips type: prompt frontmatter from an in-memory string.

  1. path > prompt_load path=path
  2. text > prompt_body src=text

print text=ext/ai/llm.prompt: pass path= or text= sys.exit code=1

ask

+ `prompt`=None
+ `path`=None
+ `model`=None
+ `base_url`=None
+ `api_key`=None

Module convenience: create a default handle and return answer text. Pass path= to load a prompt document, prompt= text, or omit both to use the entry .mq.md as a prompt artifact (type: prompt body via sys.module_source).

p = prompt

  1. path p = > prompt_load path=path
  2. prompt p = prompt

raw = > sys.module_source

  1. not raw > print text=ext/ai/llm.ask: pass prompt= or path= (or run a prompt document as entry) > sys.exit code=1
**p = > prompt_body src=`raw`**

m = > create model=model base_url=base_url api_key=api_key r = > m.ask prompt=p _ret = > json.get value=r key="text" _ret

llm

+ `model`=None
+ `base_url`=None
+ `api_key`=None
+ `backend`=openai-compatible
+ `name`=None

LLM handle — intelligence primitive, not an HTTP client.

Resolution: explicit args → entry-scope / Artifact Metadata (sys.meta_get) → sys.env_get → defaults. Prefer declaring model / api_key / base_url in frontmatter Binding, then use the lifted names.

  1. api_key key = api_key

key = > sys.meta_get key="api_key"

  1. not key key = > sys.env_get name="OPENAI_API_KEY" 1. not key key = > sys.env_get name="MARQDO_LLM_API_KEY" 2. * _ = 1
**_ = 1**
  1. backend == ollama
  2. not key key = "ollama"
**_ = 1**
  1. not key

print text=ext/ai/llm: set OPENAI_API_KEY or MARQDO_LLM_API_KEY (or pass api_key= / metadata api_key) sys.exit code=1

_ = 1

  1. base_url url = base_url

url = > sys.meta_get key="base_url"

  1. url _ = 1
  2. backend == ollama url = > sys.env_get name="OLLAMA_HOST" 1. not url url = "http://127.0.0.1:11434/v1" 2. * _ = 1
**url = > sys.env_get name="OPENAI_BASE_URL"**
1. not `url`
  **url = > sys.env_get name="MARQDO_LLM_BASE_URL"**
2. *
  **_ = 1**
1. not `url`
  **url = "https://api.openai.com/v1"**
2. *
  **_ = 1**
  1. model mdl = model

mdl = > sys.meta_get key="model"

  1. mdl _ = 1
**mdl = > sys.env_get name="OPENAI_MODEL"**
1. not `mdl`
  **mdl = > sys.env_get name="MARQDO_LLM_MODEL"**
2. *
  **_ = 1**
1. not `mdl`
  1. `backend` == ollama
    **mdl = "llama3.2"**
  2. *
    **mdl = "gpt-4o-mini"**
2. *
  **_ = 1**
  1. name nm = name

nm = None

h =

_type backend api_key base_url model suffix bearer name
llm backend key url mdl /chat/completions "Bearer " nm

h

ask

+ `prompt`=None
+ `path`=None
+ `messages`=None
+ `selection`=None
+ `strict_selection`=False

Semantic ask → LLMResult (text, model, finish, backend, usage, name, tool_calls). Prefer [text](result) for the answer string. Omit prompt= / path= to use the entry document (sys.module_source + prompt_body). Pass messages= for multi-turn history (M01). selection= + strict_selection=True freeze request identity without env fallback.

p = prompt

  1. messages p = None
  2. path p = > prompt_load path=path
  3. prompt p = prompt

raw = > sys.module_source

  1. not raw p = ""
**p = > prompt_body src=`raw`**

pack = > self.complete_result prompt=p messages=messages selection=selection strict_selection=strict_selection stream=False echo=False text = > json.get value=pack key="text" usage = > json.get value=pack key="usage" finish = > json.get value=pack key="finish" err = > json.get value=pack key="error" tc = > json.get value=pack key="tool_calls"

  1. not tc tc = > table.empty_list

_ = 1

  1. not text text = ""

_ = 1

  1. not finish
  2. err finish = "error"
**finish = "stop"**

_ = 1

Hoist receiver fields before the result table — table cells can shadow self / column names (model, usage), so [model](self) / [total_tokens](usage) inside the table is unsafe.

mdl = > json.get value=self key="model" b = > json.get value=self key="backend" nm = > json.get value=self key="name"

  1. usage tt = > json.get value=usage key="total_tokens" ustate = > json.get value=usage key="usage_state"

tt = None ustate = "unknown"

  1. ustate == "unknown" Keep null — never coerce missing usage to zero tokens (M02 / F04). tt = None
  2. not tt tt = None

_ = 1

  1. err calls = 0

calls = 1

result =

text model finish backend usage name llm_calls tokens tool_calls error selection
text mdl finish b usage nm calls tt tc err selection

result

stream

+ `prompt`=None
+ `path`=None
+ `echo`=False
+ `stream_kind`="delta"

Semantic stream: return the event list (use llm.collect to reduce to text). Omit prompt= / path= to use the entry document. stream_kind=plan tags EventBus frames for the parent plan protocol (process trail).

  1. path p = > prompt_load path=path
  2. prompt p = prompt

raw = > sys.module_source

  1. not raw p = ""
**p = > prompt_body src=`raw`**

> self.complete prompt=p stream=True echo=echo stream_kind=stream_kind

complete_result

+ `prompt`=None
+ `messages`=None
+ `selection`=None
+ `strict_selection`=False
+ `stream`=False
+ `echo`=False
+ `stream_kind`="delta"

Transport pack via ext/ai/llm/openai (Ollama uses the same OpenAI-compatible path). Non-stream → {text,usage,finish}; stream → {events}. Prefer messages= for multi-turn; optional selection= (ModelSelectionV1) with strict_selection=True for request-scoped identity (M01).

> openai.chat_completions base_url=base_url) api_key=api_key) model=model) prompt=prompt messages=messages selection=selection strict_selection=strict_selection stream=stream echo=echo stream_kind=stream_kind suffix=suffix) bearer=bearer)

complete

+ `prompt`
+ `stream`=False
+ `echo`=False
+ `stream_kind`="delta"

Internal / compatibility primitive. Prefer ask / stream in new code.

  1. stream pack = > self.complete_result prompt=prompt stream=True echo=echo stream_kind=stream_kind _ret = > json.get value=pack key="events" _ret

pack = > self.complete_result prompt=prompt stream=False echo=echo stream_kind=stream_kind _ret = > json.get value=pack key="text" _ret

chat

+ `prompt`
+ `stream`=False
+ `echo`=False
+ `stream_kind`="delta"

Alias for complete (compat).

> self.complete prompt=prompt stream=stream echo=echo stream_kind=stream_kind