Alias writeback as wb so the writeback= param does not shadow lib calls; ignore unknown stream event types so live SSE models keep working. Co-authored-by: Cursor <cursoragent@cursor.com>
2.9 KiB
title: ext/ai/llm description: OpenAI-compatible LLM object (English). Import ext/ai/llm.mq.md. import sys:lib/sys.mq.md import net:lib/net.mq.md import json:lib/json.mq.md import table:lib/table.mq.md
load_env
+ `path`=None
Load .env from cwd (optional named arg path=). Does not override existing variables.
> sys.load_dotenv path=path
stream_result
+ `events`
Reduce a complete stream=True event list to the final done.result text (or error.message). Unknown types (e.g. reasoning) are ignored.
answer = ""
- ev
t = ev[^type]
t== "done" answer = ev[^result]t== "error" answer = ev[^message]t== "delta"-
answer
llm
Construct an LLM handle from OPENAI_* / MARQDO_LLM_*.
api_key = > sys.env_get name="OPENAI_API_KEY"
-
not
api_keyapi_key = > sys.env_get name="MARQDO_LLM_API_KEY" -
not
api_key
print text=ext/ai/llm: set OPENAI_API_KEY or MARQDO_LLM_API_KEY sys.exit code=1
base_url = > sys.env_get name="OPENAI_BASE_URL"
- not
base_urlbase_url = > sys.env_get name="MARQDO_LLM_BASE_URL" - not
base_urlbase_url = "https://api.openai.com/v1"
model = > sys.env_get name="OPENAI_MODEL"
- not
modelmodel = > sys.env_get name="MARQDO_LLM_MODEL" - not
modelmodel = "gpt-4o-mini"
h =
| api_key | base_url | model | suffix | bearer |
|---|---|---|---|---|
api_key |
base_url |
model |
/chat/completions | "Bearer " |
h
complete
+ `prompt`
+ `stream`=False
+ `echo`=False
Chat completion using self handle fields.
Wire body is {model, messages:[{role,content}], stream?}. Messages are an @ row table; headers and optional stream use table.put. json is only for stringify/parse.
url = self[^base_url] + self[^suffix]
auth = self[^bearer] + self[^api_key]
headers = > table.put in=None at="Authorization" value=auth
messages =
| @ | role | content |
|---|---|---|
| 1 | user | prompt |
req =
| model | messages |
|---|---|
| self[^model] | messages |
streamreq = > table.put in=reqat="stream" value=True body = > json.stringify value=reqresp = > net.http_post_sse url=urlbody=bodyheaders=headersecho=echo- resp[^status] == 200 resp[^events]
-
> print text=ext/ai/llm: HTTP error (stream)
> print text=resp[^status]
> sys.exit code=1
body = > json.stringify value=req
resp = > net.http_post url=url body=body headers=headers
- resp[^status] == 200 data = > json.parse text=resp[^body] data[^choices][^1][^message][^content]
-
> print text=ext/ai/llm: HTTP error
> print text=resp[^status]
> print text=resp[^body]
> sys.exit code=1
chat
+ `prompt`
+ `stream`=False
+ `echo`=False
Alias for complete.
> self.complete prompt=prompt stream=stream echo=echo