Add versioned C ABI for optional native plugins.

Load .dll/.so/.dylib at runtime via libloading, register JSON-wired functions, and ship a Rust cdylib demo with lib/plugin wrappers and gold coverage.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
cflmyandCursor committed 2026-08-06 12:45:18 +08:00
1 parent b00b018056
commit b0c777c480
26 files changed
+903 -26

No files matched your search

+1
View File
@@ -122,6 +122,7 @@ title: example
- Import **English** or **Chinese** library file; call that file’s API names (do not mix).
- `lib/…` resolves via `MARQDO_LIB`, cwd `lib/`, or `lib/` next to the `marqdo` binary.
- Official optional extensions: `ext/llm.mq.md` / `ext/大模型.mq.md` — `# llm` / `# 大模型` object handles + `## complete` / `## 运行`. Free `## load_env`. See `doc/design/ext-llm.md` and `doc/design/objects.md`.
- Native plugins (optional shared libs): `lib/plugin.mq.md` / `lib/插件.mq.md` — `## load` / `## unload` / `## list`; after load, registered names are callable directly. C ABI: `include/marqdo_abi.h`, design `doc/design/ext-abi.md`, demo `plugins/demo`.
- Builtins (no import): `print`/`打印`, `input`/`输入`, `len`/`长度`, `str`/`文本`, `int`/`整数`; literals `True`/`真`, `False`/`假`, `None`/`空`; logic `and`/`且`, `or`/`或`, `not`/`非`.
Stdlib map: [reference.md](reference.md).
+1
View File
@@ -41,6 +41,7 @@ Import one file; use **that** file’s function names.
| `lib/table.mq.md` | `lib/表.mq.md` | table/rows helpers |
| `lib/fs.mq.md` | `lib/文件.mq.md` | filesystem |
| `lib/sys.mq.md` | `lib/系统.mq.md` | process / cwd / load_dotenv |
| `lib/plugin.mq.md` | `lib/插件.mq.md` | load native ABI plugins (`doc/design/ext-abi.md`) |
| `lib/time.mq.md` | `lib/时间.mq.md` | time/format |
| `lib/net.mq.md` | `lib/网络.mq.md` | HTTP(S) helpers (+ optional headers) |
| `lib/json.mq.md` | (see design) | JSON (+ `quote`) |
+5
View File
@@ -1,5 +1,10 @@
/target/
/dist/
# Copied by gold harness for ABI plugin demo
/tests/lib/demo.dll
/tests/lib/libdemo.so
/tests/lib/libdemo.dylib
**/*.rs.bk
.idea/
.vscode/
Generated
+18
View File
@@ -418,6 +418,16 @@ version = "0.2.189"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2"
[[package]]
name = "libloading"
version = "0.8.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fc2f4eb4bc735547cfed7c0a4922cbd04a4655978c09b54f1f7b228750664c34"
dependencies = [
"cfg-if",
"windows-targets",
]
[[package]]
name = "log"
version = "0.4.33"
@@ -433,6 +443,7 @@ dependencies = [
"chrono",
"clap",
"idna_adapter",
"libloading",
"predicates",
"pulldown-cmark",
"serde_json",
@@ -442,6 +453,13 @@ dependencies = [
"zeroize",
]
[[package]]
name = "marqdo_plugin_demo"
version = "0.1.0"
dependencies = [
"serde_json",
]
[[package]]
name = "memchr"
version = "2.8.3"
+5
View File
@@ -23,6 +23,11 @@ tiny_http = "0.12"
ureq = { version = "=2.9.7", default-features = false, features = ["tls", "gzip"] }
idna_adapter = "=1.1.0"
zeroize = "=1.8.1"
libloading = "=0.8.6"
[workspace]
members = ["plugins/demo"]
resolver = "2"
[dev-dependencies]
assert_cmd = "=2.0.14"
+1 -1
View File
@@ -6,7 +6,7 @@ Marqdo 把 Markdown **标记当作编程语法**:同一个 `.mq.md` 文件,
欢迎访问 [marqdo 官方网站](https://www.marqdo.com/) 阅读更多特性与可执行文档。
语法宪法:[doc/design/markdown-mapping.md](doc/design/markdown-mapping.md) · 对象:[objects.md](doc/design/objects.md) · 调用实参:[call-arguments.md](doc/design/call-arguments.md) · 用户站:[user-site.md](doc/design/user-site.md) · 浏览:[view.md](doc/design/view.md) · 调试:[view-debug.md](doc/design/view-debug.md) · OKF 清单:[catalog-cli.md](doc/design/catalog-cli.md) · VS Code 扩展:[vscode-extension.md](doc/design/vscode-extension.md)(分支 `vscode-extension`) · **AI Skill**:[skills/marqdo/](skills/marqdo/)(说明 [ai-skill.md](doc/design/ai-skill.md)) · 官方扩展:[ext-llm.md](doc/design/ext-llm.md)(`ext/llm`) · 金样例:[tests/](tests/) · 用户文档:[public/](public/)
语法宪法:[doc/design/markdown-mapping.md](doc/design/markdown-mapping.md) · 对象:[objects.md](doc/design/objects.md) · 调用实参:[call-arguments.md](doc/design/call-arguments.md) · 用户站:[user-site.md](doc/design/user-site.md) · 浏览:[view.md](doc/design/view.md) · 调试:[view-debug.md](doc/design/view-debug.md) · OKF 清单:[catalog-cli.md](doc/design/catalog-cli.md) · VS Code 扩展:[vscode-extension.md](doc/design/vscode-extension.md)(分支 `vscode-extension`) · **AI Skill**:[skills/marqdo/](skills/marqdo/)(说明 [ai-skill.md](doc/design/ai-skill.md)) · 官方扩展:[ext-llm.md](doc/design/ext-llm.md)(`ext/llm`) · 原生插件 ABI:[ext-abi.md](doc/design/ext-abi.md)(`lib/plugin`) · 金样例:[tests/](tests/) · 用户文档:[public/](public/)
---
+65
View File
@@ -0,0 +1,65 @@
# Native plugin ABI (v1)
| | |
|---|---|
| Status | Accepted |
| Date | 2026-08-06 |
| Header | [`include/marqdo_abi.h`](../../include/marqdo_abi.h) |
## Goals
- Optional **native** extensions as shared libraries (`.dll` / `.so` / `.dylib`).
- **Not** linked into the `marqdo` binary; load at runtime.
- Stable **C** ABI only (no cross-compiler Rust ABI for plugins).
- Official / local paths only — **no** third-party package registry.
## Contract
`MARQDO_ABI_VERSION = 1`.
Every plugin exports:
| Symbol | Role |
|--------|------|
| `marqdo_plugin_abi_version` | Must equal host’s supported version |
| `marqdo_plugin_init` | Register functions via `MarqdoHostApi` |
| `marqdo_plugin_shutdown` | Teardown |
Host provides `register_fn(name, params_csv, fn)`, `alloc`, `free`.
### Wire format
Arguments and results are **UTF-8 JSON**:
- Args: JSON object keyed by parameter names (from `params` CSV at registration).
- Result: any JSON value mappable to Marqdo `Value` (same rules as `lib/json`).
Plugin fn returns `0` + `*out_json` on success; non-zero + optional `*err_msg` on failure. Host frees both strings with the host `free`.
## Marqdo surface
```markdown
---
> lib/plugin.mq.md
---
# main
> load path=demo.dll
*`sum` = > demo_add a=1 b=2 *
> print text=`sum`
```
After `load`, registered names are callable like other functions (plugin registry lookup after fixed host fns, before user `#`/`##` defs).
## Demo
Workspace crate `plugins/demo` (`marqdo_plugin_demo`) builds a cdylib registering `demo_add` and `demo_echo`.
## Non-goals
- Embedding plugins in the release exe
- Callbacks from plugin into arbitrary Marqdo evaluation
- OS sandbox / seccomp
- Auto-download of plugins
+1 -1
View File
@@ -4,7 +4,7 @@
|---|---|
| Status | Accepted (v0.1.x) |
| Date | 2026-08-06 |
| Related | [objects.md](objects.md) |
| Related | [objects.md](objects.md) · native plugins later → [ext-abi.md](ext-abi.md) |
## Scope
+43
View File
@@ -0,0 +1,43 @@
/* Marqdo plugin ABI v1 — C contract (source of truth).
*
* Plugins are optional shared libraries (.dll / .so / .dylib), never linked
* into the marqdo binary. Values cross the boundary as UTF-8 JSON.
*/
#ifndef MARQDO_ABI_H
#define MARQDO_ABI_H
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
#define MARQDO_ABI_VERSION 1u
/* Plugin function: args_json is a UTF-8 JSON object whose keys are param names.
* On success: return 0, set *out_json to a heap string (host frees via free).
* On failure: return non-zero; optional *err_msg heap string (host frees).
* out_json / err_msg may be left NULL.
*/
typedef int (*MarqdoPluginFn)(const char *args_json, char **out_json, char **err_msg);
typedef struct MarqdoHostApi {
void *userdata;
/* params: comma-separated param names in order, e.g. "a,b". May be "". */
int (*register_fn)(void *userdata, const char *name, const char *params,
MarqdoPluginFn fn);
void *(*alloc)(size_t n);
void (*free)(void *p);
} MarqdoHostApi;
/* Required exports from every plugin shared library. */
uint32_t marqdo_plugin_abi_version(void);
int marqdo_plugin_init(const MarqdoHostApi *host);
void marqdo_plugin_shutdown(void);
#ifdef __cplusplus
}
#endif
#endif /* MARQDO_ABI_H */
+17
View File
@@ -0,0 +1,17 @@
---
title: lib/plugin
description: Load optional native ABI plugins (.dll / .so / .dylib)
---
## load
- path
**> host_plugin_load path=`path`**
## unload
**> host_plugin_unload**
## list
**> host_plugin_list**
+17
View File
@@ -0,0 +1,17 @@
---
title: lib/插件
description: 加载可选原生 ABI 插件(.dll / .so / .dylib)
---
## 加载
- 路径
**> host_plugin_load path=`路径`**
## 卸载
**> host_plugin_unload**
## 列出
**> host_plugin_list**
+14
View File
@@ -0,0 +1,14 @@
[package]
name = "marqdo_plugin_demo"
version = "0.1.0"
edition = "2021"
license = "Apache-2.0"
description = "Demo Marqdo ABI v1 plugin (cdylib)"
publish = false
[lib]
name = "demo"
crate-type = ["cdylib"]
[dependencies]
serde_json = "1"
+170
View File
@@ -0,0 +1,170 @@
//! Demo Marqdo plugin (C ABI v1). Registers `demo_add` and `demo_echo`.
use std::ffi::{CStr, CString};
use std::os::raw::{c_char, c_int, c_void};
use std::ptr;
const ABI_VERSION: u32 = 1;
type PluginFn = unsafe extern "C" fn(
args_json: *const c_char,
out_json: *mut *mut c_char,
err_msg: *mut *mut c_char,
) -> c_int;
#[repr(C)]
pub struct MarqdoHostApi {
pub userdata: *mut c_void,
pub register_fn: Option<
unsafe extern "C" fn(
userdata: *mut c_void,
name: *const c_char,
params: *const c_char,
fn_ptr: PluginFn,
) -> c_int,
>,
pub alloc: Option<unsafe extern "C" fn(n: usize) -> *mut c_void>,
pub free: Option<unsafe extern "C" fn(p: *mut c_void)>,
}
static mut HOST_FREE: Option<unsafe extern "C" fn(*mut c_void)> = None;
static mut HOST_ALLOC: Option<unsafe extern "C" fn(usize) -> *mut c_void> = None;
unsafe fn host_strdup(s: &str) -> *mut c_char {
let alloc = HOST_ALLOC.expect("host alloc");
let bytes = s.as_bytes();
let p = alloc(bytes.len() + 1) as *mut u8;
if p.is_null() {
return ptr::null_mut();
}
std::ptr::copy_nonoverlapping(bytes.as_ptr(), p, bytes.len());
*p.add(bytes.len()) = 0;
p as *mut c_char
}
fn set_out(out: *mut *mut c_char, s: &str) {
if out.is_null() {
return;
}
unsafe {
*out = host_strdup(s);
}
}
fn set_err(err: *mut *mut c_char, s: &str) {
if err.is_null() {
return;
}
unsafe {
*err = host_strdup(s);
}
}
unsafe extern "C" fn demo_add(
args_json: *const c_char,
out_json: *mut *mut c_char,
err_msg: *mut *mut c_char,
) -> c_int {
if args_json.is_null() {
set_err(err_msg, "demo_add: null args");
return 1;
}
let s = match CStr::from_ptr(args_json).to_str() {
Ok(s) => s,
Err(_) => {
set_err(err_msg, "demo_add: args not utf-8");
return 1;
}
};
let v: serde_json::Value = match serde_json::from_str(s) {
Ok(v) => v,
Err(e) => {
set_err(err_msg, &format!("demo_add: {e}"));
return 1;
}
};
let a = v.get("a").and_then(|x| x.as_i64()).unwrap_or(0);
let b = v.get("b").and_then(|x| x.as_i64()).unwrap_or(0);
set_out(out_json, &(a + b).to_string());
0
}
unsafe extern "C" fn demo_echo(
args_json: *const c_char,
out_json: *mut *mut c_char,
err_msg: *mut *mut c_char,
) -> c_int {
if args_json.is_null() {
set_err(err_msg, "demo_echo: null args");
return 1;
}
let s = match CStr::from_ptr(args_json).to_str() {
Ok(s) => s,
Err(_) => {
set_err(err_msg, "demo_echo: args not utf-8");
return 1;
}
};
let v: serde_json::Value = match serde_json::from_str(s) {
Ok(v) => v,
Err(e) => {
set_err(err_msg, &format!("demo_echo: {e}"));
return 1;
}
};
let text = v
.get("text")
.and_then(|x| x.as_str())
.unwrap_or("");
let out = serde_json::Value::String(text.to_string()).to_string();
set_out(out_json, &out);
0
}
#[no_mangle]
pub unsafe extern "C" fn marqdo_plugin_abi_version() -> u32 {
ABI_VERSION
}
#[no_mangle]
pub unsafe extern "C" fn marqdo_plugin_init(host: *const MarqdoHostApi) -> c_int {
if host.is_null() {
return 1;
}
let host = &*host;
HOST_ALLOC = host.alloc;
HOST_FREE = host.free;
let register = match host.register_fn {
Some(f) => f,
None => return 1,
};
let name_add = CString::new("demo_add").unwrap();
let params_add = CString::new("a,b").unwrap();
if register(
host.userdata,
name_add.as_ptr(),
params_add.as_ptr(),
demo_add,
) != 0
{
return 1;
}
let name_echo = CString::new("demo_echo").unwrap();
let params_echo = CString::new("text").unwrap();
if register(
host.userdata,
name_echo.as_ptr(),
params_echo.as_ptr(),
demo_echo,
) != 0
{
return 1;
}
0
}
#[no_mangle]
pub unsafe extern "C" fn marqdo_plugin_shutdown() {
HOST_ALLOC = None;
HOST_FREE = None;
}
+1
View File
@@ -122,6 +122,7 @@ title: example
- Import **English** or **Chinese** library file; call that file’s API names (do not mix).
- `lib/…` resolves via `MARQDO_LIB`, cwd `lib/`, or `lib/` next to the `marqdo` binary.
- Official optional extensions: `ext/llm.mq.md` / `ext/大模型.mq.md` — `# llm` / `# 大模型` object handles + `## complete` / `## 运行`. Free `## load_env`. See `doc/design/ext-llm.md` and `doc/design/objects.md`.
- Native plugins (optional shared libs): `lib/plugin.mq.md` / `lib/插件.mq.md` — `## load` / `## unload` / `## list`; after load, registered names are callable directly. C ABI: `include/marqdo_abi.h`, design `doc/design/ext-abi.md`, demo `plugins/demo`.
- Builtins (no import): `print`/`打印`, `input`/`输入`, `len`/`长度`, `str`/`文本`, `int`/`整数`; literals `True`/`真`, `False`/`假`, `None`/`空`; logic `and`/`且`, `or`/`或`, `not`/`非`.
Stdlib map: [reference.md](reference.md).
+1
View File
@@ -41,6 +41,7 @@ Import one file; use **that** file’s function names.
| `lib/table.mq.md` | `lib/表.mq.md` | table/rows helpers |
| `lib/fs.mq.md` | `lib/文件.mq.md` | filesystem |
| `lib/sys.mq.md` | `lib/系统.mq.md` | process / cwd / load_dotenv |
| `lib/plugin.mq.md` | `lib/插件.mq.md` | load native ABI plugins (`doc/design/ext-abi.md`) |
| `lib/time.mq.md` | `lib/时间.mq.md` | time/format |
| `lib/net.mq.md` | `lib/网络.mq.md` | HTTP(S) helpers (+ optional headers) |
| `lib/json.mq.md` | (see design) | JSON (+ `quote`) |
+67 -21
View File
@@ -512,8 +512,54 @@ impl<'a> FnCompiler<'a> {
_ => {}
}
let fid = self.resolve_call(&call.callee)?;
let params = self.flat[fid].params.clone();
match self.resolve_call(&call.callee) {
Ok(fid) => {
let params = self.flat[fid].params.clone();
let mut named: HashMap<String, &Expr> = HashMap::new();
let mut positionals = Vec::new();
for a in &call.args {
match a {
Arg::Positional(e) => positionals.push(e),
Arg::Named { name, value } => {
named.insert(name.clone(), value);
}
}
}
let mut pos_i = 0;
for p in &params {
if let Some(e) = named.get(p) {
self.compile_expr(e)?;
} else if pos_i < positionals.len() {
self.compile_expr(positionals[pos_i])?;
pos_i += 1;
} else {
return Err(self.err(format!("missing argument for parameter `{p}`")));
}
}
if pos_i < positionals.len() {
return Err(self.err(format!(
"too many positional arguments ({} extra)",
positionals.len() - pos_i
)));
}
self.emit(Op::Call(fid as u16, params.len() as u8));
if self.flat[fid].parent.is_none() && self.flat[fid].level == 1 {
let ti = self.add_const(Value::Text(self.flat[fid].name.clone()));
self.emit(Op::TagInstance(ti));
}
if as_stmt {
self.emit(Op::Pop);
}
Ok(())
}
Err(_) => self.compile_plugin_call(&call, as_stmt),
}
}
/// Unresolved callee → runtime plugin registry (or unknown-fn error).
/// Named args: stack pairs (name, value)×N, argc = 2N, name const prefixed with `@`.
/// Positional: stack values, argc = N, name const as-is.
fn compile_plugin_call(&mut self, call: &CallExpr, as_stmt: bool) -> Result<()> {
let mut named: HashMap<String, &Expr> = HashMap::new();
let mut positionals = Vec::new();
for a in &call.args {
@@ -524,27 +570,27 @@ impl<'a> FnCompiler<'a> {
}
}
}
let mut pos_i = 0;
for p in &params {
if let Some(e) = named.get(p) {
if !named.is_empty() && !positionals.is_empty() {
return Err(self.err(
"plugin/unknown calls cannot mix positional and named args in bytecode",
));
}
if !named.is_empty() {
let mut pairs: Vec<_> = named.into_iter().collect();
pairs.sort_by(|a, b| a.0.cmp(&b.0));
for (k, e) in &pairs {
let ki = self.add_const(Value::Text(k.clone()));
self.emit(Op::Constant(ki));
self.compile_expr(e)?;
} else if pos_i < positionals.len() {
self.compile_expr(positionals[pos_i])?;
pos_i += 1;
} else {
return Err(self.err(format!("missing argument for parameter `{p}`")));
}
}
if pos_i < positionals.len() {
return Err(self.err(format!(
"too many positional arguments ({} extra)",
positionals.len() - pos_i
)));
}
self.emit(Op::Call(fid as u16, params.len() as u8));
if self.flat[fid].parent.is_none() && self.flat[fid].level == 1 {
let ti = self.add_const(Value::Text(self.flat[fid].name.clone()));
self.emit(Op::TagInstance(ti));
let ni = self.add_const(Value::Text(format!("@{}", call.callee)));
self.emit(Op::PluginCall(ni, (pairs.len() * 2) as u8));
} else {
for e in &positionals {
self.compile_expr(e)?;
}
let ni = self.add_const(Value::Text(call.callee.clone()));
self.emit(Op::PluginCall(ni, positionals.len() as u8));
}
if as_stmt {
self.emit(Op::Pop);
+2
View File
@@ -59,6 +59,8 @@ pub enum Op {
/// Method call: stack has argc args (param order) then receiver on top.
/// constants[name_idx] is method name text; resolve via receiver `_type`.
MethodCall(u16, u8),
/// Plugin call: stack has argc args in registered param order; name in constants.
PluginCall(u16, u8),
/// Host primitive: pop `argc` args (param order), push result.
HostCall(u16, u8),
Return,
+62
View File
@@ -399,6 +399,68 @@ impl Vm {
slots,
});
}
Op::PluginCall(name_idx, argc) => {
let argc = argc as usize;
let raw_name = match fun.constants.get(name_idx as usize) {
Some(Value::Text(s)) => s.clone(),
_ => {
return Err(self.err_at(span, "PluginCall needs name constant"));
}
};
let mut bound = HashMap::new();
if let Some(name) = raw_name.strip_prefix('@') {
// named pairs: (name, value) × N, argc = 2N
if argc % 2 != 0 {
return Err(self.err_at(span, "PluginCall named argc must be even"));
}
let pairs = argc / 2;
for _ in 0..pairs {
let v = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
let k = pop(&mut stack).map_err(|m| self.err_at(span, m))?;
let ks = match k {
Value::Text(s) => s,
_ => {
return Err(self.err_at(span, "plugin arg name must be text"));
}
};
bound.insert(ks, v);
}
let result = crate::host::plugin::call_registered(&self.host, name, &bound)
.map_err(|m| self.err_at(span, m))?;
stack.push(result);
} else {
let mut vals = Vec::with_capacity(argc);
for _ in 0..argc {
vals.push(pop(&mut stack).map_err(|m| self.err_at(span, m))?);
}
vals.reverse();
let reg = self
.host
.plugins
.get(&raw_name)
.cloned()
.ok_or_else(|| {
self.err_at(span, format!("unknown function `{raw_name}`"))
})?;
if reg.params.len() != vals.len() {
return Err(self.err_at(
span,
format!(
"plugin `{raw_name}` expects {} args, got {}",
reg.params.len(),
vals.len()
),
));
}
for (p, v) in reg.params.iter().zip(vals.into_iter()) {
bound.insert(p.clone(), v);
}
let result =
crate::host::plugin::call_registered(&self.host, &raw_name, &bound)
.map_err(|m| self.err_at(span, m))?;
stack.push(result);
}
}
Op::Return => {
let ret = pop(&mut stack).unwrap_or(Value::None);
frames.pop();
+18 -1
View File
@@ -2,7 +2,7 @@
use std::collections::HashMap;
use crate::host::{foreign, fs, json, math, net, sys, time, HostContext};
use crate::host::{foreign, fs, json, math, net, plugin, sys, time, HostContext};
use crate::value::Value;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
@@ -77,6 +77,9 @@ pub enum HostFn {
DotenvLoad = 66,
HttpRequest = 67,
JsonQuote = 68,
PluginLoad = 69,
PluginUnload = 70,
PluginList = 71,
}
impl HostFn {
@@ -150,6 +153,9 @@ impl HostFn {
66 => Self::DotenvLoad,
67 => Self::HttpRequest,
68 => Self::JsonQuote,
69 => Self::PluginLoad,
70 => Self::PluginUnload,
71 => Self::PluginList,
_ => return None,
})
}
@@ -227,6 +233,9 @@ impl HostFn {
"host_dotenv_load" | "load_dotenv" => Self::DotenvLoad,
"host_http_request" | "http_request" => Self::HttpRequest,
"host_json_quote" | "quote" => Self::JsonQuote,
"host_plugin_load" | "plugin_load" => Self::PluginLoad,
"host_plugin_unload" | "plugin_unload" => Self::PluginUnload,
"host_plugin_list" | "plugin_list" => Self::PluginList,
_ => return None,
})
}
@@ -305,6 +314,9 @@ impl HostFn {
Self::DotenvLoad => "host_dotenv_load",
Self::HttpRequest => "host_http_request",
Self::JsonQuote => "host_json_quote",
Self::PluginLoad => "host_plugin_load",
Self::PluginUnload => "host_plugin_unload",
Self::PluginList => "host_plugin_list",
}
}
@@ -347,6 +359,8 @@ impl HostFn {
Self::ForeignRun => &["code"],
Self::ForeignRunLang => &["lang", "source"],
Self::ForeignLangs => &[],
Self::PluginLoad => &["path"],
Self::PluginUnload | Self::PluginList => &[],
}
}
@@ -522,6 +536,9 @@ pub fn call_host(
bound.get("stdin"),
),
HostFn::ForeignLangs => foreign::langs(ctx),
HostFn::PluginLoad => plugin::load(ctx, require(bound, "path")?),
HostFn::PluginUnload => plugin::unload(ctx),
HostFn::PluginList => plugin::list(ctx),
}
}
+2 -2
View File
@@ -64,7 +64,7 @@ pub fn quote(text: &Value) -> Result<Value, String> {
.map_err(|e| format!("json quote: {e}"))
}
fn json_to_value(v: &serde_json::Value) -> Result<Value, String> {
pub(crate) fn json_to_value(v: &serde_json::Value) -> Result<Value, String> {
Ok(match v {
serde_json::Value::Null => Value::None,
serde_json::Value::Bool(b) => Value::Bool(*b),
@@ -95,7 +95,7 @@ fn json_to_value(v: &serde_json::Value) -> Result<Value, String> {
})
}
fn value_to_json(v: &Value) -> Result<serde_json::Value, String> {
pub(crate) fn value_to_json(v: &Value) -> Result<serde_json::Value, String> {
Ok(match v {
Value::None => serde_json::Value::Null,
Value::Bool(b) => serde_json::Value::Bool(*b),
+11
View File
@@ -10,11 +10,13 @@ mod fs;
mod json;
pub mod math;
mod net;
pub mod plugin;
mod sys;
mod time;
pub use dispatch::{call_host, HostFn};
pub use fs::path_under_root;
pub use plugin::PluginState;
use std::collections::HashMap;
use std::path::{Path, PathBuf};
@@ -25,6 +27,7 @@ pub struct HostCaps {
pub fs_write: bool,
pub exec: bool,
pub net: bool,
pub plugin: bool,
}
impl Default for HostCaps {
@@ -33,6 +36,7 @@ impl Default for HostCaps {
fs_write: true,
exec: true,
net: true,
plugin: true,
}
}
}
@@ -59,6 +63,8 @@ pub struct HostContext {
pub plots: Vec<PlotArtifact>,
/// lang → interpreter argv (from `set_cmd`).
pub foreign_cmds: HashMap<String, Vec<String>>,
/// Loaded native plugins (ABI v1).
pub plugins: PluginState,
}
impl Default for HostContext {
@@ -73,6 +79,7 @@ impl Default for HostContext {
rng: 0xC0FF_EE42_CAFE_BABE,
plots: Vec::new(),
foreign_cmds: HashMap::new(),
plugins: PluginState::default(),
}
}
}
@@ -116,6 +123,10 @@ impl HostContext {
self.caps.net
}
pub fn allow_plugin(&self) -> bool {
self.caps.plugin
}
pub fn next_u64(&mut self) -> u64 {
// SplitMix64-ish LCG step
self.rng = self
+311
View File
@@ -0,0 +1,311 @@
//! Native plugin loader (C ABI v1). Shared libs are not linked into marqdo.
use std::collections::HashMap;
use std::ffi::{CStr, CString};
use std::os::raw::{c_char, c_int, c_void};
use libloading::{Library, Symbol};
use crate::host::json::{json_to_value, value_to_json};
use crate::host::HostContext;
use crate::value::Value;
pub const ABI_VERSION: u32 = 1;
type PluginFn = unsafe extern "C" fn(
args_json: *const c_char,
out_json: *mut *mut c_char,
err_msg: *mut *mut c_char,
) -> c_int;
type AbiVersionFn = unsafe extern "C" fn() -> u32;
type InitFn = unsafe extern "C" fn(host: *const MarqdoHostApi) -> c_int;
type ShutdownFn = unsafe extern "C" fn();
#[repr(C)]
struct MarqdoHostApi {
userdata: *mut c_void,
register_fn: Option<
unsafe extern "C" fn(
userdata: *mut c_void,
name: *const c_char,
params: *const c_char,
fn_ptr: PluginFn,
) -> c_int,
>,
alloc: Option<unsafe extern "C" fn(n: usize) -> *mut c_void>,
free: Option<unsafe extern "C" fn(p: *mut c_void)>,
}
struct RegisterBuf {
fns: HashMap<String, RegisteredFn>,
error: Option<String>,
}
#[derive(Clone)]
pub struct RegisteredFn {
pub params: Vec<String>,
fn_ptr: PluginFn,
}
pub struct PluginState {
libs: Vec<Library>,
fns: HashMap<String, RegisteredFn>,
}
impl Default for PluginState {
fn default() -> Self {
Self {
libs: Vec::new(),
fns: HashMap::new(),
}
}
}
impl std::fmt::Debug for PluginState {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("PluginState")
.field("libs", &self.libs.len())
.field("fns", &self.list_names())
.finish()
}
}
impl Clone for PluginState {
fn clone(&self) -> Self {
// `Library` handles are not cloneable; a cloned host starts with no plugins.
Self::default()
}
}
impl PluginState {
pub fn get(&self, name: &str) -> Option<&RegisteredFn> {
self.fns.get(name)
}
pub fn list_names(&self) -> Vec<String> {
let mut names: Vec<_> = self.fns.keys().cloned().collect();
names.sort();
names
}
}
fn as_text<'a>(v: &'a Value, label: &str) -> Result<&'a str, String> {
match v {
Value::Text(s) => Ok(s.as_str()),
_ => Err(format!("{label} must be text")),
}
}
extern "C" {
fn malloc(n: usize) -> *mut c_void;
fn free(p: *mut c_void);
}
unsafe extern "C" fn host_alloc(n: usize) -> *mut c_void {
malloc(n)
}
unsafe extern "C" fn host_free(p: *mut c_void) {
if !p.is_null() {
free(p);
}
}
unsafe extern "C" fn host_register(
userdata: *mut c_void,
name: *const c_char,
params: *const c_char,
fn_ptr: PluginFn,
) -> c_int {
if userdata.is_null() || name.is_null() {
return 1;
}
let buf = &mut *(userdata as *mut RegisterBuf);
let name = match CStr::from_ptr(name).to_str() {
Ok(s) => s.to_string(),
Err(_) => {
buf.error = Some("plugin register: name is not UTF-8".into());
return 1;
}
};
if name.is_empty() {
buf.error = Some("plugin register: empty name".into());
return 1;
}
let params_s = if params.is_null() {
""
} else {
match CStr::from_ptr(params).to_str() {
Ok(s) => s,
Err(_) => {
buf.error = Some("plugin register: params is not UTF-8".into());
return 1;
}
}
};
let param_list: Vec<String> = if params_s.trim().is_empty() {
Vec::new()
} else {
params_s
.split(',')
.map(|p| p.trim().to_string())
.filter(|p| !p.is_empty())
.collect()
};
if buf.fns.contains_key(&name) {
buf.error = Some(format!("plugin register: duplicate `{name}`"));
return 1;
}
buf.fns.insert(
name,
RegisteredFn {
params: param_list,
fn_ptr,
},
);
0
}
pub fn load(ctx: &mut HostContext, path: &Value) -> Result<Value, String> {
if !ctx.allow_plugin() {
return Err("plugin_load denied by host policy".into());
}
let rel = as_text(path, "path")?;
let file = ctx.resolve_path(rel)?;
if !file.is_file() {
return Err(format!("plugin_load: not a file: {}", file.display()));
}
let lib = unsafe { Library::new(&file) }
.map_err(|e| format!("plugin_load {}: {e}", file.display()))?;
let version_fn: Symbol<AbiVersionFn> = unsafe {
lib.get(b"marqdo_plugin_abi_version\0")
.map_err(|e| format!("plugin missing marqdo_plugin_abi_version: {e}"))?
};
let ver = unsafe { version_fn() };
if ver != ABI_VERSION {
return Err(format!(
"plugin ABI version {ver} != host {ABI_VERSION} ({})",
file.display()
));
}
let init_fn: Symbol<InitFn> = unsafe {
lib.get(b"marqdo_plugin_init\0")
.map_err(|e| format!("plugin missing marqdo_plugin_init: {e}"))?
};
let mut buf = RegisterBuf {
fns: HashMap::new(),
error: None,
};
let api = MarqdoHostApi {
userdata: &mut buf as *mut RegisterBuf as *mut c_void,
register_fn: Some(host_register),
alloc: Some(host_alloc),
free: Some(host_free),
};
let rc = unsafe { init_fn(&api) };
if rc != 0 {
return Err(buf
.error
.unwrap_or_else(|| format!("plugin init failed (code {rc})")));
}
if let Some(e) = buf.error {
return Err(e);
}
for k in buf.fns.keys() {
if ctx.plugins.fns.contains_key(k) {
return Err(format!("plugin function `{k}` already registered"));
}
}
let added = buf.fns.len();
for (k, v) in buf.fns {
ctx.plugins.fns.insert(k, v);
}
ctx.plugins.libs.push(lib);
Ok(Value::Int(added as i64))
}
pub fn unload(ctx: &mut HostContext) -> Result<Value, String> {
if !ctx.allow_plugin() {
return Err("plugin_unload denied by host policy".into());
}
for lib in ctx.plugins.libs.drain(..) {
if let Ok(shutdown) = unsafe { lib.get::<ShutdownFn>(b"marqdo_plugin_shutdown\0") } {
unsafe { shutdown() };
}
drop(lib);
}
ctx.plugins.fns.clear();
Ok(Value::None)
}
pub fn list(ctx: &HostContext) -> Result<Value, String> {
Ok(Value::List(
ctx.plugins
.list_names()
.into_iter()
.map(Value::Text)
.collect(),
))
}
/// Call a registered plugin function with already-bound args (param name → Value).
pub fn call_registered(
ctx: &HostContext,
name: &str,
bound: &HashMap<String, Value>,
) -> Result<Value, String> {
if !ctx.allow_plugin() {
return Err("plugin call denied by host policy".into());
}
let reg = ctx
.plugins
.get(name)
.ok_or_else(|| format!("unknown function `{name}`"))?;
let mut map = serde_json::Map::new();
for p in &reg.params {
let v = bound.get(p).unwrap_or(&Value::None);
map.insert(p.clone(), value_to_json(v)?);
}
for (k, v) in bound {
if !map.contains_key(k) {
map.insert(k.clone(), value_to_json(v)?);
}
}
let args_json = serde_json::Value::Object(map).to_string();
let c_args = CString::new(args_json).map_err(|e| format!("plugin args: {e}"))?;
let mut out_ptr: *mut c_char = std::ptr::null_mut();
let mut err_ptr: *mut c_char = std::ptr::null_mut();
let rc = unsafe { (reg.fn_ptr)(c_args.as_ptr(), &mut out_ptr, &mut err_ptr) };
let err_s = take_c_string(err_ptr);
let out_s = take_c_string(out_ptr);
if rc != 0 {
return Err(err_s.unwrap_or_else(|| format!("plugin `{name}` failed (code {rc})")));
}
let out_s = out_s.unwrap_or_else(|| "null".to_string());
let j: serde_json::Value =
serde_json::from_str(&out_s).map_err(|e| format!("plugin `{name}` bad JSON result: {e}"))?;
json_to_value(&j)
}
fn take_c_string(p: *mut c_char) -> Option<String> {
if p.is_null() {
return None;
}
let s = unsafe { CStr::from_ptr(p) }
.to_str()
.map(|s| s.to_string())
.ok();
unsafe { host_free(p as *mut c_void) };
s
}
+6
View File
@@ -568,6 +568,12 @@ impl Interpreter {
_ => {}
}
if let Some(reg) = self.host.plugins.get(&call.callee).cloned() {
let bound = bind_args(&reg.params, &ev_args, false).map_err(|m| self.err(m))?;
return crate::host::plugin::call_registered(&self.host, &call.callee, &bound)
.map_err(|m| self.err(m));
}
let target = lookup_function(module, fun, &call.callee)
.ok_or_else(|| self.err(format!("unknown function `{}`", call.callee)))?;
+1
View File
@@ -122,6 +122,7 @@ impl RunOptions {
fs_write: self.allow_fs_write,
exec: self.allow_exec,
net: self.allow_net,
plugin: true,
}
}
}
+43
View File
@@ -595,6 +595,49 @@ fn lib_json_quote() {
assert_out("tests/lib/json-quote.mq.md", "\"hi\"");
}
#[test]
fn lib_plugin_demo() {
let status = Command::new(env!("CARGO"))
.args(["build", "-p", "marqdo_plugin_demo"])
.status()
.expect("cargo build plugin demo");
assert!(status.success(), "failed to build marqdo_plugin_demo");
let lib_name = if cfg!(windows) {
"demo.dll"
} else if cfg!(target_os = "macos") {
"libdemo.dylib"
} else {
"libdemo.so"
};
let built = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("target")
.join("debug")
.join(lib_name);
assert!(
built.is_file(),
"missing plugin artifact {}",
built.display()
);
// Must live under the program's fs_root (`tests/lib/`) for path sandbox.
let plugin = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests")
.join("lib")
.join(lib_name);
std::fs::copy(&built, &plugin).expect("copy demo plugin into tests/lib");
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(["run", "tests/lib/plugin-demo.mq.md"])
.env("MARQDO_TEST_PLUGIN", lib_name)
.output()
.expect("failed to run marqdo");
let code = output.status.code().unwrap_or(1);
let stdout = String::from_utf8_lossy(&output.stdout);
let stderr = String::from_utf8_lossy(&output.stderr);
assert_eq!(code, 0, "plugin-demo stderr={stderr}");
assert_eq!(stdout.trim_end(), "3\nok", "plugin-demo");
}
#[test]
fn ext_llm_import() {
assert_out("tests/ext/llm-import.mq.md", "ext-ok");
+20
View File
@@ -0,0 +1,20 @@
---
title: plugin demo ABI
description: Loads demo cdylib; path from MARQDO_TEST_PLUGIN (set by gold harness).
> lib/plugin.mq.md
> lib/sys.mq.md
---
# main
*`p` = > env_get name=MARQDO_TEST_PLUGIN *
> load path=`p`
*`sum` = > demo_add a=1 b=2 *
> print text=`sum`
*`echo` = > demo_echo text=ok *
> print text=`echo`