120 Commits
Author SHA1 Message Date
cflmy 6f04c17f7a feat(cli,package): add/remove dependencies and cleaner manifest serialization
- `marqdo add <name> --path <dir>` / `--git URL --rev SHA` and
  `marqdo remove <name>` edit marqdo.yaml only after validating the dependency
  source (path-relative, git pinned by exact rev, no duplicate entry).
  Removing a dependency also drops its stale lock entry; locking stays an
  explicit step so no version changes happen silently.
- Manifest serialization omits empty dependency/plugin maps and empty
  capability requests, keeping generated manifests readable and stable.
2026-11-20 10:45:07 +08:00
cflmy 3d1c37d217 fix(markdown): flatten AST kind into node JSON objects
`inspect --ast` and AST goldens now show `{"kind":"heading","level":2,...}`
instead of a nested `{"kind":{"kind":"heading",...}}` object, matching
spec/markdown-profile.md section 5 and the shape new contributors expect.
2026-11-20 10:44:14 +08:00
cflmy e1ba630068 feat(cli): marqdo init/run/call/check/inspect/fmt/plugin/lock/sync
Composition root wiring the frontend, runtime, host and package layers:

- Project loader: deterministic project-root discovery, explicit front-matter
  imports only (`@dependency` or relative `.mq.md`), recursive module loading
  with cycle diagnostics, resolved import edges and cross-module call checks.
  Loading never executes anything.
- run/call: sandbox mode enforcement (strict refuses when unavailable),
  explicit capability grants, fuel/stack/output/timeout overrides, typed
  `--arg name=value` constants, program stdout separated from diagnostics, and
  machine-readable JSON output that embeds program output without mixing.
- check: static diagnostics only, human-readable locations with source lines
  and stable machine codes in JSON.
- inspect: canonical AST, heading units, signatures (explicit/inferred),
  executable IR, static reference edges (including cross-file links analysed
  without execution) and platform capability report.
- fmt: conservative, AST-preserving normalisation of line endings and trailing
  newlines; --check mode.
- plugin inspect: manifest/ABI/artifact validation and hash verification,
  never loading plugin code.
- init: minimal project (manifest, entry document, .gitignore).
- lock/sync --locked: deterministic path-dependency lockfile, stale lock or
  changed dependency content is an error (no silent upgrades).
- examples: hello, factorial, math formula, Python foreign call, static
  references and a two-module project with an explicit import.
2026-11-20 10:44:03 +08:00
cflmy 2e49c7b855 feat(runtime): tree-walk interpreter for Marqdo IR
Blueprint sections 7.2 and 3: Marqdo executes with its own Rust interpreter.

- Frames with isolated scopes, named/positional argument binding, explicit
  signature rules, compile-time constant defaults evaluated in an empty frame.
- Statements: assignment, expression statements, link-form calls, branch
  (first true arm), loop (condition re-evaluated, return propagates), return.
- Expressions: exact Int arithmetic, `/` always Float, mixed arithmetic Float,
  exact Int/Float comparisons via rationals, Text concatenation, short-circuit
  and/or, Bool-only conditions, indexes with negative positions, map/Formula/
  Code member access, calls (same module, imports, FunctionRef values).
- Builtins: print/len/type/str/int/float/table/ref plus capability-host
  dispatch for math.*, foreign.*, fs.*; `ref` resolves functions or exactly one
  inert resource and errors on ambiguity instead of picking the first.
- Fuel, stack-depth and wall-clock budgets are enforced; call errors carry a
  structured stack of frames.
- End-to-end tests: recursion, loops, explicit positional signatures, builtins,
  function references, non-Bool conditions, fuel/stack exhaustion, ambiguous
  references.
2026-11-20 10:43:59 +08:00
cflmy d54fda9e78 feat(frontend): cross-module call validation and builtin namespace inference
- calls: validate_program runs after a whole project is loaded and checks
  qualified calls through explicit import aliases against the imported module's
  signatures (missing function, missing/unknown arguments), plus qualified
  builtin calls such as math.eval; local calls stay checked by
  validate_module so diagnostics are not duplicated.
- infer: `math.eval(...)` / `foreign.call(...)` callee namespaces are builtin
  namespaces, not free variable reads, so they no longer become inferred
  parameters or unknown-variable errors. Explicit signatures behave likewise.
2026-11-20 10:43:59 +08:00
cflmy 3f8dce013b feat(package): project manifest, deterministic lockfile and root discovery
Blueprint section 8, minimum viable P3:

- marqdo.yaml parsing/validation: project identity, dependency sources (path or
  git with mandatory exact rev), plugin requirements, capability requests;
  absolute dependency paths, ambiguous sources and unpinned git are errors.
- marqdo.lock: lock_version, sorted packages, canonical YAML with no
  timestamps or machine paths so repeated locking is byte-identical.
- discover_root walks up to the nearest marqdo.yaml.
- Manifest::template for `marqdo init`.
2026-11-20 10:43:59 +08:00
cflmy 6da206a906 feat(plugin-abi,sdk): versioned C ABI v1, host bindings, Rust SDK and C example
Blueprint section 10: a first-class plugin ABI independent of the legacy
MARQDO_ABI_VERSION=2, with a C ABI that is explicitly not a sandbox.

- include/marqdo_plugin_abi_v1.h: frozen stable-layout C contract with
  documented buffer ownership/lifetimes, size limits, NULL/zero-length
  semantics, single-threaded invocation, no hot-unload policy, version
  negotiation and target-triple requirements.
- marqdo-plugin-abi (host side): repr(C) mirrors with compile-time layout
  assertions, major/minor negotiation (wrong major fails, minor clamps),
  plugin.yaml parsing/validation, target selection and sha256 artifact
  verification, wire_version 1 typed JSON request/response envelopes,
  structured PluginError, and a libloading-based loader with host alloc/free,
  export registration and runtime ABI re-negotiation.
- marqdo-plugin-sdk (plugin side): mq_plugin_entry! macro, panic containment,
  safe byte-buffer wrappers, capability/logging context and JSON helpers; it
  depends only on the shared ABI crate.
- examples/plugin-demo: dependency-free C plugin (solver.add), manifest,
  build.sh that refreshes the artifact hash, README explaining that install,
  inspect and load are three different trust decisions.
- Tests: C plugin cross-compiled with cc and loaded, plus wrong ABI major,
  missing symbol/artifact, hash mismatch and malformed manifest cases.
2026-11-20 10:43:59 +08:00
cflmy f6c3efa849 feat(math,foreign): native formula kernel and isolated Python/Node kernels
Marqdo-math: limited, documented LaTeX-subset kernel (blueprint section 5).
- Exact BigRational arithmetic, structured Float only for irrational results;
  Int result when exact; bounds on exponents, source size, AST nodes and result
  bits so hostile input cannot hang or exhaust memory.
- Supports numbers, Unicode identifiers, \pi/\e, + - * / ^, \frac, \sqrt,
  \cdot/\times, braces, parentheses, one '=' equation, and linear solving for a
  target variable (F=ma solved for F/m/a).
- Unsupported constructs are structured errors; the kernel never claims any
  mathematical proof.

Marqdo-foreign: explicit kernels (blueprint section 6).
- run/call/detect only; reading a document never spawns a process.
- Direct exec of python3/python/node with a temp program file, clear env,
  private cwd, JSON request envelope on stdin, captured bounded stdout/stderr,
  wall-clock timeout with kill, typed wire decoding for call, structured
  ForeignError for spawn/protocol/timeout/oversized output.
- Process isolation only: crate docs state this is not a strict OS sandbox.
2026-11-20 10:43:50 +08:00
cflmy 17b68da7f9 feat(host): capability host, budgets, effect tracing and kernel adapters
Implements the single side-effect boundary from blueprint sections 7.3 and 9:

- Capabilities: console/math, project-relative fs globs, net hosts, process,
  foreign, plugins; pure/project defaults deny external I/O and grants are
  explicit, checked at execution time (never at parse time).
- Budget: evaluation fuel, call-stack depth, program output bytes and a
  wall-clock deadline, all reported as resource.exhausted.
- EffectTrace: op/module/span/duration/result entries with no secret data.
- StdHost: canonical Host implementation dispatching print, fs read/write and
  the math/foreign kernels; all errors are structured RuntimeErrors, including
  a permission.denied path with a grant hint.
- ProjectFs: project-rooted, rejects absolute paths, `..` traversal and symlink
  escapes outside the root; bounded UTF-8 reads and safe writes.
- Math adapter: parse/eval/substitute through marqdo-math with exact rational
  binding conversion and explicit unsupported-operation errors.
- Foreign adapter: capability-gated one-shot run/call through marqdo-foreign,
  typed wire input, structured result map, timeout/output limits mapped to
  budget errors.
2026-11-20 10:43:50 +08:00
cflmy d799617076 feat(sandbox): sandbox modes and honest strict-mode refusal
Blueprint section 9: advertise a security guarantee only when it can be
enforced.

- SandboxMode pure/project/strict with stable names.
- PlatformCapabilities detection reports strict isolation as unavailable in
  this build with an explicit reason; worker processes are reported false.
- enforce() refuses strict mode when OS-level isolation is not implemented,
  instead of pretending a cooperative in-process check is a sandbox.
- Pure/project modes remain available.
2026-11-20 10:43:50 +08:00
cflmy 46c9ca2fd3 feat(value,ir): ref.ambiguous error, exact conversion helpers and mixed builtin calls
- value: add ErrorCategory::RefAmbiguous (code ref.ambiguous) for the S-07 rule
  that a reference matching several resources must never silently pick the
  first one; re-export float/integer conversion helpers at the crate root.
- ir/builtins: math/foreign calls accept positional arguments followed by named
  ones (matching the documented examples) instead of being named-only; arity
  and unknown-name validation still apply.
- tests updated for the new builtin call shape.
2026-11-20 10:43:50 +08:00
cflmy 8003907f4e feat(frontend): heading scopes, semantic mapping, inference and call validation
Implement the P0/P1 frontend from spec/mapping.md, spec/evaluation.md:

- units: heading scope index (level-1 namespace, level-2 function, level>=3
  section), range computation, duplicate/empty-name diagnostics, GitHub anchor
  table kept separate from canonical ids, and warnings for would-be execution
  nodes outside function bodies.
- map: AST-to-IR mapping where role comes from node kind + position. Strong is
  an execution statement, Emph a return, ordered list a branch, bullet list a
  loop sequence, block quote a call; invalid shapes are hard
  markdown.invalid_construct errors, never prose fallback. Payload parsing
  errors are mapped from payload-relative to file-relative spans.
- payload: strict text/code payload concatenation with exact span mapping
  (including code-span delimiter handling) used to rewrite expression spans
  onto the source file.
- signature: optional `(name: Type = const)` paragraph declaration; defaults
  must be compile-time constants; duplicates and malformed declarations are
  errors.
- infer: definite-assignment data-flow parameter inference (must/may/ambiguous
  states, loop first-iteration handling, branch joins) per blueprint S-04.
- calls: static call validation of arity/named arguments/builtin signatures
  after all signatures are known; inferred signatures accept named args only.
- resources: inert Formula/Code resources with owning units and SHA-256
  content hashes.
- analyze: pipeline (parse -> units -> frontmatter imports -> map -> signature/
  inference -> module -> call checks) that never executes anything.

Tests include factorial recursion, loop counter, prose-invariance (IR equal
modulo spans), and hard-error cases.
2026-11-20 10:36:12 +08:00
cflmy 62a9af05fd feat(markdown,value): record code span delimiters, inert-executable code and profile refinements
- md code nodes now carry `ticks` (delimiter length) so executable payload
  offsets inside code spans map back to exact source bytes instead of pointing
  at backticks; this keeps diagnostics precise for mixed text/code payloads.
- value: add the markdown.inert_executable warning code and a small clippy fix,
  without changing the wire format.
- spec: document the code-span tick metadata, declare strikethrough lazy, and
  make the inert-execution warning explicit for nodes outside any function.
- workspace dependencies: add unicode-ident (grammar) and glob (capability
  path patterns) for the frontend/host layers.

Tests: 15 markdown + 10 value + 2 span tests pass; fmt/clippy clean.
2026-11-20 10:36:07 +08:00
cflmy 4db6e6afd6 feat(ir): executable IR, module/unit identities and builtin metadata
Add the independent program IR from blueprint sections 7.1 and 11:

- ModuleId, FunctionId and UnitId canonical identities; UnitInfo/UnitKind for
  heading scopes (root, namespace, function, section).
- Function with explicit or inferred signature metadata, params and defaults;
  assignments, expression statements, link-form calls, branches, loops and
  returns, all carrying source spans.
- CallTarget distinguishes same-module, import-alias-qualified and
  link-resolved calls.
- ImportDecl from explicit front-matter imports; Resource records inert
  Formula/Code artifacts with SHA-256 content hashes and owning units.
- Module anchors, resolved import edges and Program lookup helpers used by the
  runtime and CLI.
- Builtin registry (names, aliases, arity, named-only/effect flags) shared by
  static validation and runtime dispatch; adding a builtin is a spec change.

Prose is absent from the IR by construction, so narrative edits cannot change
the executable projection.
2026-11-20 10:35:54 +08:00
cflmy 9897ecc37f feat(expression): strict expression and statement grammar for executable payloads
Implement the independent expression parser frozen in spec/expression.ebnf:

- Unicode XID identifiers (Chinese names supported), reserved keywords
  true/false/None/and/or/not.
- Arbitrary-precision Int literals, binary64 Float literals (digits required on
  both sides, overflow rejected), double-quoted Text with \u{...} escapes.
- List and record literals, calls with positional-then-named arguments and
  trailing commas, member access, indexing, chained postfix operators.
- Precedence exactly as specified: right-associative ** with unary minus
  binding outside, multiplicative/additive, single non-associative comparison,
  short-circuit and/or.
- Assignment only as `identifier = expression` at statement start; no comments,
  no second Markdown parser, max nesting depth 128 with structured errors.
- parse_expression/parse_statement/parse_arguments never panic; ParseError has
  stable code expr.parse, payload-relative byte spans and Display/Error.

50 unit tests + 2 doctests, cargo fmt/clippy clean.
2026-11-20 10:35:49 +08:00
cflmy 55a2b2d82d spec(P0): freeze markdown profile, AST mapping, expression grammar and evaluation semantics
Add the formal specifications required before implementation:

- spec/markdown-profile.md: parser version, exact option table, byte spans,
  BOM policy, front matter rule, AST JSON view, structural equivalence.
- spec/mapping.md: normative AST-to-program mapping (heading scopes, lazy
  regions, strong/emph/list/blockquote roles), strict payload extraction and
  the hard-error checklist. Invalid executable shapes never degrade to prose.
- spec/expression.ebnf: the small independent expression/statement grammar
  (Unicode identifiers, arbitrary precision integers, precedence, named
  arguments, no comments, no truthiness).
- spec/evaluation.md: value model, exact integer semantics, scopes, order,
  parameter inference via definite-assignment data flow, budgets and the v1
  builtin table.
2026-11-20 10:26:15 +08:00
cflmy 10c0c0a013 feat(markdown): freeze the Markdown profile and canonical AST view
P0 baseline: one Markdown parser, one frozen option set, no silent upgrades.

- profile: CommonMark 0.31.2 core + GFM tables/strikethrough/task lists +
  dollar math + YAML front matter at byte 0, with every other extension
  disabled and asserted by tests. comrak is pinned to exactly 1.0.0.
- ast::MdNode: owned, serializable node tree carrying kind, byte Span and
  literal payloads. Nodes outside the profile are preserved as MdKind::Other
  instead of being dropped; task list items keep their checkbox state.
- frontmatter: extracts the YAML body from the parser node, parses it once,
  and exposes only the explicit `import` alias mapping; invalid YAML is
  reported, never guessed.
- text: strict executable payload extraction (Text/Code only, everything else
  is a hard error) and inert plain-text extraction for heading names.
- UTF-8 BOM is tolerated; all spans are re-based onto the original file.

No semantic mapping or execution exists in this crate by design.
2026-11-20 10:25:02 +08:00
cflmy 1520244923 feat(value): value model, structured diagnostics and wire format
Implements the shared data contract from blueprint sections 3.4, 7.2 and 10.3:

- Value: Unit/None/Bool/Int(BigInt)/Float/Text/List/Map/Table/Formula/Code/
  FunctionRef/PluginHandle; Unit and None stay distinct and there is no
  implicit string/number coercion.
- Diagnostic with stable machine-readable codes (markdown.invalid_construct,
  expr.parse, symbol.unknown, permission.denied, ...) and span/related/hints.
- RuntimeError with category codes and call frames; renders to a Diagnostic.
- Versioned, type-tagged JSON wire format (wire_version = 1) that keeps
  arbitrary-precision integers as decimal strings, tags NaN/Infinity, enforces
  depth/size limits on untrusted input and never trusts JSON numbers for Int.
- Byte spans and line/column mapping live in the new marqdo-span crate.
2026-11-20 10:24:13 +08:00
cflmy df878b434a chore(workspace): scaffold Rust workspace and crate graph
Create the Marqdo CLI Rust workspace with the module boundaries from the
development blueprint (section 11):

- marqdo-span: byte spans and line/column mapping shared by every layer
- marqdo-value, marqdo-expression, marqdo-markdown, marqdo-ir
- marqdo-frontend, marqdo-runtime, marqdo-host, marqdo-sandbox
- marqdo-math, marqdo-foreign, marqdo-package
- marqdo-plugin-abi, marqdo-plugin-sdk
- marqdo-cli (binary `marqdo`)

Pin the Markdown parser exactly (comrak =1.0.0, no default features) so the
frozen profile can never drift silently. Dependencies are centralized in
workspace.dependencies; no crate is allowed to add its own version of a shared
dependency.
2026-11-20 10:23:39 +08:00
cflmy 6cfb1bf967 init 2026-11-20 09:31:20 +08:00