Commit Graph
122 Commits
Author SHA1 Message Date
cflmy 89555382e0 feat(ext): extension selection + lock v4, inspect --extensions (RFC-EXT-0001 PR-04)
- marqdo.yaml.extensions accepts only ./path or @alias/path, resolves to a
  regular file inside its owning package, never grants
- lock_version 4 records plugin/kernel/provider manifest+artifact identity;
  v3 requires explicit 'marqdo lock' migration; sync/check/run compare identity
- check stays static; run re-verifies artifact bytes; pure/strict refuse
- inspect --extensions reports declared/locked identity, plugin descriptors and
  stdlib aliases with authorization 'not_granted'
- black-box extension_selection suite (13) + package-layer tests; specs/ADR-0056
- workspace gate: fmt/clippy clean, 599 passed across 67 targets
2026-11-22 00:43:18 +08:00
cflmy 1f6fd644a0 feat(ext): single callable binding, provider v2 CLI, capability handlers, stdlib aliases
Implements RFC-EXT-0001 PR-00..PR-07 first pass:
- CallSiteKey/CallBindingTable + one frontend binding pass; runtime dispatch by identity
- prepare composition root shared by check/run/inspect/Analysis; exact plugin reconciliation
- provider.json v2 --provider/--allow-provider worker CLI with fail-closed verification
- capability handler registry with real fs.read and audit
- verified stdlib public aliases replace print_override/AST scan
- docs: specs, ADR-0054/0055/0057/0058, README, CI/release gates, evidence

PR-04 extension selection/lock v4 and final frozen-binary evidence remain.
2026-11-22 00:17:36 +08:00
cflmy 5fcc111a30 fix(package): reject rooted Windows dependency paths
CI / runtime contract (windows-latest) (push) Waiting to run
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
A drive-less rooted path such as /tmp/x or \tmp\x is not Path::is_absolute()
on Windows, but joining it still escapes the project root. Validate with
has_root() so every rooted dependency path is rejected on every platform. The
existing rejection test now also covers the Windows root-relative form.
2026-11-21 18:55:53 +08:00
cflmy a24161c934 fix(foreign): keep Windows temp paths non-verbatim
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
fs::canonicalize on Windows returns a \\?\C:\... path. Node's module loader
lstats the working/script path and rejects the verbatim form, so all Node
foreign-kernel tests failed on windows-latest. Use the native temp directory on
Windows and retain canonicalization on Unix for symlink safety; widen the CI
annotator so any remaining native failure shows a full assertion block.
2026-11-21 18:49:05 +08:00
cflmy 4265231a14 fix(portable): run the CLI on an 8 MiB thread and scope G3 tests
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
Windows gives the main thread 1 MiB, unlike Linux/macOS at 8 MiB. The bundled
regex self-test overflowed that stack even though the same document passed on
Linux, so run the CLI dispatch on an explicit 8 MiB thread on every platform.

The macOS package test compared a raw temp path with a canonicalized one; on
macOS /var is a symlink to /private/var. Compare canonical forms on both sides.

The adversarial security matrix is G3-only: ordinary cargo test --workspace on
a namespace-capable but non-disposable hosted runner turned G0 red. Make it
opt-in (MARQDO_REQUIRE_STRICT_SECURITY=1 for the G3 lane, or
MARQDO_RUN_STRICT_SECURITY=1 for a developer); the lane still requires the
mechanism and fails on a missing one.
2026-11-21 18:42:53 +08:00
cflmy b898c419fe test(portable): fix native stack and process-fixture assumptions
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
macOS arm64 aborted in nesting_depth_limit_is_enforced because the test-harness
thread stack is smaller than a production main thread. Run that contract test
on an explicit 8 MiB stack; the 128-level parser bound itself is unchanged.

The stdlib process product test hard-coded /bin/echo, which does not exist on
Windows. Use the test CLI itself as the granted program: the fixture reads
MARQDO_TEST_PROCESS_PROGRAM from the granted env, runs it with --version, and
the harness compares against the same invocation. process.run now forwards
SYSTEMROOT/WINDIR on Windows (process creation/DLL loading, not a capability),
and the bundled stdlib digest is regenerated for the changed test fixture.
2026-11-21 18:35:33 +08:00
cflmy 3a29fce061 fix(foreign): resolve Windows PATH executables with PATHEXT
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
The built-in Python provider scanned PATH for the extension-less token
python3; a Windows PATH contains python3.exe, so a working interpreter was
reported as kernel.unavailable and the semantic golden suite failed on
windows-latest. Expand candidates with PATHEXT (conservative default) and use
the same platform-aware resolver in the conformance test's availability probe.

Also make native failures diagnosable: strip ANSI before matching and emit the
panic/build context as public check annotations, and run Linux tests through
the same annotator. This fixes the Windows native lane and gives the macOS
failure the context needed to diagnose it.
2026-11-21 18:22:22 +08:00
cflmy 0d7e11b6f1 ci: aggregate SHA-bound release evidence and clear status drift
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
Add tools/ci/release_evidence.py (marqdo.release-gates.v1) plus unit tests:
missing/skipped/FAIL/UNVERIFIED and tested_sha mismatches stay
DO_NOT_RELEASE, performance may be PASS/WARN, unknown gate names are rejected.
The GitHub CI now has a release-evidence job that maps the Linux/schema and
native matrix results onto the manifest and uploads it, and both CI systems
run the evidence-tool tests.

Clear the contradictions the RFC calls out: remove the stale CURRENT.md rows
that claimed DNS/TLS/UDP and fs metadata were absent while the same table said
they were implemented; correct docs/ci.md to state that the GitHub-hosted
Windows/macOS matrix exists (Woodpecker has no such agent); keep the old
convergence plan historical; require the evidence manifest in releasing.md;
and add a docs-consistency guard for these claims.

This is step 6 of the CI/security RFC (B05/B06/B07).
2026-11-21 17:29:48 +08:00
cflmy c9f509e161 test(cli): guard the platform-neutral strict child boundary
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
Add an architecture test that rejects direct marker_present/verify_active
references from the CLI and keeps the Linux-only functions out of the
marqdo-sandbox crate root. Windows cross-target cargo check now compiles the
whole workspace (macOS cross-check is blocked only by ring's Darwin C/asm
build on this Linux host; its cfg(not(linux)) path is the same one Windows
exercises).

This completes the step 3 verification for the CI/security RFC B01.
2026-11-21 17:10:13 +08:00
cflmy f817cceb3b fix(cli): normalize strict failures to one JSON document
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
Capture the strict child through launch_strict_capture and classify the
outcome before writing anything. In --json mode a control-plane failure now
emits exactly one error document with a stable code instead of forwarding
bubblewrap stderr and leaving stdout empty; a verified child's valid JSON is
forwarded unchanged with its own exit code. Empty, partial or non-JSON child
output becomes strict.child_invalid.

Route the remaining run --json error paths (mode, grants, missing entry,
trace) through the same contract, classify sandbox launch failures with
StrictFailureKind so probe/spawn/timeout/resource failures stay
distinguishable, and add deterministic routing tests plus forged-marker CLI
coverage. Document the decision in ADR-0053 and spec/security.md 1.2.

This is step 2 of the CI/security RFC (B02/B03).
2026-11-21 16:46:27 +08:00
cflmy f4a404bae9 fix(sandbox): add platform-neutral strict child inspection
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
The CLI referenced Linux-only marker_present/verify_active directly, so
macOS and Windows builds failed with E0425. Add StrictChildState and
inspect_strict_child() to marqdo-sandbox: Linux performs the full identity
verification, while every other platform rejects even a forged
MARQDO_STRICT_ACTIVE marker. The Linux-only functions are no longer
re-exported from the crate root, so future callers cannot bypass the
platform boundary.

This is step 1 of the CI/security RFC (B01).
2026-11-21 16:36:49 +08:00
cflmy 9c1723d44f test(markdown): cover the removed syntax-selector key probe
frontmatter::has_key() is what rejects marqdo_semantics/syntax directives;
add unit coverage for a present key, an absent key and invalid YAML (which
reports no key rather than panicking).

Verified: cargo test -p marqdo-markdown and the workspace suite green
(507 passed, 0 failed, 2 ignored).
2026-11-21 15:57:13 +08:00
cflmy b3429cbeb3 fix(frontend): emit diagnostics in the specified stable order
spec/mapping.md §8 requires diagnostics ordered by source position, then
code, so IDE/CI diffs are stable. analyze_source appended diagnostics by
pipeline stage (units, mapping, inference, call validation), so a later
heading error could precede an earlier payload error.

Sort the final list by (span.start, code) with span-less diagnostics last,
and pin the order with a test that mixes an indexing-stage duplicate-heading
error with an earlier mapping-stage syntax error.

Verified: clippy -D warnings clean; cargo test --workspace --locked green.
2026-11-21 15:55:23 +08:00
cflmy 0bbe0f6524 fix(frontend): honour the Setext no-declaration rule; drop dead helper
RFC-LANG-0002 allows a mode declaration only as an ATX heading prefix; a
Setext heading merely inherits the nearest ancestor mode. heading_declaration
was applied to every heading, so a Setext 'title' beginning with @exec/
@knowledge silently switched modes and could create a function.

- skip declaration parsing for Setext headings (they keep their literal name
  and inherited mode); add a regression test
- remove the dead normalized_heading_name helper and its imports, which only
  existed for the removed link-form call resolution

Verified: clippy -D warnings clean; cargo test --workspace --locked green
(505 passed, 0 failed, 2 ignored).
2026-11-21 15:52:46 +08:00
cflmy a3f9770761 fix(frontend): point unmatched end diagnostics at the end payload
A stray end after an earlier statement was reported with the span of the
previous payload (position-1) and a confusing 'ends outside its owning
function segment' message. Report it on the end payload itself with the
standard sem.unmatched_end wording, and pin the span with a unit test.

Verified: clippy -D warnings clean and cargo test --workspace --locked
green (62 targets).
2026-11-21 15:50:46 +08:00
cflmy 9fb97bc2df fix(frontend): recognize signatures only from Exec payloads
signature::detect() scanned the function body independently of heading
modes, skipping only level>=3 headings. A 'params(...)' code span in an
### @knowledge section was therefore taken as the function signature
('inspect --signature' reported it as explicit), even though that span is
inert documentation and is never executed.

Unify recognition with the executable traversal:

- MappedBody now reports the params(...) payload the mapper accepted; the
  mapper already enforces Exec mode, directness, first-function-wide
  position and single declaration
- signature::from_payload validates that text and rewrites default-value
  spans; signature::detect is removed along with its mode-blind heuristic
- analyze uses the mapper result, so Knowledge sections cannot declare a
  signature and prose/Knowledge subsections may still precede the first
  Exec declaration
- tests: Knowledge params ignored; Exec params after prose/Knowledge is the
  signature; spec/mapping.md states the recognition rule

Verified: clippy -D warnings clean; cargo test --workspace --locked green
(62 targets).
2026-11-21 15:48:37 +08:00
cflmy d5002ac826 docs(code): remove stale pre-RFC role wording and fix review aid
Follow-up cleanup of text left by the concurrent implementation:

- marqdo-markdown AST, marqdo-ir statements/branch arms, frontend units,
  payload and signature module docs now describe RFC-LANG-0002 roles
  (payload carriers, heading modes, if/else groups, while/end, params)
- marqdo-expression public docs list parse_payload_statement and stop
  describing a Markdown-link call that no longer exists
- retired markdown.invalid_construct/markdown.inert_executable constants
  are documented as decode-only history for old Analysis/Trace JSON
- drop the stale 'expected red' comment in the CLI integration test
- the dev-only .mq.md review aid no longer reports semicolons inside
  string literals, and skips AST baselines / expected-invalid negatives

Verified: fmt, clippy -D warnings and cargo test --workspace --locked green.
2026-11-21 15:44:31 +08:00
cflmy 361a6cc341 refactor(ir): drop the dead call-statement representation
The pre-RFC link-form block-quote call was removed by RFC-LANG-0002, but the
IR still carried Stmt::Call, CallSite and CallTarget with matching branches in
the runtime, infer, call validation, loader, analysis projection and JSON
output. No mapper produces that variant any more, so it was a second call
representation that could never be reached.

- remove Stmt::Call, CallSite and CallTarget from marqdo-ir
- remove the runtime exec_call_site path and every exhaustive match arm
- remove the 'call_site' Analysis node and its JSON helpers
- state in spec/analysis-v2.md that calls are expressions; keep v1 as history
  with a note that call_site is not produced by the current language

Verified: cargo clippy --workspace --all-targets -D warnings clean and
cargo test --workspace --locked green (62 targets).
2026-11-21 15:39:29 +08:00
cflmy b38b769932 fix(frontend): enforce function-wide params position
The concurrent mapper tracked the first params declaration in the Mapper (so
duplicates across heading segments were caught) but checked 'first payload'
with a per-segment vector index. A params(...) in a later ### @exec segment
was therefore accepted even after an earlier segment had executed statements,
and a duplicate could even be reported as a position error.

- track whether any statement has been lowered for the function and require
  params(...) to come before it, across all heading segments
- check duplicates first, then execution order, then the in-segment index
- add negative fixtures 48 (after an earlier segment) and 49 (duplicate
  across segments); update the corpus count and spec wording

Verified: cargo test --workspace --locked green (62 targets).
2026-11-21 15:36:30 +08:00
cflmy ffd1cc9862 refactor(expression): own the v2 payload statement grammar
Move keyword recognition out of the Markdown mapper and into the single
expression/statement parser:

- add parse_payload_statement + PayloadStatement for if/else/while/end/
  return/params/ordinary; keyword matching uses complete tokens, requires
  whitespace before a control expression, rejects trailing tokens/colons
  and reserves the statement words in expression positions
- keep expression spans payload-relative so the frontend can rewrite them
  onto the file; malformed params(...) interiors stay owned by the
  signature parser (one sem.syntax diagnostic, not a duplicate)
- simplify the frontend mapper to control-stack lowering and mapped
  semantic diagnostics only
- emit sem.no_entry_function from run when no Exec H2 main exists
- add else-under-while, params-in-quote, Knowledge-inertness and
  no-entry tests; update the corpus counts
- make the EBNF parameter type annotation optional (params(n) is valid)

Verified: cargo fmt --check, cargo clippy -D warnings and
cargo test --workspace --locked: 501 passed, 0 failed, 2 ignored.
2026-11-21 12:29:07 +08:00
cflmy e029853e32 feat(lang): implement RFC-LANG-0002 unified semantics (breaking)
One Markdown-to-program mapping, no version selector or legacy mapper.

Frontend/runtime:
- resolve @exec/@knowledge heading declarations with nearest-ancestor
  inheritance; only an Exec H2 is callable; add UnitKind::Knowledge and
  sem.* diagnostics (mode marker, unit name, exec-without-function,
  payload nesting/empty, syntax, control stack, params position,
  version directive, inferred-signature lint)
- extract Strong/Emph/inline Code as one-statement carriers, refuse
  nested carriers, and honour heading segment boundaries so an if/while
  cannot be closed across a heading
- lower if/else/while/end to the existing Branch/Loop/Return IR with a
  compile-time control stack; keep payload-relative source spans through
  expression parsing and rewrite them onto the file
- accept params(...) only as the first direct Exec H2 payload; keep the
  existing free-variable inference and named-only rule for inferred
  signatures; fix duplicate-mode/else/end and Knowledge-body inertness
- reject marqdo_semantics/syntax front matter directives

Data and tests:
- migrate stdlib/**/*.mq.md, regenerate stdlib.json digest
- migrate examples and the full semantics/negative/runtime corpus to
  the only current syntax; expected values/runtime codes preserved
- migrate Rust-embedded .mq.md fixtures, init template, CLI help and
  security-lane fixtures

Verified: cargo fmt --check, cargo clippy -D warnings and
cargo test --workspace --locked (green).
2026-11-21 12:26:46 +08:00
Marqdo Agent 0d8ed44010 feat(http): parse and build HTTP/1.1 framing by bytes 2026-11-21 11:05:15 +08:00
Marqdo Agent 28b178fc70 feat(regex): add host-backed groups, alternation and captures 2026-11-21 10:58:06 +08:00
Marqdo Agent 94a13880c4 feat(json,csv): bound depth and add multi-value/multi-row APIs 2026-11-21 10:51:55 +08:00
Marqdo Agent efbb8007e1 feat(compress): add bounded gzip product codec 2026-11-21 10:48:27 +08:00
Marqdo Agent ff3f19a2ea feat(net): add DNS, UDP, TCP listener and TLS endpoints 2026-11-21 10:43:08 +08:00
Marqdo Agent ee0c6cc80a feat(random): add explicitly granted OS entropy API 2026-11-21 10:25:11 +08:00
Marqdo Agent 6338fa10e9 feat(fs): add append, read_at and recursive walk 2026-11-21 10:20:07 +08:00
Marqdo Agent 7859c26213 refactor(cli,host): finish command, dispatch and package module split 2026-11-21 08:59:50 +08:00
Marqdo Agent 550e66b1e2 refactor(cli,host): split command entry points and provider/effects modules 2026-11-21 08:54:29 +08:00
Marqdo Agent 791819a725 refactor(cli,host): split sandbox, resource-open and process modules 2026-11-21 08:49:25 +08:00
Marqdo Agent 3ef18fb496 feat(process): add explicitly granted process API 2026-11-21 08:46:02 +08:00
Marqdo Agent 9922bfae98 feat(stdlib): add explicitly granted env API 2026-11-21 08:40:54 +08:00
Marqdo Agent ae69391bc0 feat(stdlib): add datetime and project-mode clock APIs 2026-11-21 08:36:40 +08:00
Marqdo Agent cffb6790be feat(stdlib): add path and regex product modules 2026-11-21 08:32:32 +08:00
Marqdo Agent bde6f1d0ba feat(stdlib): add JSON and CSV product parsers 2026-11-21 08:23:54 +08:00
Marqdo Agent fbe36bcc43 feat(resource): add grant-checked fs resource operations 2026-11-21 08:20:31 +08:00
Marqdo Agent cd90f059b4 feat(stdlib): add bounded std:fs/file read, write and copy 2026-11-21 08:10:10 +08:00
Marqdo Agent 0d40de7645 test(stdlib): add the demo module to the product lane 2026-11-21 01:56:04 +08:00
Marqdo Agent 04eae8746e test(stdlib): cover every bundled module through std imports 2026-11-21 01:50:01 +08:00
Marqdo Agent a8ccac9e67 test(architecture): gate stdlib layout, digest and resource endpoints 2026-11-21 01:45:07 +08:00
Marqdo Agent ce70bc4c42 test(docs): gate the current architecture and evidence links 2026-11-21 01:38:21 +08:00
cflmy 24c88bcc78 fix(resource): reject provider read and write over-reports
ci/woodpecker/push/woodpecker Pipeline was successful
CI / fmt + clippy + test (Linux x86_64) (push) Canceled after 0s
CI / schema instance validation + consumers (push) Canceled after 0s
CI / runtime contract (windows-latest) (push) Canceled after 0s
CI / release candidate artifact (macos-latest) (push) Canceled after 0s
CI / release candidate artifact (ubuntu-latest) (push) Canceled after 0s
CI / runtime contract (macos-latest) (push) Canceled after 0s
2026-11-21 01:05:14 +08:00
cflmy 4a0caabc03 fix(runtime): keep resource.read destination bounds strict 2026-11-21 00:58:42 +08:00
cflmy 02aecaa747 test(conformance): load nested stdlib imports without false cycles 2026-11-21 00:58:34 +08:00
cflmy 64a6b48866 fix(resource): share one stdin cursor across handles 2026-11-21 00:49:07 +08:00
cflmy 3b13d8499a feat(stdlib): add streaming IO and console line APIs 2026-11-21 00:48:46 +08:00
cflmy aed6a80ba7 feat(resource): add stdin and stderr stdio endpoints 2026-11-21 00:36:25 +08:00
cflmy 757c5ed15e feat(provider): add C ABI provider worker binding v1
- Add the separate marqdo-provider-worker binary: loads one trusted c-abi-1
  shared library and serves the ADR-0035 framed-JSON resource protocol, with
  fail-closed startup and no in-process fallback (ADR-0038).
- Reuse StdHost::register_worker_resource_provider; language still sees only
  resource.* and project-mode/name grants.
- Compile a C test provider and run open/read/write/close through the helper;
  cover missing-library fail-closed. Trusted process boundary, not a sandbox;
  existing plugin-Worker ABI binding remains open.
- Add ADR/spec/status/README/changelog/evidence; workspace member and tests.

Gate: fmt, clippy -D warnings, 440 tests passed.
2026-11-20 23:30:22 +08:00
cflmy 313ac035e8 test(p2): prove foreign/plugin shortcuts fail in the pure lane
- Add negative tests: foreign.call -> permission.denied and plugin.invoke ->
  structured failure under --mode pure, complementing the 21-fixture pure lane.
- Update P2 evidence/tests README/changelog.

Gate: fmt, clippy -D warnings, 438 tests passed.
2026-11-20 23:29:14 +08:00