Commit Graph
166 Commits
Author SHA1 Message Date
cflmy f01e8a9bb0 docs(status): record the CI/security convergence evidence
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was canceled
Record the six-step RFC-CI-0001 series, the Woodpecker pipelines, the Windows
cross-target check, the local disposable-VM G-C PASS at 0d7e11b (7/7 plus the
strict JSON and inert-@knowledge contracts), and the correctness-verified
performance SMOKE run. Keep the release decision at DO_NOT_RELEASE because the
same SHA still lacks a remote Windows/macOS native PASS, a remote G3 artifact
and a MEASURED G4 report.

Include the RFC itself and link it from docs/ci.md. This is the documentation
follow-up to step 6; the tested code SHA is 0d7e11b.
2026-11-21 17:37:31 +08:00
cflmy 0d7e11b6f1 ci: aggregate SHA-bound release evidence and clear status drift
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
Add tools/ci/release_evidence.py (marqdo.release-gates.v1) plus unit tests:
missing/skipped/FAIL/UNVERIFIED and tested_sha mismatches stay
DO_NOT_RELEASE, performance may be PASS/WARN, unknown gate names are rejected.
The GitHub CI now has a release-evidence job that maps the Linux/schema and
native matrix results onto the manifest and uploads it, and both CI systems
run the evidence-tool tests.

Clear the contradictions the RFC calls out: remove the stale CURRENT.md rows
that claimed DNS/TLS/UDP and fs metadata were absent while the same table said
they were implemented; correct docs/ci.md to state that the GitHub-hosted
Windows/macOS matrix exists (Woodpecker has no such agent); keep the old
convergence plan historical; require the evidence manifest in releasing.md;
and add a docs-consistency guard for these claims.

This is step 6 of the CI/security RFC (B05/B06/B07).
2026-11-21 17:29:48 +08:00
cflmy d2267e912a perf: add correctness-verified marqdo.perf.v2 sampler
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was canceled
Replace the three-run baseline profile with tools/perf/runner.py: one fixed
release binary, per-child rusage via posix_spawn/wait4, 5 warmups and 20
samples by default, SHA-derived interleaved order, p50/p95/MAD, CPU user/sys
and peak RSS. Every warmup output is normalized and compared with a committed
wire digest before timing; a mismatch is CORRECTNESS_FAIL, never a fast
sample. Short or unapproved-host runs are SMOKE, and baselines from another
platform_id/profile are refused.

Add workloads.json, expected digests for nine existing workloads, a dedicated
fixed-node workflow (manual, self-hosted, requires --cpu-list), and
docs/performance.md. bench.py stays as a compatibility shim. This is step 5 of
the CI/security RFC (B08).
2026-11-21 17:28:05 +08:00
cflmy 990e984ab6 ci(security): bind the G-C lane to one SHA and emit evidence
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
Add tools/ci/strict-evidence.py, which records only measured host/source
facts: tested SHA (git-verified when a checkout exists), source archive
digest, kernel, bwrap/rustc/cargo, uid, cgroup2, runner kind and outcome. The
lane now writes this document on every exit path, refuses a root-only PASS,
and covers the strict @knowledge-inert and forged-marker JSON contracts in
addition to the existing 7/7 adversarial matrix and positive CLI checks.

Rework security-linux.yml to run on trusted master pushes or an explicit
source_sha, bind the checkout to the tested SHA, create a git archive for the
digest, avoid PR execution and upload the evidence artifact. This is step 4 of
the CI/security RFC (B04/B09/B10); the runner must still be an ephemeral VM.
2026-11-21 17:14:43 +08:00
cflmy c9f509e161 test(cli): guard the platform-neutral strict child boundary
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
Add an architecture test that rejects direct marker_present/verify_active
references from the CLI and keeps the Linux-only functions out of the
marqdo-sandbox crate root. Windows cross-target cargo check now compiles the
whole workspace (macOS cross-check is blocked only by ring's Darwin C/asm
build on this Linux host; its cfg(not(linux)) path is the same one Windows
exercises).

This completes the step 3 verification for the CI/security RFC B01.
2026-11-21 17:10:13 +08:00
cflmy f817cceb3b fix(cli): normalize strict failures to one JSON document
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
Capture the strict child through launch_strict_capture and classify the
outcome before writing anything. In --json mode a control-plane failure now
emits exactly one error document with a stable code instead of forwarding
bubblewrap stderr and leaving stdout empty; a verified child's valid JSON is
forwarded unchanged with its own exit code. Empty, partial or non-JSON child
output becomes strict.child_invalid.

Route the remaining run --json error paths (mode, grants, missing entry,
trace) through the same contract, classify sandbox launch failures with
StrictFailureKind so probe/spawn/timeout/resource failures stay
distinguishable, and add deterministic routing tests plus forged-marker CLI
coverage. Document the decision in ADR-0053 and spec/security.md 1.2.

This is step 2 of the CI/security RFC (B02/B03).
2026-11-21 16:46:27 +08:00
cflmy f4a404bae9 fix(sandbox): add platform-neutral strict child inspection
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
The CLI referenced Linux-only marker_present/verify_active directly, so
macOS and Windows builds failed with E0425. Add StrictChildState and
inspect_strict_child() to marqdo-sandbox: Linux performs the full identity
verification, while every other platform rejects even a forged
MARQDO_STRICT_ACTIVE marker. The Linux-only functions are no longer
re-exported from the crate root, so future callers cannot bypass the
platform boundary.

This is step 1 of the CI/security RFC (B01).
2026-11-21 16:36:49 +08:00
cflmy 32d8fe1f7a docs(status): record the merged master series and follow-up fixes
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
Master now contains the fast-forwarded RFC-LANG-0002 series; the former
semantics-v2-refactor branch was deleted, so only master remains.

- STATUS lists the full linear series (docs, implementation, expression
  grammar, status, seven follow-up fixes) and the verification at 9c1723d:
  fmt/clippy clean, 507 passed / 0 failed / 2 ignored, stdlib digest verified
- counts updated to 49 compile-negative fixtures and 507 workspace tests
  across README, COMPATIBILITY, CHANGELOG, CURRENT and the migration guide
- green evidence pins code commit 9c1723d and lists the follow-up fixes;
  CHANGELOG gains the follow-up correctness section
- migration inventory regenerated: 1019 classified files, 0 pending
- CURRENT.md baseline is the 40-hex 9c1723d commit required by the docs gate

Verified: docs_consistency 4/4, fmt, clippy -D warnings and the full
workspace suite green; python3 tools/pack-stdlib.py PASS.
2026-11-21 16:01:19 +08:00
cflmy 9c1723d44f test(markdown): cover the removed syntax-selector key probe
frontmatter::has_key() is what rejects marqdo_semantics/syntax directives;
add unit coverage for a present key, an absent key and invalid YAML (which
reports no key rather than panicking).

Verified: cargo test -p marqdo-markdown and the workspace suite green
(507 passed, 0 failed, 2 ignored).
2026-11-21 15:57:13 +08:00
cflmy b3429cbeb3 fix(frontend): emit diagnostics in the specified stable order
spec/mapping.md §8 requires diagnostics ordered by source position, then
code, so IDE/CI diffs are stable. analyze_source appended diagnostics by
pipeline stage (units, mapping, inference, call validation), so a later
heading error could precede an earlier payload error.

Sort the final list by (span.start, code) with span-less diagnostics last,
and pin the order with a test that mixes an indexing-stage duplicate-heading
error with an earlier mapping-stage syntax error.

Verified: clippy -D warnings clean; cargo test --workspace --locked green.
2026-11-21 15:55:23 +08:00
cflmy 0bbe0f6524 fix(frontend): honour the Setext no-declaration rule; drop dead helper
RFC-LANG-0002 allows a mode declaration only as an ATX heading prefix; a
Setext heading merely inherits the nearest ancestor mode. heading_declaration
was applied to every heading, so a Setext 'title' beginning with @exec/
@knowledge silently switched modes and could create a function.

- skip declaration parsing for Setext headings (they keep their literal name
  and inherited mode); add a regression test
- remove the dead normalized_heading_name helper and its imports, which only
  existed for the removed link-form call resolution

Verified: clippy -D warnings clean; cargo test --workspace --locked green
(505 passed, 0 failed, 2 ignored).
2026-11-21 15:52:46 +08:00
cflmy a3f9770761 fix(frontend): point unmatched end diagnostics at the end payload
A stray end after an earlier statement was reported with the span of the
previous payload (position-1) and a confusing 'ends outside its owning
function segment' message. Report it on the end payload itself with the
standard sem.unmatched_end wording, and pin the span with a unit test.

Verified: clippy -D warnings clean and cargo test --workspace --locked
green (62 targets).
2026-11-21 15:50:46 +08:00
cflmy 9fb97bc2df fix(frontend): recognize signatures only from Exec payloads
signature::detect() scanned the function body independently of heading
modes, skipping only level>=3 headings. A 'params(...)' code span in an
### @knowledge section was therefore taken as the function signature
('inspect --signature' reported it as explicit), even though that span is
inert documentation and is never executed.

Unify recognition with the executable traversal:

- MappedBody now reports the params(...) payload the mapper accepted; the
  mapper already enforces Exec mode, directness, first-function-wide
  position and single declaration
- signature::from_payload validates that text and rewrites default-value
  spans; signature::detect is removed along with its mode-blind heuristic
- analyze uses the mapper result, so Knowledge sections cannot declare a
  signature and prose/Knowledge subsections may still precede the first
  Exec declaration
- tests: Knowledge params ignored; Exec params after prose/Knowledge is the
  signature; spec/mapping.md states the recognition rule

Verified: clippy -D warnings clean; cargo test --workspace --locked green
(62 targets).
2026-11-21 15:48:37 +08:00
cflmy d5002ac826 docs(code): remove stale pre-RFC role wording and fix review aid
Follow-up cleanup of text left by the concurrent implementation:

- marqdo-markdown AST, marqdo-ir statements/branch arms, frontend units,
  payload and signature module docs now describe RFC-LANG-0002 roles
  (payload carriers, heading modes, if/else groups, while/end, params)
- marqdo-expression public docs list parse_payload_statement and stop
  describing a Markdown-link call that no longer exists
- retired markdown.invalid_construct/markdown.inert_executable constants
  are documented as decode-only history for old Analysis/Trace JSON
- drop the stale 'expected red' comment in the CLI integration test
- the dev-only .mq.md review aid no longer reports semicolons inside
  string literals, and skips AST baselines / expected-invalid negatives

Verified: fmt, clippy -D warnings and cargo test --workspace --locked green.
2026-11-21 15:44:31 +08:00
cflmy 361a6cc341 refactor(ir): drop the dead call-statement representation
The pre-RFC link-form block-quote call was removed by RFC-LANG-0002, but the
IR still carried Stmt::Call, CallSite and CallTarget with matching branches in
the runtime, infer, call validation, loader, analysis projection and JSON
output. No mapper produces that variant any more, so it was a second call
representation that could never be reached.

- remove Stmt::Call, CallSite and CallTarget from marqdo-ir
- remove the runtime exec_call_site path and every exhaustive match arm
- remove the 'call_site' Analysis node and its JSON helpers
- state in spec/analysis-v2.md that calls are expressions; keep v1 as history
  with a note that call_site is not produced by the current language

Verified: cargo clippy --workspace --all-targets -D warnings clean and
cargo test --workspace --locked green (62 targets).
2026-11-21 15:39:29 +08:00
cflmy b38b769932 fix(frontend): enforce function-wide params position
The concurrent mapper tracked the first params declaration in the Mapper (so
duplicates across heading segments were caught) but checked 'first payload'
with a per-segment vector index. A params(...) in a later ### @exec segment
was therefore accepted even after an earlier segment had executed statements,
and a duplicate could even be reported as a position error.

- track whether any statement has been lowered for the function and require
  params(...) to come before it, across all heading segments
- check duplicates first, then execution order, then the in-segment index
- add negative fixtures 48 (after an earlier segment) and 49 (duplicate
  across segments); update the corpus count and spec wording

Verified: cargo test --workspace --locked green (62 targets).
2026-11-21 15:36:30 +08:00
cflmy bbeb1de768 docs(status): record the landed RFC-LANG-0002 implementation
Update status, compatibility, changelog, README, architecture overview,
ADR status, migration guide, contributor/CI/security/release notes and
the per-file inventory from 'migration pending' to the verified state:

- language SHA ffd1cc9: fmt/clippy clean, workspace 501 passed / 0 failed
  / 2 ignored, stdlib digest regenerated and pack-verified
- green evidence report added; the prior intermediate evidence is marked
  superseded
- inventory regenerated: 1015 classified files, 0 pending
- explicit non-claims: Windows/macOS native lanes and the disposable-VM
  strict lane were not re-run at this SHA, so the release decision remains
  DO NOT RELEASE

Verified: cargo test -p marqdo-cli --test docs_consistency (4/4) and
git diff --check.
2026-11-21 12:31:38 +08:00
cflmy ffd1cc9862 refactor(expression): own the v2 payload statement grammar
Move keyword recognition out of the Markdown mapper and into the single
expression/statement parser:

- add parse_payload_statement + PayloadStatement for if/else/while/end/
  return/params/ordinary; keyword matching uses complete tokens, requires
  whitespace before a control expression, rejects trailing tokens/colons
  and reserves the statement words in expression positions
- keep expression spans payload-relative so the frontend can rewrite them
  onto the file; malformed params(...) interiors stay owned by the
  signature parser (one sem.syntax diagnostic, not a duplicate)
- simplify the frontend mapper to control-stack lowering and mapped
  semantic diagnostics only
- emit sem.no_entry_function from run when no Exec H2 main exists
- add else-under-while, params-in-quote, Knowledge-inertness and
  no-entry tests; update the corpus counts
- make the EBNF parameter type annotation optional (params(n) is valid)

Verified: cargo fmt --check, cargo clippy -D warnings and
cargo test --workspace --locked: 501 passed, 0 failed, 2 ignored.
2026-11-21 12:29:07 +08:00
cflmy e029853e32 feat(lang): implement RFC-LANG-0002 unified semantics (breaking)
One Markdown-to-program mapping, no version selector or legacy mapper.

Frontend/runtime:
- resolve @exec/@knowledge heading declarations with nearest-ancestor
  inheritance; only an Exec H2 is callable; add UnitKind::Knowledge and
  sem.* diagnostics (mode marker, unit name, exec-without-function,
  payload nesting/empty, syntax, control stack, params position,
  version directive, inferred-signature lint)
- extract Strong/Emph/inline Code as one-statement carriers, refuse
  nested carriers, and honour heading segment boundaries so an if/while
  cannot be closed across a heading
- lower if/else/while/end to the existing Branch/Loop/Return IR with a
  compile-time control stack; keep payload-relative source spans through
  expression parsing and rewrite them onto the file
- accept params(...) only as the first direct Exec H2 payload; keep the
  existing free-variable inference and named-only rule for inferred
  signatures; fix duplicate-mode/else/end and Knowledge-body inertness
- reject marqdo_semantics/syntax front matter directives

Data and tests:
- migrate stdlib/**/*.mq.md, regenerate stdlib.json digest
- migrate examples and the full semantics/negative/runtime corpus to
  the only current syntax; expected values/runtime codes preserved
- migrate Rust-embedded .mq.md fixtures, init template, CLI help and
  security-lane fixtures

Verified: cargo fmt --check, cargo clippy -D warnings and
cargo test --workspace --locked (green).
2026-11-21 12:26:46 +08:00
cflmy 49835424ee docs(spec): adopt RFC-LANG-0002 unified language semantics
Replace the Strong/Emph/List/Quote mapping in place with the single
@exec/@knowledge heading-mode model, unified payload carriers and keyword
control flow.

- add the authoritative RFC, ADR-0052 and a historical ADR index
- rewrite spec/mapping.md and spec/expression.ebnf as the normative
  mapping/statement grammar; update evaluation, references, math,
  foreign, package-format, plugin-abi, security and schema notes
- rewrite README, AGENTS, COMPATIBILITY and CHANGELOG for the breaking
  language change; update the architecture index/overview and active docs
- mark superseded planning docs, compute RFCs, evidence and syntax-role
  ADRs as historical/superseded without rewriting their decisions
- add migration guide, per-file inventory and dev-only audit scripts

The frontend/stdlib migration is completed in the following commits; this
commit contains documentation only.
2026-11-21 12:24:19 +08:00
Marqdo Agent 0d8ed44010 feat(http): parse and build HTTP/1.1 framing by bytes 2026-11-21 11:05:15 +08:00
Marqdo Agent 28b178fc70 feat(regex): add host-backed groups, alternation and captures 2026-11-21 10:58:06 +08:00
Marqdo Agent 94a13880c4 feat(json,csv): bound depth and add multi-value/multi-row APIs 2026-11-21 10:51:55 +08:00
Marqdo Agent efbb8007e1 feat(compress): add bounded gzip product codec 2026-11-21 10:48:27 +08:00
Marqdo Agent ff3f19a2ea feat(net): add DNS, UDP, TCP listener and TLS endpoints 2026-11-21 10:43:08 +08:00
Marqdo Agent ee0c6cc80a feat(random): add explicitly granted OS entropy API 2026-11-21 10:25:11 +08:00
Marqdo Agent 6338fa10e9 feat(fs): add append, read_at and recursive walk 2026-11-21 10:20:07 +08:00
Marqdo Agent 7859c26213 refactor(cli,host): finish command, dispatch and package module split 2026-11-21 08:59:50 +08:00
Marqdo Agent 8a0c068856 docs(stdlib): list every product test and fix digest snippet 2026-11-21 08:55:09 +08:00
Marqdo Agent 550e66b1e2 refactor(cli,host): split command entry points and provider/effects modules 2026-11-21 08:54:29 +08:00
Marqdo Agent 791819a725 refactor(cli,host): split sandbox, resource-open and process modules 2026-11-21 08:49:25 +08:00
Marqdo Agent 3ef18fb496 feat(process): add explicitly granted process API 2026-11-21 08:46:02 +08:00
Marqdo Agent 9922bfae98 feat(stdlib): add explicitly granted env API 2026-11-21 08:40:54 +08:00
Marqdo Agent ae69391bc0 feat(stdlib): add datetime and project-mode clock APIs 2026-11-21 08:36:40 +08:00
Marqdo Agent cffb6790be feat(stdlib): add path and regex product modules 2026-11-21 08:32:32 +08:00
Marqdo Agent bde6f1d0ba feat(stdlib): add JSON and CSV product parsers 2026-11-21 08:23:54 +08:00
Marqdo Agent fbe36bcc43 feat(resource): add grant-checked fs resource operations 2026-11-21 08:20:31 +08:00
Marqdo Agent cd90f059b4 feat(stdlib): add bounded std:fs/file read, write and copy 2026-11-21 08:10:10 +08:00
Marqdo Agent 8dd3d5eb76 docs(evidence): refresh the migration source baseline 2026-11-21 01:56:57 +08:00
Marqdo Agent 0d40de7645 test(stdlib): add the demo module to the product lane 2026-11-21 01:56:04 +08:00
Marqdo Agent 7a93fe8e7b docs(evidence): record the doc and stdlib test migration gate 2026-11-21 01:54:20 +08:00
Marqdo Agent f0c25be01d docs(tests): map architecture and e2e gates to executable lanes 2026-11-21 01:51:51 +08:00
Marqdo Agent 04eae8746e test(stdlib): cover every bundled module through std imports 2026-11-21 01:50:01 +08:00
Marqdo Agent a8ccac9e67 test(architecture): gate stdlib layout, digest and resource endpoints 2026-11-21 01:45:07 +08:00
Marqdo Agent ce70bc4c42 test(docs): gate the current architecture and evidence links 2026-11-21 01:38:21 +08:00
Marqdo Agent c2730ea445 docs(architecture): consolidate current state and history 2026-11-21 01:36:42 +08:00
cflmy 24c88bcc78 fix(resource): reject provider read and write over-reports
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
2026-11-21 01:05:14 +08:00
cflmy 6aa3d605af fix(stdlib): return the validated write-all byte count 2026-11-21 01:02:45 +08:00
cflmy e6b5678350 fix(stdlib): strip CR from console lines 2026-11-21 00:59:58 +08:00
cflmy 4a0caabc03 fix(runtime): keep resource.read destination bounds strict 2026-11-21 00:58:42 +08:00