Compare commits

..
71 Commits
Author SHA1 Message Date
cflmy 595279fe38 跟踪未跟踪的文档
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/main Pipeline was successful
2026-11-21 21:04:42 +08:00
cflmy c11c1c858a perf: record the Woodpecker G4 measured baseline
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
ci/woodpecker/push/main Pipeline was successful
The manual .woodpecker/performance.yml workflow ran on the designated fixed
agent (platform_id woodpecker-agent-1): 5 warmups + 20 samples, all 9
workloads correctness-verified, CPU 0 pinned from the container cpuset. Two
runs differed by -10.9%..+7.1% p50 and -1.9% RSS. The node is a shared Docker
container with the powersave governor and loadavg ~5.7-6.1, so the release gate
is WARN, not PASS, and comparisons are same-platform only.

Store both marqdo.perf.v2 reports and the SHA-bound release manifest. The
manifest is DO_NOT_RELEASE only because strict_vm remains UNVERIFIED; Linux,
Windows, macOS and performance have real same-SHA results.
2026-11-21 20:07:58 +08:00
cflmy 1199c88e3e ci(perf): use POSIX sh options in the G4 step
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/main Pipeline was successful
ci/woodpecker/manual/performance Pipeline was successful
Woodpecker runs commands with /bin/sh (dash), which has no pipefail option.
The G4 script failed before measuring; use set -eu.
2026-11-21 19:58:37 +08:00
cflmy a3f3ecf9d0 ci(perf): add the fixed Woodpecker G4 workflow
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/manual/performance Pipeline failed
ci/woodpecker/push/main Pipeline was successful
Move the push/PR pipeline to .woodpecker/main.yml so the repository can use
Woodpecker's multi-workflow directory. Add .woodpecker/performance.yml: manual
only, master only, on the single fixed Woodpecker agent (platform_id
woodpecker-agent-1). It builds one release binary, pins the first CPU in the
container cpuset, runs 5 warmups + 20 samples with --approved-node, and prints
the full marqdo.perf.v2 report plus a compact G4 summary for extraction.

No PR code is executed on the performance agent; the workflow is manual.
2026-11-21 19:51:09 +08:00
cflmy a9cc1729e1 docs(status): clear stale native-unverified blockers
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
The native platform block is closed at 5fcc111 (GitHub run 30). Update the
work-package rows and open blockers: the remaining release gates are the remote
G3 artifact and a MEASURED G4 report, not Windows/macOS. Keep the historical
sections and their older SHAs intact.
2026-11-21 19:08:34 +08:00
cflmy fd04ffb98e docs(status): record the native-green matrix and final G3 evidence
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
GitHub run 30 on 5fcc111 is green for Linux, windows-latest and macos-latest
(the native failures were Windows PATHEXT/temp/rooted-path issues, macOS test
temp canonicalization and a Windows 1 MiB main-thread overflow). Woodpecker
#25 is green on the same SHA, and the disposable-VM G3 lane PASSes 7/7 plus the
strict JSON/inert-@knowledge contracts.

The release decision stays DO_NOT_RELEASE, now only for the remote G3 artifact
with matching tested_sha and the MEASURED G4 report; the native platform block
is closed. Document that the adversarial suite is G3-opt-in so ordinary
cargo test --workspace cannot turn a hosted namespace-capable host into a G0
failure.
2026-11-21 19:07:07 +08:00
cflmy 5fcc111a30 fix(package): reject rooted Windows dependency paths
CI / runtime contract (windows-latest) (push) Waiting to run
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
A drive-less rooted path such as /tmp/x or \tmp\x is not Path::is_absolute()
on Windows, but joining it still escapes the project root. Validate with
has_root() so every rooted dependency path is rejected on every platform. The
existing rejection test now also covers the Windows root-relative form.
2026-11-21 18:55:53 +08:00
cflmy a24161c934 fix(foreign): keep Windows temp paths non-verbatim
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
fs::canonicalize on Windows returns a \\?\C:\... path. Node's module loader
lstats the working/script path and rejects the verbatim form, so all Node
foreign-kernel tests failed on windows-latest. Use the native temp directory on
Windows and retain canonicalization on Unix for symlink safety; widen the CI
annotator so any remaining native failure shows a full assertion block.
2026-11-21 18:49:05 +08:00
cflmy 4265231a14 fix(portable): run the CLI on an 8 MiB thread and scope G3 tests
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
Windows gives the main thread 1 MiB, unlike Linux/macOS at 8 MiB. The bundled
regex self-test overflowed that stack even though the same document passed on
Linux, so run the CLI dispatch on an explicit 8 MiB thread on every platform.

The macOS package test compared a raw temp path with a canonicalized one; on
macOS /var is a symlink to /private/var. Compare canonical forms on both sides.

The adversarial security matrix is G3-only: ordinary cargo test --workspace on
a namespace-capable but non-disposable hosted runner turned G0 red. Make it
opt-in (MARQDO_REQUIRE_STRICT_SECURITY=1 for the G3 lane, or
MARQDO_RUN_STRICT_SECURITY=1 for a developer); the lane still requires the
mechanism and fails on a missing one.
2026-11-21 18:42:53 +08:00
cflmy b898c419fe test(portable): fix native stack and process-fixture assumptions
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
macOS arm64 aborted in nesting_depth_limit_is_enforced because the test-harness
thread stack is smaller than a production main thread. Run that contract test
on an explicit 8 MiB stack; the 128-level parser bound itself is unchanged.

The stdlib process product test hard-coded /bin/echo, which does not exist on
Windows. Use the test CLI itself as the granted program: the fixture reads
MARQDO_TEST_PROCESS_PROGRAM from the granted env, runs it with --version, and
the harness compares against the same invocation. process.run now forwards
SYSTEMROOT/WINDIR on Windows (process creation/DLL loading, not a capability),
and the bundled stdlib digest is regenerated for the changed test fixture.
2026-11-21 18:35:33 +08:00
cflmy 3a29fce061 fix(foreign): resolve Windows PATH executables with PATHEXT
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
The built-in Python provider scanned PATH for the extension-less token
python3; a Windows PATH contains python3.exe, so a working interpreter was
reported as kernel.unavailable and the semantic golden suite failed on
windows-latest. Expand candidates with PATHEXT (conservative default) and use
the same platform-aware resolver in the conformance test's availability probe.

Also make native failures diagnosable: strip ANSI before matching and emit the
panic/build context as public check annotations, and run Linux tests through
the same annotator. This fixes the Windows native lane and gives the macOS
failure the context needed to diagnose it.
2026-11-21 18:22:22 +08:00
cflmy d441d6f4cc ci: annotate the native test panic context
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
The native matrix only annotated the first matching line, which omitted the
fixture name and expected/actual values. Annotate the panic block and build
error context instead so the failing golden is identifiable from the public
check-run annotations.
2026-11-21 18:13:43 +08:00
cflmy 95e4c6b1a4 ci: annotate native workspace-test failures
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was canceled
The Windows/macOS jobs fail inside cargo test --workspace, but public check
annotations only show the step exit. Tee the output and emit the failing test,
panic or build-error line as a check annotation so the native failure is
diagnosable without repository-admin log access. Behaviour and exit status are
unchanged.
2026-11-21 18:09:28 +08:00
cflmy f9ef2328c6 docs(status): list the hardening and evidence commits
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
Complete the RFC-CI-0001 step table with the 512b9f5 lane-hardening and
beb68a9 evidence/CI fixes, and record pipeline #17.
2026-11-21 17:50:56 +08:00
cflmy beb68a911c docs(status): update G-C evidence to the hardened lane SHA
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
Record the 512b9f5 local disposable-VM run (7/7 plus strict JSON and inert
@knowledge, evidence.json/archive/log digests) and Woodpecker pipeline #16.
The lane now fails if strict-evidence.json cannot be written, so the green run
also proves the evidence path. Fix the cross-platform JSON smoke to tolerate
the expected non-zero strict refusal exit before parsing.

Tested code SHA: 512b9f5. Release decision remains DO_NOT_RELEASE for the
missing remote Windows/macOS native PASS, remote G3 artifact and MEASURED G4.
2026-11-21 17:46:16 +08:00
cflmy 512b9f5f1d ci(security): fail the lane when evidence generation fails
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
A green lane without a strict-evidence.json must not count as a gate. The EXIT
trap now turns a generator failure into a lane failure. The self-hosted
workflow exports the archive path in the run step (env written by GITHUB_ENV is
not guaranteed in the env expression context), records the pinned image digest
as an input, and verifies schema/outcome/tested_sha/non-root before uploading.
Also define SOURCE_SHA before the VM overlay commands.
2026-11-21 17:41:23 +08:00
cflmy f01e8a9bb0 docs(status): record the CI/security convergence evidence
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was canceled
Record the six-step RFC-CI-0001 series, the Woodpecker pipelines, the Windows
cross-target check, the local disposable-VM G-C PASS at 0d7e11b (7/7 plus the
strict JSON and inert-@knowledge contracts), and the correctness-verified
performance SMOKE run. Keep the release decision at DO_NOT_RELEASE because the
same SHA still lacks a remote Windows/macOS native PASS, a remote G3 artifact
and a MEASURED G4 report.

Include the RFC itself and link it from docs/ci.md. This is the documentation
follow-up to step 6; the tested code SHA is 0d7e11b.
2026-11-21 17:37:31 +08:00
cflmy 0d7e11b6f1 ci: aggregate SHA-bound release evidence and clear status drift
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
Add tools/ci/release_evidence.py (marqdo.release-gates.v1) plus unit tests:
missing/skipped/FAIL/UNVERIFIED and tested_sha mismatches stay
DO_NOT_RELEASE, performance may be PASS/WARN, unknown gate names are rejected.
The GitHub CI now has a release-evidence job that maps the Linux/schema and
native matrix results onto the manifest and uploads it, and both CI systems
run the evidence-tool tests.

Clear the contradictions the RFC calls out: remove the stale CURRENT.md rows
that claimed DNS/TLS/UDP and fs metadata were absent while the same table said
they were implemented; correct docs/ci.md to state that the GitHub-hosted
Windows/macOS matrix exists (Woodpecker has no such agent); keep the old
convergence plan historical; require the evidence manifest in releasing.md;
and add a docs-consistency guard for these claims.

This is step 6 of the CI/security RFC (B05/B06/B07).
2026-11-21 17:29:48 +08:00
cflmy d2267e912a perf: add correctness-verified marqdo.perf.v2 sampler
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was canceled
Replace the three-run baseline profile with tools/perf/runner.py: one fixed
release binary, per-child rusage via posix_spawn/wait4, 5 warmups and 20
samples by default, SHA-derived interleaved order, p50/p95/MAD, CPU user/sys
and peak RSS. Every warmup output is normalized and compared with a committed
wire digest before timing; a mismatch is CORRECTNESS_FAIL, never a fast
sample. Short or unapproved-host runs are SMOKE, and baselines from another
platform_id/profile are refused.

Add workloads.json, expected digests for nine existing workloads, a dedicated
fixed-node workflow (manual, self-hosted, requires --cpu-list), and
docs/performance.md. bench.py stays as a compatibility shim. This is step 5 of
the CI/security RFC (B08).
2026-11-21 17:28:05 +08:00
cflmy 990e984ab6 ci(security): bind the G-C lane to one SHA and emit evidence
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
Linux strict security lane (G-C) / strict isolation escape matrix (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
Add tools/ci/strict-evidence.py, which records only measured host/source
facts: tested SHA (git-verified when a checkout exists), source archive
digest, kernel, bwrap/rustc/cargo, uid, cgroup2, runner kind and outcome. The
lane now writes this document on every exit path, refuses a root-only PASS,
and covers the strict @knowledge-inert and forged-marker JSON contracts in
addition to the existing 7/7 adversarial matrix and positive CLI checks.

Rework security-linux.yml to run on trusted master pushes or an explicit
source_sha, bind the checkout to the tested SHA, create a git archive for the
digest, avoid PR execution and upload the evidence artifact. This is step 4 of
the CI/security RFC (B04/B09/B10); the runner must still be an ephemeral VM.
2026-11-21 17:14:43 +08:00
cflmy c9f509e161 test(cli): guard the platform-neutral strict child boundary
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
Add an architecture test that rejects direct marker_present/verify_active
references from the CLI and keeps the Linux-only functions out of the
marqdo-sandbox crate root. Windows cross-target cargo check now compiles the
whole workspace (macOS cross-check is blocked only by ring's Darwin C/asm
build on this Linux host; its cfg(not(linux)) path is the same one Windows
exercises).

This completes the step 3 verification for the CI/security RFC B01.
2026-11-21 17:10:13 +08:00
cflmy f817cceb3b fix(cli): normalize strict failures to one JSON document
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
Capture the strict child through launch_strict_capture and classify the
outcome before writing anything. In --json mode a control-plane failure now
emits exactly one error document with a stable code instead of forwarding
bubblewrap stderr and leaving stdout empty; a verified child's valid JSON is
forwarded unchanged with its own exit code. Empty, partial or non-JSON child
output becomes strict.child_invalid.

Route the remaining run --json error paths (mode, grants, missing entry,
trace) through the same contract, classify sandbox launch failures with
StrictFailureKind so probe/spawn/timeout/resource failures stay
distinguishable, and add deterministic routing tests plus forged-marker CLI
coverage. Document the decision in ADR-0053 and spec/security.md 1.2.

This is step 2 of the CI/security RFC (B02/B03).
2026-11-21 16:46:27 +08:00
cflmy f4a404bae9 fix(sandbox): add platform-neutral strict child inspection
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
The CLI referenced Linux-only marker_present/verify_active directly, so
macOS and Windows builds failed with E0425. Add StrictChildState and
inspect_strict_child() to marqdo-sandbox: Linux performs the full identity
verification, while every other platform rejects even a forged
MARQDO_STRICT_ACTIVE marker. The Linux-only functions are no longer
re-exported from the crate root, so future callers cannot bypass the
platform boundary.

This is step 1 of the CI/security RFC (B01).
2026-11-21 16:36:49 +08:00
cflmy 32d8fe1f7a docs(status): record the merged master series and follow-up fixes
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
Master now contains the fast-forwarded RFC-LANG-0002 series; the former
semantics-v2-refactor branch was deleted, so only master remains.

- STATUS lists the full linear series (docs, implementation, expression
  grammar, status, seven follow-up fixes) and the verification at 9c1723d:
  fmt/clippy clean, 507 passed / 0 failed / 2 ignored, stdlib digest verified
- counts updated to 49 compile-negative fixtures and 507 workspace tests
  across README, COMPATIBILITY, CHANGELOG, CURRENT and the migration guide
- green evidence pins code commit 9c1723d and lists the follow-up fixes;
  CHANGELOG gains the follow-up correctness section
- migration inventory regenerated: 1019 classified files, 0 pending
- CURRENT.md baseline is the 40-hex 9c1723d commit required by the docs gate

Verified: docs_consistency 4/4, fmt, clippy -D warnings and the full
workspace suite green; python3 tools/pack-stdlib.py PASS.
2026-11-21 16:01:19 +08:00
cflmy 9c1723d44f test(markdown): cover the removed syntax-selector key probe
frontmatter::has_key() is what rejects marqdo_semantics/syntax directives;
add unit coverage for a present key, an absent key and invalid YAML (which
reports no key rather than panicking).

Verified: cargo test -p marqdo-markdown and the workspace suite green
(507 passed, 0 failed, 2 ignored).
2026-11-21 15:57:13 +08:00
cflmy b3429cbeb3 fix(frontend): emit diagnostics in the specified stable order
spec/mapping.md §8 requires diagnostics ordered by source position, then
code, so IDE/CI diffs are stable. analyze_source appended diagnostics by
pipeline stage (units, mapping, inference, call validation), so a later
heading error could precede an earlier payload error.

Sort the final list by (span.start, code) with span-less diagnostics last,
and pin the order with a test that mixes an indexing-stage duplicate-heading
error with an earlier mapping-stage syntax error.

Verified: clippy -D warnings clean; cargo test --workspace --locked green.
2026-11-21 15:55:23 +08:00
cflmy 0bbe0f6524 fix(frontend): honour the Setext no-declaration rule; drop dead helper
RFC-LANG-0002 allows a mode declaration only as an ATX heading prefix; a
Setext heading merely inherits the nearest ancestor mode. heading_declaration
was applied to every heading, so a Setext 'title' beginning with @exec/
@knowledge silently switched modes and could create a function.

- skip declaration parsing for Setext headings (they keep their literal name
  and inherited mode); add a regression test
- remove the dead normalized_heading_name helper and its imports, which only
  existed for the removed link-form call resolution

Verified: clippy -D warnings clean; cargo test --workspace --locked green
(505 passed, 0 failed, 2 ignored).
2026-11-21 15:52:46 +08:00
cflmy a3f9770761 fix(frontend): point unmatched end diagnostics at the end payload
A stray end after an earlier statement was reported with the span of the
previous payload (position-1) and a confusing 'ends outside its owning
function segment' message. Report it on the end payload itself with the
standard sem.unmatched_end wording, and pin the span with a unit test.

Verified: clippy -D warnings clean and cargo test --workspace --locked
green (62 targets).
2026-11-21 15:50:46 +08:00
cflmy 9fb97bc2df fix(frontend): recognize signatures only from Exec payloads
signature::detect() scanned the function body independently of heading
modes, skipping only level>=3 headings. A 'params(...)' code span in an
### @knowledge section was therefore taken as the function signature
('inspect --signature' reported it as explicit), even though that span is
inert documentation and is never executed.

Unify recognition with the executable traversal:

- MappedBody now reports the params(...) payload the mapper accepted; the
  mapper already enforces Exec mode, directness, first-function-wide
  position and single declaration
- signature::from_payload validates that text and rewrites default-value
  spans; signature::detect is removed along with its mode-blind heuristic
- analyze uses the mapper result, so Knowledge sections cannot declare a
  signature and prose/Knowledge subsections may still precede the first
  Exec declaration
- tests: Knowledge params ignored; Exec params after prose/Knowledge is the
  signature; spec/mapping.md states the recognition rule

Verified: clippy -D warnings clean; cargo test --workspace --locked green
(62 targets).
2026-11-21 15:48:37 +08:00
cflmy d5002ac826 docs(code): remove stale pre-RFC role wording and fix review aid
Follow-up cleanup of text left by the concurrent implementation:

- marqdo-markdown AST, marqdo-ir statements/branch arms, frontend units,
  payload and signature module docs now describe RFC-LANG-0002 roles
  (payload carriers, heading modes, if/else groups, while/end, params)
- marqdo-expression public docs list parse_payload_statement and stop
  describing a Markdown-link call that no longer exists
- retired markdown.invalid_construct/markdown.inert_executable constants
  are documented as decode-only history for old Analysis/Trace JSON
- drop the stale 'expected red' comment in the CLI integration test
- the dev-only .mq.md review aid no longer reports semicolons inside
  string literals, and skips AST baselines / expected-invalid negatives

Verified: fmt, clippy -D warnings and cargo test --workspace --locked green.
2026-11-21 15:44:31 +08:00
cflmy 361a6cc341 refactor(ir): drop the dead call-statement representation
The pre-RFC link-form block-quote call was removed by RFC-LANG-0002, but the
IR still carried Stmt::Call, CallSite and CallTarget with matching branches in
the runtime, infer, call validation, loader, analysis projection and JSON
output. No mapper produces that variant any more, so it was a second call
representation that could never be reached.

- remove Stmt::Call, CallSite and CallTarget from marqdo-ir
- remove the runtime exec_call_site path and every exhaustive match arm
- remove the 'call_site' Analysis node and its JSON helpers
- state in spec/analysis-v2.md that calls are expressions; keep v1 as history
  with a note that call_site is not produced by the current language

Verified: cargo clippy --workspace --all-targets -D warnings clean and
cargo test --workspace --locked green (62 targets).
2026-11-21 15:39:29 +08:00
cflmy b38b769932 fix(frontend): enforce function-wide params position
The concurrent mapper tracked the first params declaration in the Mapper (so
duplicates across heading segments were caught) but checked 'first payload'
with a per-segment vector index. A params(...) in a later ### @exec segment
was therefore accepted even after an earlier segment had executed statements,
and a duplicate could even be reported as a position error.

- track whether any statement has been lowered for the function and require
  params(...) to come before it, across all heading segments
- check duplicates first, then execution order, then the in-segment index
- add negative fixtures 48 (after an earlier segment) and 49 (duplicate
  across segments); update the corpus count and spec wording

Verified: cargo test --workspace --locked green (62 targets).
2026-11-21 15:36:30 +08:00
cflmy bbeb1de768 docs(status): record the landed RFC-LANG-0002 implementation
Update status, compatibility, changelog, README, architecture overview,
ADR status, migration guide, contributor/CI/security/release notes and
the per-file inventory from 'migration pending' to the verified state:

- language SHA ffd1cc9: fmt/clippy clean, workspace 501 passed / 0 failed
  / 2 ignored, stdlib digest regenerated and pack-verified
- green evidence report added; the prior intermediate evidence is marked
  superseded
- inventory regenerated: 1015 classified files, 0 pending
- explicit non-claims: Windows/macOS native lanes and the disposable-VM
  strict lane were not re-run at this SHA, so the release decision remains
  DO NOT RELEASE

Verified: cargo test -p marqdo-cli --test docs_consistency (4/4) and
git diff --check.
2026-11-21 12:31:38 +08:00
cflmy ffd1cc9862 refactor(expression): own the v2 payload statement grammar
Move keyword recognition out of the Markdown mapper and into the single
expression/statement parser:

- add parse_payload_statement + PayloadStatement for if/else/while/end/
  return/params/ordinary; keyword matching uses complete tokens, requires
  whitespace before a control expression, rejects trailing tokens/colons
  and reserves the statement words in expression positions
- keep expression spans payload-relative so the frontend can rewrite them
  onto the file; malformed params(...) interiors stay owned by the
  signature parser (one sem.syntax diagnostic, not a duplicate)
- simplify the frontend mapper to control-stack lowering and mapped
  semantic diagnostics only
- emit sem.no_entry_function from run when no Exec H2 main exists
- add else-under-while, params-in-quote, Knowledge-inertness and
  no-entry tests; update the corpus counts
- make the EBNF parameter type annotation optional (params(n) is valid)

Verified: cargo fmt --check, cargo clippy -D warnings and
cargo test --workspace --locked: 501 passed, 0 failed, 2 ignored.
2026-11-21 12:29:07 +08:00
cflmy e029853e32 feat(lang): implement RFC-LANG-0002 unified semantics (breaking)
One Markdown-to-program mapping, no version selector or legacy mapper.

Frontend/runtime:
- resolve @exec/@knowledge heading declarations with nearest-ancestor
  inheritance; only an Exec H2 is callable; add UnitKind::Knowledge and
  sem.* diagnostics (mode marker, unit name, exec-without-function,
  payload nesting/empty, syntax, control stack, params position,
  version directive, inferred-signature lint)
- extract Strong/Emph/inline Code as one-statement carriers, refuse
  nested carriers, and honour heading segment boundaries so an if/while
  cannot be closed across a heading
- lower if/else/while/end to the existing Branch/Loop/Return IR with a
  compile-time control stack; keep payload-relative source spans through
  expression parsing and rewrite them onto the file
- accept params(...) only as the first direct Exec H2 payload; keep the
  existing free-variable inference and named-only rule for inferred
  signatures; fix duplicate-mode/else/end and Knowledge-body inertness
- reject marqdo_semantics/syntax front matter directives

Data and tests:
- migrate stdlib/**/*.mq.md, regenerate stdlib.json digest
- migrate examples and the full semantics/negative/runtime corpus to
  the only current syntax; expected values/runtime codes preserved
- migrate Rust-embedded .mq.md fixtures, init template, CLI help and
  security-lane fixtures

Verified: cargo fmt --check, cargo clippy -D warnings and
cargo test --workspace --locked (green).
2026-11-21 12:26:46 +08:00
cflmy 49835424ee docs(spec): adopt RFC-LANG-0002 unified language semantics
Replace the Strong/Emph/List/Quote mapping in place with the single
@exec/@knowledge heading-mode model, unified payload carriers and keyword
control flow.

- add the authoritative RFC, ADR-0052 and a historical ADR index
- rewrite spec/mapping.md and spec/expression.ebnf as the normative
  mapping/statement grammar; update evaluation, references, math,
  foreign, package-format, plugin-abi, security and schema notes
- rewrite README, AGENTS, COMPATIBILITY and CHANGELOG for the breaking
  language change; update the architecture index/overview and active docs
- mark superseded planning docs, compute RFCs, evidence and syntax-role
  ADRs as historical/superseded without rewriting their decisions
- add migration guide, per-file inventory and dev-only audit scripts

The frontend/stdlib migration is completed in the following commits; this
commit contains documentation only.
2026-11-21 12:24:19 +08:00
Marqdo Agent 0d8ed44010 feat(http): parse and build HTTP/1.1 framing by bytes 2026-11-21 11:05:15 +08:00
Marqdo Agent 28b178fc70 feat(regex): add host-backed groups, alternation and captures 2026-11-21 10:58:06 +08:00
Marqdo Agent 94a13880c4 feat(json,csv): bound depth and add multi-value/multi-row APIs 2026-11-21 10:51:55 +08:00
Marqdo Agent efbb8007e1 feat(compress): add bounded gzip product codec 2026-11-21 10:48:27 +08:00
Marqdo Agent ff3f19a2ea feat(net): add DNS, UDP, TCP listener and TLS endpoints 2026-11-21 10:43:08 +08:00
Marqdo Agent ee0c6cc80a feat(random): add explicitly granted OS entropy API 2026-11-21 10:25:11 +08:00
Marqdo Agent 6338fa10e9 feat(fs): add append, read_at and recursive walk 2026-11-21 10:20:07 +08:00
Marqdo Agent 7859c26213 refactor(cli,host): finish command, dispatch and package module split 2026-11-21 08:59:50 +08:00
Marqdo Agent 8a0c068856 docs(stdlib): list every product test and fix digest snippet 2026-11-21 08:55:09 +08:00
Marqdo Agent 550e66b1e2 refactor(cli,host): split command entry points and provider/effects modules 2026-11-21 08:54:29 +08:00
Marqdo Agent 791819a725 refactor(cli,host): split sandbox, resource-open and process modules 2026-11-21 08:49:25 +08:00
Marqdo Agent 3ef18fb496 feat(process): add explicitly granted process API 2026-11-21 08:46:02 +08:00
Marqdo Agent 9922bfae98 feat(stdlib): add explicitly granted env API 2026-11-21 08:40:54 +08:00
Marqdo Agent ae69391bc0 feat(stdlib): add datetime and project-mode clock APIs 2026-11-21 08:36:40 +08:00
Marqdo Agent cffb6790be feat(stdlib): add path and regex product modules 2026-11-21 08:32:32 +08:00
Marqdo Agent bde6f1d0ba feat(stdlib): add JSON and CSV product parsers 2026-11-21 08:23:54 +08:00
Marqdo Agent fbe36bcc43 feat(resource): add grant-checked fs resource operations 2026-11-21 08:20:31 +08:00
Marqdo Agent cd90f059b4 feat(stdlib): add bounded std:fs/file read, write and copy 2026-11-21 08:10:10 +08:00
Marqdo Agent 8dd3d5eb76 docs(evidence): refresh the migration source baseline 2026-11-21 01:56:57 +08:00
Marqdo Agent 0d40de7645 test(stdlib): add the demo module to the product lane 2026-11-21 01:56:04 +08:00
Marqdo Agent 7a93fe8e7b docs(evidence): record the doc and stdlib test migration gate 2026-11-21 01:54:20 +08:00
Marqdo Agent f0c25be01d docs(tests): map architecture and e2e gates to executable lanes 2026-11-21 01:51:51 +08:00
Marqdo Agent 04eae8746e test(stdlib): cover every bundled module through std imports 2026-11-21 01:50:01 +08:00
Marqdo Agent a8ccac9e67 test(architecture): gate stdlib layout, digest and resource endpoints 2026-11-21 01:45:07 +08:00
Marqdo Agent ce70bc4c42 test(docs): gate the current architecture and evidence links 2026-11-21 01:38:21 +08:00
Marqdo Agent c2730ea445 docs(architecture): consolidate current state and history 2026-11-21 01:36:42 +08:00
cflmy 24c88bcc78 fix(resource): reject provider read and write over-reports
CI / fmt + clippy + test (Linux x86_64) (push) Waiting to run
CI / schema instance validation + consumers (push) Waiting to run
CI / runtime contract (macos-latest) (push) Waiting to run
CI / runtime contract (windows-latest) (push) Waiting to run
ci/woodpecker/push/woodpecker Pipeline was successful
2026-11-21 01:05:14 +08:00
cflmy 6aa3d605af fix(stdlib): return the validated write-all byte count 2026-11-21 01:02:45 +08:00
cflmy e6b5678350 fix(stdlib): strip CR from console lines 2026-11-21 00:59:58 +08:00
cflmy 4a0caabc03 fix(runtime): keep resource.read destination bounds strict 2026-11-21 00:58:42 +08:00
cflmy 23e68cba5f docs(resource): record stdio stream endpoints and stdlib APIs 2026-11-21 00:58:34 +08:00
cflmy 02aecaa747 test(conformance): load nested stdlib imports without false cycles 2026-11-21 00:58:34 +08:00
cflmy 64a6b48866 fix(resource): share one stdin cursor across handles 2026-11-21 00:49:07 +08:00
cflmy 3b13d8499a feat(stdlib): add streaming IO and console line APIs 2026-11-21 00:48:46 +08:00
cflmy aed6a80ba7 feat(resource): add stdin and stderr stdio endpoints 2026-11-21 00:36:25 +08:00
548 changed files with 29346 additions and 6298 deletions

No files matched your search

+83 -2
View File
@@ -24,7 +24,25 @@ jobs:
- name: Clippy
run: cargo clippy --workspace --all-targets -- -D warnings
- name: Tests
run: cargo test --workspace
shell: bash
run: |
set +e
CARGO_TERM_COLOR=never cargo test --workspace 2>&1 | tee workspace-tests.log
status=${PIPESTATUS[0]}
set -e
if [ "$status" -ne 0 ]; then
python3 tools/ci/annotate_workspace_tests.py \
workspace-tests.log "Linux workspace tests"
fi
exit "$status"
- name: Documentation consistency
run: cargo test -p marqdo-cli --test docs_consistency
- name: Architecture boundaries
run: cargo test -p marqdo-cli --test architecture_boundaries
- name: CI evidence tool tests
run: python3 -m unittest discover -s tools/ci -p "test_*.py"
- name: Whole stdlib product tests
run: cargo test -p marqdo-cli --test stdlib_product
- name: Validate public JSON schemas
run: |
python3 - <<'PY'
@@ -116,7 +134,17 @@ jobs:
(command -v python3 && python3 --version) || echo "python3: unavailable"
(command -v node && node --version) || echo "node: unavailable"
- name: Workspace tests
run: cargo test --workspace
shell: bash
run: |
set +e
CARGO_TERM_COLOR=never cargo test --workspace 2>&1 | tee workspace-tests.log
status=${PIPESTATUS[0]}
set -e
if [ "$status" -ne 0 ]; then
python3 tools/ci/annotate_workspace_tests.py \
workspace-tests.log "Workspace tests on ${{ matrix.os }}"
fi
exit "$status"
- name: Portable CLI, analysis and trace smoke
shell: bash
run: |
@@ -138,6 +166,59 @@ jobs:
test "$code" -eq 1
echo "$output" | grep -qi "strict"
echo "$output"
- name: Strict JSON refusal is exactly one document
shell: bash
run: |
set -euo pipefail
set +e
json=$(cargo run -q -p marqdo-cli -- run examples/hello.mq.md --mode strict --json)
code=$?
set -e
test "$code" -eq 1
JSON_DOC="$json" python3 - <<'PY'
import json, os
document = json.loads(os.environ["JSON_DOC"])
assert document["success"] is False, document
assert document["command"] == "run", document
assert document["code"] == "strict.probe_unavailable", document
PY
release-evidence:
name: release evidence manifest
needs: [test, schema, cross-platform]
if: always()
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build SHA-bound manifest
shell: bash
env:
TARGET_SHA: ${{ github.sha }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
LINUX_RESULT: ${{ needs.test.result }}
SCHEMA_RESULT: ${{ needs.schema.result }}
PORTABLE_RESULT: ${{ needs.cross-platform.result }}
run: |
set -euo pipefail
state() {
case "$1" in
success) echo PASS ;;
failure) echo FAIL ;;
*) echo UNVERIFIED ;;
esac
}
linux="$(state "$LINUX_RESULT")"
portable="$(state "$PORTABLE_RESULT")"
if [ "$(state "$SCHEMA_RESULT")" != "PASS" ]; then
linux="FAIL"
fi
python3 tools/ci/release_evidence.py --target-sha "$TARGET_SHA" --out "docs/evidence/ci/$TARGET_SHA/manifest.json" --gate "linux=$linux:$TARGET_SHA:$RUN_URL" --gate "windows=$portable:$TARGET_SHA:$RUN_URL" --gate "macos=$portable:$TARGET_SHA:$RUN_URL" --gate "strict_vm=UNVERIFIED::${{ github.server_url }}/${{ github.repository }}/actions/workflows/security-linux.yml" --gate "performance=UNVERIFIED::"
- name: Upload release manifest
uses: actions/upload-artifact@v4
with:
name: release-evidence-${{ github.sha }}
path: docs/evidence/ci/${{ github.sha }}/manifest.json
if-no-files-found: error
release-candidate:
name: release candidate artifact (${{ matrix.os }})
+70
View File
@@ -0,0 +1,70 @@
name: Linux performance measurements (G4)
# This workflow needs a dedicated, fixed Linux performance node. It is never
# run on GitHub-hosted runners or on TCG/QEMU guests: those numbers are not
# comparable with a native release baseline. A short smoke run is not release
# evidence; runner.py marks it SMOKE unless 5 warmups / 20 samples and
# --approved-node are used.
#
# Trust model: the node carries no untrusted PR code and no repository
# secrets. It must not share hardware with the G3 security VM.
on:
workflow_dispatch:
inputs:
platform_id:
description: "Fixed node identifier, e.g. linux-x86_64-fixed-node-01"
required: true
type: string
cpu_list:
description: "taskset CPU list, e.g. 2,3 (empty = unbound smoke only)"
required: false
default: ""
type: string
baseline_run_id:
description: "Optional previous run id whose artifact is the baseline"
required: false
default: ""
type: string
permissions:
contents: read
jobs:
measure:
name: correctness-verified performance sample
runs-on: [self-hosted, linux, marqdo-perf]
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Toolchain report
run: |
rustc -vV
uname -a
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor 2>/dev/null || true
cat /proc/loadavg
- name: Measure
shell: bash
env:
PLATFORM_ID: ${{ inputs.platform_id }}
CPU_LIST: ${{ inputs.cpu_list }}
run: |
set -euo pipefail
args=(--warmups 5 --samples 20 --approved-node --platform-id "$PLATFORM_ID" --out perf.json)
if [ -n "$CPU_LIST" ]; then
args+=(--cpu-list "$CPU_LIST")
else
echo "error: a release measurement requires a fixed --cpu-list" >&2
exit 1
fi
python3 tools/perf/runner.py "${args[@]}"
- name: Upload report
if: always()
uses: actions/upload-artifact@v4
with:
name: marqdo-perf-${{ inputs.platform_id }}
path: perf.json
if-no-files-found: warn
retention-days: 90
+89 -4
View File
@@ -5,27 +5,112 @@ name: Linux strict security lane (G-C)
# mount, PID and network namespaces and the Bubblewrap AppArmor profile.
# A missing runner or a skipped test fails the gate; it is never counted as a
# pass. See docs/security-gate.md and ADR-0021.
#
# Trust model: the runner registration must be issued by a trusted, short-lived
# orchestrator and the VM destroyed after the job. A persistent self-hosted
# runner with GitHub/SSH/cloud credentials is NOT acceptable. The workflow
# records tested_sha separately from workflow_sha and never runs on PRs.
on:
push:
branches: [master]
workflow_dispatch:
schedule:
- cron: "17 3 * * 1"
inputs:
source_sha:
description: "Exact reviewed commit SHA"
required: true
type: string
runner_image_sha256:
description: "Pinned VM base-image digest (evidence, not a secret)"
required: false
default: "unrecorded"
type: string
permissions:
contents: read
jobs:
strict-security:
name: strict isolation escape matrix
if: github.event_name != 'pull_request'
runs-on: [self-hosted, linux, marqdo-security]
timeout-minutes: 45
steps:
- uses: actions/checkout@v4
- name: Host capability report
with:
ref: ${{ github.event_name == 'workflow_dispatch' && inputs.source_sha || github.sha }}
persist-credentials: false
- name: Bind tested SHA and archive
shell: bash
run: |
set -euo pipefail
tested_sha="$(git rev-parse HEAD)"
expected="${{ github.event_name == 'workflow_dispatch' && inputs.source_sha || github.sha }}"
if [ "$tested_sha" != "$expected" ]; then
echo "tested SHA $tested_sha does not match expected $expected" >&2
exit 1
fi
printf 'TESTED_SHA=%s\n' "$tested_sha" >> "$GITHUB_ENV"
printf 'WORKFLOW_SHA=%s\n' "$GITHUB_SHA" >> "$GITHUB_ENV"
archive="$RUNNER_TEMP/marqdo-source-$tested_sha.tar"
git archive --format=tar "$tested_sha" > "$archive"
printf 'SOURCE_ARCHIVE_SHA256=%s\n' \
"sha256:$(sha256sum "$archive" | cut -d' ' -f1)" >> "$GITHUB_ENV"
- name: Host capability report
shell: bash
run: |
set -euo pipefail
test "$(id -u)" -ne 0
id
uname -a
systemd-detect-virt --container || true
bwrap --version
cc --version | head -1
rustc -vV
cat /proc/self/uid_map
findmnt -t cgroup2 || true
- name: Run G-C matrix
run: ./tests/security/run-lane.sh
shell: bash
env:
MARQDO_STRICT_EVIDENCE: ${{ runner.temp }}/strict-evidence.json
MARQDO_STRICT_LANE_LOG: ${{ runner.temp }}/gc-lane.log
MARQDO_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
MARQDO_RUNNER_KIND: disposable-vm
MARQDO_RUNNER_IMAGE_SHA256: ${{ inputs.runner_image_sha256 || 'unrecorded' }}
run: |
set -euo pipefail
export MARQDO_SOURCE_ARCHIVE="$RUNNER_TEMP/marqdo-source-${TESTED_SHA}.tar"
./tests/security/run-lane.sh 2>&1 | tee "$MARQDO_STRICT_LANE_LOG"
- name: Verify strict evidence matches the tested SHA
if: success()
shell: bash
env:
MARQDO_STRICT_EVIDENCE: ${{ runner.temp }}/strict-evidence.json
run: |
set -euo pipefail
python3 - <<'PY'
import json, os
path = os.environ["MARQDO_STRICT_EVIDENCE"]
with open(path, encoding="utf-8") as handle:
evidence = json.load(handle)
assert evidence["schema"] == "marqdo.strict-evidence.v1", evidence
assert evidence["run"]["outcome"] == "PASS", evidence
assert evidence["tested_sha"] == os.environ["TESTED_SHA"], evidence
assert evidence["environment"]["uid"] != 0, evidence
print("strict evidence verified for", evidence["tested_sha"])
PY
- name: Upload strict evidence
if: always()
uses: actions/upload-artifact@v4
with:
name: strict-evidence-${{ inputs.source_sha || github.sha }}
path: |
${{ runner.temp }}/strict-evidence.json
${{ runner.temp }}/gc-lane.log
if-no-files-found: warn
retention-days: 30
+11
View File
@@ -10,6 +10,8 @@ examples/plugin-multi-export/solver.so
# Local cargo home used on hosts with a read-only $HOME
.cargo-home/
# Local rustup home used for cross-target `cargo check` verification
.rustup-home/
# Generated plugin fixture manifests/artifacts
examples/plugin-demo/build/
@@ -24,3 +26,12 @@ dist/
# Local-only credentials and environment files
.secrets/
.env.local
# stdlib product test output
stdlib/tests/out/
# Python tooling caches and perf capture files
__pycache__/
*.pyc
.perf-out-*
.perf-err-*
+5 -1
View File
@@ -10,12 +10,16 @@ steps:
commands:
- rustup component add clippy rustfmt
- apt-get update
- apt-get install -y --no-install-recommends curl ca-certificates
- apt-get install -y --no-install-recommends curl ca-certificates python3
- curl -fsSL https://deb.nodesource.com/setup_22.x | bash -
- apt-get install -y --no-install-recommends nodejs
- cargo fmt --all -- --check
- cargo clippy --workspace --all-targets -- -D warnings
- cargo test --workspace --locked
- cargo test -p marqdo-cli --test docs_consistency
- cargo test -p marqdo-cli --test architecture_boundaries
- python3 -m unittest discover -s tools/ci -p "test_*.py"
- cargo test -p marqdo-cli --test stdlib_product
build:
image: rust:1.100-bookworm
+52
View File
@@ -0,0 +1,52 @@
# G4 fixed performance node.
#
# Manual only: this workflow never runs on push or pull_request and therefore
# never executes untrusted PR code on the designated performance agent. The
# repository currently has one Woodpecker agent (agent_id 1), so it is the
# fixed node; the CPU affinity and hardware/toolchain manifest are recorded in
# every report. Do not compare its timings with another machine or with
# QEMU/TCG.
when:
event: [manual]
branch: [master]
steps:
measure:
image: rust:1.100-bookworm
commands:
- apt-get update
- apt-get install -y --no-install-recommends python3 util-linux ca-certificates
- cargo build --release --locked -p marqdo-cli
- |
set -eu
allowed="$(awk '/^Cpus_allowed_list:/ {print $2}' /proc/self/status)"
cpu_list="${MARQDO_PERF_CPU_LIST:-$(printf '%s' "$allowed" | cut -d, -f1 | cut -d- -f1)}"
echo "G4_NODE host=$(uname -n) kernel=$(uname -r) arch=$(uname -m) allowed_cpus=$allowed pin=$cpu_list"
echo "G4_CPU $(awk -F: '/model name/ {print $2; exit}' /proc/cpuinfo | xargs)"
echo "G4_GOVERNOR $(cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor 2>/dev/null || echo unavailable)"
python3 tools/perf/runner.py --no-build --warmups 5 --samples 20 \
--approved-node --platform-id woodpecker-agent-1 \
--cpu-list "$cpu_list" --out perf.json
python3 - <<'PY'
import json
with open("perf.json", encoding="utf-8") as handle:
report = json.load(handle)
print(
"G4_STATUS "
f"{report['status']} correctness={report['correctness']} "
f"platform={report['platform_id']} "
f"binary={report['binary']['sha256']}"
)
for workload in report["workloads"]:
print(
"G4_WORKLOAD "
f"{workload['name']} {workload['status']} "
f"p50={workload.get('median_ms')} "
f"p95={workload.get('p95_ms')} "
f"rss={workload.get('max_rss_kb_median')}"
)
PY
echo "G4_REPORT_BEGIN"
cat perf.json
echo "G4_REPORT_END"
+15 -13
View File
@@ -1,6 +1,6 @@
# Marqdo CLI — Agent / Contributor Operating Rules
**Start here:** read `docs/DEVELOPMENT_BLUEPRINT.md` before proposing code. It is the project architecture baseline. Then read `docs/STATUS.md` (what is actually implemented), the relevant `spec/` files, `docs/adr/` for frozen decisions, `docs/contributor-guide.md` for the workflow, and `tests/README.md` plus the conformance fixtures. Do not assume any current implementation or legacy Marqdo syntax is authoritative.
**Start here:** read `docs/architecture/RFC-LANG-0002-unified-semantics.md` — it is the single current language authority — and its normative summary `spec/mapping.md` + `spec/expression.ebnf`. Then read `docs/DEVELOPMENT_BLUEPRINT.md` (architecture baseline; its language-mapping sections are historical), `docs/STATUS.md` (what is actually implemented, including the landed RFC-LANG-0002 migration), the relevant `spec/` files, `docs/adr/` for decisions at their date, `docs/contributor-guide.md`, and `tests/README.md` plus the conformance fixtures. Do not assume any current implementation or legacy Marqdo syntax is authoritative.
For next-phase work, the single authoritative plan is
`docs/STABLE_RELEASE_IMPLEMENTATION.md` (stable-release gates G-A/G-B/G-C,
@@ -21,29 +21,31 @@ correctness and release evidence before expanding the surface.
1. New independent language/toolchain. **Do not silently modify `cflmy/marqdo`** or claim compatibility with its syntax, package format or plugin ABI.
2. CommonMark/GFM/Math AST is the sole Markdown syntax structure. Do **not** implement a second Markdown parser or use regex heuristics to decide whether text is executable.
3. Executable roles are based on AST type and valid structural context: `Strong` execution, `Emph` return, ordered list branch, bullet list loop, blockquote invocation, heading scoping. Invalid executable shapes are compile errors, not ordinary prose fallbacks.
4. Prose and executable nodes may mix freely in one function/branch/loop body. Prose itself never causes an action.
3. The language has **one** Markdown→program mapping (RFC-LANG-0002): `@exec`/`@knowledge` heading modes with ancestor inheritance; only an Exec H2 is callable; `Strong`, `Emph` and inline `Code` are interchangeable one-statement payload carriers; `if`/`else`/`while`/`end`/`return`/`params` decide control flow; lists, block quotes and paragraphs are presentation containers with no instruction type. Invalid payloads are hard errors, never prose fallbacks. Do not reintroduce Strong=execute, Emph=return, ordered-list=branch, bullet-list=loop or block-quote=call, and do not add a per-file syntax switch, `map_v1`, `--legacy` or a second active spec.
4. Prose and executable payloads may mix freely in one function/branch/loop body. Prose itself never causes an action; executable payloads with no owning H2 Exec function are errors (`sem.exec_without_function`).
5. Marqdo is executed by **its own Rust interpreter**. Python/Node are optional foreign kernels, never the official compiler target.
6. Code fences and math expressions create inert native `Code` and `Formula` values. Evaluation requires explicit calls and relevant capability grants.
7. YAML project/dependency and frontmatter imports must be explicit. Markdown Links are static references, not implicit calls or imports.
6. Code fences and math expressions create inert native `Code` and `Formula` values. Evaluation requires explicit calls and relevant capability grants. Inline `Code` spans are payload carriers only in an Exec scope; fenced code is never executed.
7. YAML project/dependency and frontmatter imports must be explicit. Markdown Links are static references, never calls or imports. Calls are expressions inside executable payloads.
8. Third-party plugins require a versioned **C ABI and SDK**. A C ABI is not a sandbox. Default third-party execution must be isolated; in-process native loading requires explicit trust.
9. Side effects go through `CapabilityHost`; do not invent host bypasses for tests, plugins, math or foreign kernels. `check` and `inspect` cannot execute untrusted project code.
10. Language semantics, package format, ABI, authorization defaults or public CLI behavior changes require an ADR + tests + spec updates.
11. RFC-LANG-0002 is implemented by the frontend, `marqdo-expression`, IR lowering, runtime and the bundled stdlib. Keep `spec/`, implementation, fixtures, stdlib digest and docs in lockstep; there is no legacy mapper or per-file syntax switch to fall back to. Do not make a release claim beyond the evidence recorded in `docs/STATUS.md`.
## Development workflow
1. Identify the affected rule and cite the section of `docs/DEVELOPMENT_BLUEPRINT.md`.
2. Add a small `.mq.md` positive/negative conformance fixture and define the expected Markdown AST and execution IR.
3. Implement only the smallest module that satisfies this fixture. Keep Markdown parser, semantic mapper, runtime and host separated.
4. Run formatting, clippy, unit tests, gold tests and relevant security/ABI compatibility tests (`cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace`, `cargo test -p marqdo-cli`).
5. Write the tradeoff and state of completion in the PR. Never say a feature is finished because a design example looks executable. Update `spec/` and `docs/STATUS.md` when behavior or status changes; the docs describe what the code does and flag blueprint divergences explicitly.
1. Identify the affected rule and cite RFC-LANG-0002 / `spec/mapping.md` (or, for architecture-only changes, the relevant `docs/DEVELOPMENT_BLUEPRINT.md` section).
2. Update the normative spec **in place** first; there is never a second active syntax spec.
3. Add a small `.mq.md` positive/negative conformance fixture and define the expected Markdown AST, execution IR and diagnostics.
4. Implement only the smallest module that satisfies this fixture. Keep Markdown parser, semantic mapper, expression parser, runtime and host separated.
5. Run formatting, clippy, unit tests, golden tests and relevant security/ABI compatibility tests (`cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace`, `cargo test -p marqdo-cli`).
6. Write the tradeoff and state of completion in the PR. Never say a feature is finished because a design example looks executable. Update `spec/` and `docs/STATUS.md` when behavior or status changes; the docs describe what the code does and flag divergences explicitly.
## Unresolved design topics
The blueprint's section **17** lists issues that need separate Accepted ADRs (e.g. emphasis conflicts, parameter inference ambiguity, plugin memory ownership, strict platform isolation). Use its proposed baseline for prototypes, but do not freeze these topics through undocumented code.
RFC-LANG-0002 resolves the old emphasis/control-role conflict: `Strong`, `Emph` and inline `Code` are equivalent carriers and controls are keywords with explicit `end`. Remaining topics still need separate Accepted ADRs (parameter inference policy, plugin memory ownership, strict platform isolation, future `elif`/`for`/`break`, knowledge anchors). Do not freeze them through undocumented code.
## Priorities
P0 AST/grammar contracts → P1 independent interpreter → P2 references/CLI → P3 package/env → P4 sandbox → P5 math/foreign → P6 plugin ABI/SDK → P7 performance/release. Design the Host security boundary and ABI early, even if full implementation arrives later.
P0 language contract maintenance (RFC-LANG-0002, single grammar, corpus and stdlib) → P1 independent interpreter → P2 references/CLI → P3 package/env → P4 sandbox → P5 math/foreign → P6 plugin ABI/SDK → P7 performance/release. Design the Host security boundary and ABI early, even if full implementation arrives later.
**Never** translate Marqdo into Python as the official execution method; **never** silently downgrade an invalid execution node to prose or a requested strict sandbox to an unrestricted process.
**Never** translate Marqdo into Python as the official execution method; **never** silently downgrade an invalid payload to prose, a requested strict sandbox to an unrestricted process, or the current syntax to a legacy mapper.
+88 -2
View File
@@ -5,6 +5,83 @@ currently `0.1.0`; no release tag has been cut. Dates use ISO 8601.
## Unreleased (stabilization candidate)
### Breaking changes — language source syntax (RFC-LANG-0002)
**The `.mq.md` source language changed incompatibly. Old source files are not
executable.** The current contract is
[`docs/architecture/RFC-LANG-0002-unified-semantics.md`](docs/architecture/RFC-LANG-0002-unified-semantics.md)
(accepted by [`docs/adr/0052-unified-markdown-semantics.md`](docs/adr/0052-unified-markdown-semantics.md)),
summarized normatively in [`spec/mapping.md`](spec/mapping.md) and
[`spec/expression.ebnf`](spec/expression.ebnf).
- Removed the old mapping in which `Strong` executes, `Emph` returns, an
ordered list is a branch, a bullet list is a loop, a block quote is a call and
the first standalone code span is a signature.
- Headings now declare `@exec` / `@knowledge`; unmarked headings inherit the
nearest ancestor's mode, and the document root is Knowledge.
- Only an Exec H2 is a callable function; H2 Knowledge is inert and H3–H6 are
non-callable sections. A payload with no owning Exec H2 is
`sem.exec_without_function`.
- `Strong`, `Emph` and inline `Code` are interchangeable carriers of exactly one
statement; the AST kind no longer selects the operation.
- Control flow comes from keywords in the payload: `if`/`else`/`while`/`end`
(explicitly closed) plus `return <expr>` and `params(...)`. Ordered lists,
bullet lists, block quotes and paragraphs are presentation containers with no
instruction type; a plain list or quote produces zero statements.
- `params(...)` is the only function-signature declaration, legal only as the
first direct payload of an Exec H2. An Emph/Strong payload in a signature
position is no longer special.
- There is no syntax version switch, per-file selector, `--legacy` path or
fallback mapper; `marqdo_semantics`/syntax-selection front matter is rejected.
**Implementation status: landed** in the `semantics-v2-refactor` history
(docs/spec `4983542`, frontend/runtime/stdlib/corpus `e029853`, expression-owned
statement grammar `ffd1cc9`, then the follow-up fixes through `9c1723d`). At
`9c1723d`,
`cargo test --workspace --locked` is green (507 passed, 0 failed, 2 ignored),
`cargo clippy --workspace --all-targets -- -D warnings` is clean and the
stdlib digest is regenerated. See
[`docs/STATUS.md`](docs/STATUS.md) and
[`docs/evidence/2026-11-21-semantics-v2-green.md`](docs/evidence/2026-11-21-semantics-v2-green.md).
Platform CI and the disposable-VM strict lane were not re-run at this SHA.
**Follow-up correctness fixes in the same series:**
- `params(...)` is enforced as the first function-wide direct Exec H2 payload, so
a later `### @exec` segment cannot add a signature after earlier statements;
- a `params(...)` code span in a `@knowledge` section is inert documentation and
never becomes a signature (`inspect --signature` now reports `inferred`);
- a stray `end` diagnostic points at the `end` payload instead of the previous
statement;
- Setext headings inherit but never declare a mode, as `spec/mapping.md` states;
- diagnostics are emitted in the documented stable order (source position, then
code);
- the unreachable `Stmt::Call`/`CallSite`/`CallTarget` IR left over from the
removed link-form block-quote call was deleted.
**Required migration actions for consumers and the repository:**
1. Rewrite every active `.mq.md` source by AST role (not by a global regex):
stdlib modules, examples, fixtures, project templates, Rust-embedded samples
and documentation snippets.
2. Give callable functions an explicit `## @exec 名称` heading, or place them
under an `# @exec` namespace. Knowledge documents and visual emphasis belong
in `@knowledge` sections; a `Strong`/`Emph` payload in an Exec section is a
statement and may fail as an unknown symbol.
3. Replace italic returns with `return <expr>` in any carrier.
4. Convert an ordered-list branch to a first-true nested `if`/`else`/`end`
chain; convert each bullet-list item to its own `while`/`end` loop, preserving
item order and early-return behavior.
5. Convert block-quote calls to payload calls (`**print(x)**`,
`` `print(x)` `` or `> **print(x)**`) and standalone code-span signatures to
`params(...)` as the first direct Exec H2 payload.
6. Replace old shape expectations with the `sem.*` diagnostics in
`spec/mapping.md` §8; ordinary lists/quotes are now valid inert Markdown.
7. Regenerate `stdlib/stdlib.json` only in the same change that migrates the
stdlib sources, and never use `UPDATE_*_GOLDENS=1` to mask a semantic change.
8. A dependency or lock built from old source syntax is not compatible; relock
and re-verify the package instance identity after migration.
### Added
- `.gitattributes` byte-identity policy: canonical LF for text files and one
@@ -35,7 +112,9 @@ currently `0.1.0`; no release tag has been cut. Dates use ISO 8601.
compile-negative fixtures and 2 runtime-negative fixtures covering
right-associative `**`, unary-minus precedence, parentheses, float negative
exponents, Strong+CodeSpan payloads, branch/loop conditions, inert prose and
code fences, and source spans inside one/two-backtick code spans.
code fences, and source spans inside one/two-backtick code spans. (The
fixtures were migrated to the unified payload/keyword syntax and are green at
`9c1723d`.)
- Independent Tool Protocol v1 consumers `sample-view`, `sample-knowledge` and
`sample-proof` (analysis/trace readers) with install, dispatch and remove
tests; none is a hard-coded CLI command.
@@ -181,6 +260,11 @@ currently `0.1.0`; no release tag has been cut. Dates use ISO 8601.
### Changed
- `spec/mapping.md` and `spec/expression.ebnf` were replaced in place by the
RFC-LANG-0002 contract (heading modes, unified payload carriers, keyword
control flow). `spec/markdown-profile.md` remains `marqdo.md.v1`; only the
Markdown→program mapping changed. These specs describe the target language and
are ahead of the Rust implementation; see "Breaking changes" above.
- Assignment targets are no longer restricted to bare identifiers: a single
index place is accepted at statement level. Member assignment stays a parse
error; chained assignment stays a parse error. `spec/expression.ebnf`,
@@ -196,7 +280,9 @@ currently `0.1.0`; no release tag has been cut. Dates use ISO 8601.
- List-item condition paragraphs now accept only inert description; an
executable `Strong`/`Emph` there is a hard `markdown.invalid_construct`
error instead of being silently swallowed. Condition expression spans are
mapped from the literal bytes inside the code-span backticks.
mapped from the literal bytes inside the code-span backticks. (Pre-RFC
mapping; superseded by RFC-LANG-0002, where lists have no condition head at
all.)
- Linux strict `inspect --platform` remains a snapshot, not a promise. A run
that loses namespace/resources between probe and launch must fail closed.
+59 -11
View File
@@ -1,15 +1,52 @@
# Compatibility and support matrix
Snapshot: workspace version **0.1.0**, code/security SHA **`8af7262`**, no
release tag cut. This file records what was actually verified and separates it
from what is only implemented.
Snapshot: workspace version **0.1.0**, language SHA **`9c1723d`**, security SHA
**`8af7262`**, no release tag cut. This file records what was actually verified and separates it
from what is only implemented. Executed status is in
[`docs/STATUS.md`](docs/STATUS.md).
## Language source syntax (breaking change)
**The `.mq.md` source language is a breaking change.** The current contract is
[RFC-LANG-0002](docs/architecture/RFC-LANG-0002-unified-semantics.md)
(accepted by [ADR-0052](docs/adr/0052-unified-markdown-semantics.md)), with the
normative summaries in [`spec/mapping.md`](spec/mapping.md) and
[`spec/expression.ebnf`](spec/expression.ebnf):
- H1–H6 headings declare `@exec`/`@knowledge` modes; unmarked headings inherit
the nearest ancestor's mode and the document root is Knowledge.
- Only an Exec H2 is a callable function. H2 Knowledge is inert and H3–H6 are
non-callable sections.
- In an Exec scope, `Strong`, `Emph` and inline `Code` are interchangeable
carriers of one statement; the original emphasis role is gone.
- Control flow comes from keywords in the payload (`if`/`else`/`while`/`end`/
`return`/`params(...)`). Ordered lists, bullet lists and block quotes are
inert presentation containers.
- There is no version selector, `--legacy` path or fallback mapper.
Consequences for compatibility:
- **Old pre-RFC `.mq.md` source is not executable.** A file that relied on
`Strong` = execute, `Emph` = return, ordered list = branch, bullet list =
loop, block quote = call or a first code-span signature must be rewritten by
AST role (see `docs/migration/README.md`). A parse failure is a hard
diagnostic, never a fallback to prose.
- The Markdown parser profile stays `marqdo.md.v1`; only the Markdown→program
mapping changed. Rendering of otherwise-ordinary Markdown is unaffected.
- **Implementation state:** RFC-LANG-0002 is implemented by the frontend,
expression parser, IR lowering, runtime and bundled stdlib. At
`9c1723d` the workspace gate is
`cargo test --workspace --locked`: 507 passed, 0 failed, 2 ignored, and the
stdlib digest is regenerated and verified. See
[`docs/STATUS.md`](docs/STATUS.md) and
[`docs/evidence/2026-11-21-semantics-v2-green.md`](docs/evidence/2026-11-21-semantics-v2-green.md).
## Public protocol versions
| Surface | Version | Compatibility rule |
|---|---|---|
| Markdown profile | `marqdo.md.v1` | Frozen parser/options; semantic change needs an ADR, new goldens and a profile id bump. |
| Language / CLI | workspace `0.1.0` | Pre-1.0. CLI flags may change with a changelog note; `strict` refusal is contractual. |
| Markdown profile | `marqdo.md.v1` | Frozen parser/options. The RFC-LANG-0002 mapping rewrite keeps this profile; the source-language break is tracked above, not by a new profile id. |
| Language / CLI | workspace `0.1.0` | Pre-1.0. The `.mq.md` source language changed incompatibly (RFC-LANG-0002); there is no old-syntax execution path. CLI flags may change with a changelog note; `strict` refusal is contractual. |
| `marqdo.lock` | v3 | v1/v2 are rejected with a relock instruction; `run`/`call`/`check` never rewrite or fetch the lock. |
| Analysis | v2 | Consumers reject unknown majors; incompatible field changes require v3. |
| Trace | v2 | Binds to the exact Analysis bytes; unknown majors rejected. |
@@ -26,7 +63,7 @@ portable/tested locally but the native platform lane has not run here;
| Feature | Ubuntu Linux x86_64 | macOS arm64 | Windows x86_64 |
|---|---|---|---|
| Markdown profile / AST goldens (160) | PASS | UNVERIFIED (cfg fix + CRLF fix) | UNVERIFIED (cfg fix + CRLF fix) |
| Rust interpreter / semantics (70) | PASS | UNVERIFIED | UNVERIFIED |
| Rust interpreter / unified semantics (73) | PASS | UNVERIFIED | UNVERIFIED |
| Addressable `Memory` P1 (ADR-0022) | PASS | UNVERIFIED | UNVERIFIED |
| Resource streams CM-A06 (ADR-0025/0026: stdout + project file read/write) | PASS | UNVERIFIED | **FAIL-CLOSED** (`strict` refuses resource opens) |
| `run` / `call` / `check` / `inspect` / `fmt` | PASS | UNVERIFIED | UNVERIFIED |
@@ -72,6 +109,10 @@ reproduce the Windows checkout byte behavior.
## Explicitly unsupported in this candidate
- Pre-RFC `.mq.md` source syntax (Strong-execute / Emph-return / ordered-list
branch / bullet-list loop / block-quote call / first code-span signature).
Old sources are a rewrite task, not a compatibility mode: there is no
`--legacy`, `marqdo_semantics` selector or second mapper.
- OS-level strict isolation on macOS/Windows (Linux only, probe-gated).
- Strict L2: Worker/Foreign execution inside strict (refused in L1).
- Windows Native/Worker ABI runtime support.
@@ -80,11 +121,18 @@ reproduce the Windows checkout byte behavior.
- Registry/package publishing, JIT, Knowledge/Proof/View built-ins,
sessions, and general-purpose CAS. The three `sample-*` tools are test
fixtures for the generic Tool protocol, not product features.
- Windows/macOS release archives from this host; the GitHub release job may
build macOS but has not run for the convergence SHA.
- Tagged Windows/macOS release archives: the native **build/test** matrix is
green at `5fcc111`, but the tag/dispatch release-candidate job has not
produced those archives.
## Release decision
**DO NOT RELEASE.** G1/G2/G3 (native CI) and G9 (self-hosted security-lane CI
URL) have no real remote PASS for this SHA. See
[`docs/evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md`](docs/evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md).
**DO NOT RELEASE.** The RFC-CI-0001 series fixed the platform `E0425` build
break, the strict `--json` empty-stdout defect (ADR-0053) and the native
Windows/macOS portability failures. Linux, Windows and macOS native jobs are
green, and G4 is `MEASURED`/`WARN` on the designated Woodpecker agent
(`1199c88`, 9/9 correctness-verified, pinned CPU). The release still lacks the
**remote G3 security artifact** with a matching `tested_sha`; a local VM PASS
is not a remote G3 proof. The G4 node is a shared powersave host, so its gate
is provisional. See [`docs/STATUS.md`](docs/STATUS.md) §0.1 and
[`docs/evidence/ci/README.md`](docs/evidence/ci/README.md).
Generated
+428 -7
View File
@@ -2,6 +2,21 @@
# It is not intended for manual editing.
version = 3
[[package]]
name = "adler2"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa"
[[package]]
name = "aho-corasick"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c982642fa9e8606056828ee9a8505737230110bb1099153c79efe865c59d12ba"
dependencies = [
"memchr",
]
[[package]]
name = "anstream"
version = "1.0.0"
@@ -38,7 +53,7 @@ version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc"
dependencies = [
"windows-sys",
"windows-sys 0.61.2",
]
[[package]]
@@ -49,7 +64,7 @@ checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d"
dependencies = [
"anstyle",
"once_cell_polyfill",
"windows-sys",
"windows-sys 0.61.2",
]
[[package]]
@@ -58,6 +73,12 @@ version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
[[package]]
name = "base64"
version = "0.22.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"
[[package]]
name = "bitflags"
version = "2.13.2"
@@ -82,6 +103,16 @@ dependencies = [
"unicode-normalization",
]
[[package]]
name = "cc"
version = "1.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "02f0e023adb368345ce0ca6cdfc0f24a06554468d0e8503e14a0841de79bfa69"
dependencies = [
"find-msvc-tools",
"shlex",
]
[[package]]
name = "cfg-if"
version = "1.0.5"
@@ -151,6 +182,22 @@ dependencies = [
"typed-arena",
]
[[package]]
name = "core-foundation"
version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b2a6cd9ae233e7f62ba4e9353e81a88df7fc8a5987b8d445b4d90c879bd156f6"
dependencies = [
"core-foundation-sys",
"libc",
]
[[package]]
name = "core-foundation-sys"
version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b"
[[package]]
name = "cpufeatures"
version = "0.2.17"
@@ -160,6 +207,15 @@ dependencies = [
"libc",
]
[[package]]
name = "crc32fast"
version = "1.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b929dd2a135771f40afabd1073f85eed59c800b7c61901e5c254270497dad64"
dependencies = [
"cfg-if",
]
[[package]]
name = "crypto-common"
version = "0.1.7"
@@ -170,6 +226,12 @@ dependencies = [
"typenum",
]
[[package]]
name = "deranged"
version = "0.5.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9e9de72ce2ad1f90dc62fa25f0f430ef85eb4b0d8fa0be4f30373bc40a21d28e"
[[package]]
name = "digest"
version = "0.10.7"
@@ -199,7 +261,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e35d6e78f4cebb90e51d543db8fe41528851d6deeef6ef21ebd4bfdc1e784776"
dependencies = [
"libc",
"windows-sys",
"windows-sys 0.61.2",
]
[[package]]
@@ -225,12 +287,29 @@ dependencies = [
"syn",
]
[[package]]
name = "find-msvc-tools"
version = "0.1.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1c66915cc5391097de7a2b0b972e8505aa33c0df401fd3a6811dbdb32881ba48"
[[package]]
name = "finl_unicode"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09ded33109e99472f809cb7495544506eea45e366725dacf2a383e184f660456"
[[package]]
name = "flate2"
version = "1.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "626d4928326b646e70f65ed36fc7cf9f206bdb68a0019ee7c6b4709da4e084c4"
dependencies = [
"crc32fast",
"miniz_oxide",
"zlib-rs",
]
[[package]]
name = "generic-array"
version = "0.14.7"
@@ -241,6 +320,17 @@ dependencies = [
"version_check",
]
[[package]]
name = "getrandom"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0"
dependencies = [
"cfg-if",
"libc",
"wasi",
]
[[package]]
name = "getrandom"
version = "0.4.3"
@@ -393,6 +483,8 @@ dependencies = [
name = "marqdo-host"
version = "0.1.0"
dependencies = [
"flate2",
"getrandom 0.4.3",
"glob",
"libloading",
"marqdo-foreign",
@@ -401,6 +493,11 @@ dependencies = [
"marqdo-sandbox",
"marqdo-span",
"marqdo-value",
"rcgen",
"regex",
"rustls",
"rustls-native-certs",
"rustls-pemfile",
"serde",
"serde_json",
"tempfile",
@@ -549,6 +646,16 @@ dependencies = [
"fearless_simd_macros",
]
[[package]]
name = "miniz_oxide"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b63fbc4a50860e98e7b2aa7804ded1db5cbc3aff9193adaff57a6931bf7c4b4c"
dependencies = [
"adler2",
"simd-adler32",
]
[[package]]
name = "num-bigint"
version = "0.4.8"
@@ -560,6 +667,12 @@ dependencies = [
"serde",
]
[[package]]
name = "num-conv"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "521739c6d2bac4aa25192232afe6841231376b2b26d4d9fae5ecf8ca5772e441"
[[package]]
name = "num-integer"
version = "0.1.47"
@@ -602,6 +715,22 @@ version = "1.70.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
[[package]]
name = "openssl-probe"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7e036b411754132fd6b74e94b74e9a13c5cf606fa93d1885c35860d3060820b1"
[[package]]
name = "pem"
version = "3.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d30c53c26bc5b31a98cd02d20f25a7c8567146caf63ed593a9d87b2775291be"
dependencies = [
"base64",
"serde_core",
]
[[package]]
name = "phf"
version = "0.14.0"
@@ -641,6 +770,12 @@ dependencies = [
"siphasher",
]
[[package]]
name = "powerfmt"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4a6394b9e965e73d0a289ee54f589087e2c676aedf60885baf52c76b771e4958"
[[package]]
name = "proc-macro2"
version = "1.0.107"
@@ -665,6 +800,62 @@ version = "6.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf"
[[package]]
name = "rcgen"
version = "0.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75e669e5202259b5314d1ea5397316ad400819437857b90861765f24c4cf80a2"
dependencies = [
"pem",
"ring",
"rustls-pki-types",
"time",
"yasna",
]
[[package]]
name = "regex"
version = "1.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9807af0e638c12e753aa1b4e601791bf2a8f71274023c8bbeec12fc149950857"
dependencies = [
"aho-corasick",
"memchr",
"regex-automata",
"regex-syntax",
]
[[package]]
name = "regex-automata"
version = "0.4.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41d60ef07df457506a0a1a2073112eb4aae5733dbd2813c3da77d1b0fd441158"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax",
]
[[package]]
name = "regex-syntax"
version = "0.8.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d902046e6a28a1e4c3cab97258a0ce11e994903f01c2d97a9447a0b3df85f4f6"
[[package]]
name = "ring"
version = "0.17.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a4689e6c2294d81e88dc6261c768b63bc4fcdb852be6d1352498b114f61383b7"
dependencies = [
"cc",
"cfg-if",
"getrandom 0.2.17",
"libc",
"untrusted",
"windows-sys 0.52.0",
]
[[package]]
name = "rustc-hash"
version = "2.1.3"
@@ -681,7 +872,62 @@ dependencies = [
"errno",
"libc",
"linux-raw-sys",
"windows-sys",
"windows-sys 0.61.2",
]
[[package]]
name = "rustls"
version = "0.23.46"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "48e13bd8c0e9365c43cfa5c9e8f9ad49d3c8444926c9aac819e0e4dc503c8fdf"
dependencies = [
"once_cell",
"ring",
"rustls-pki-types",
"rustls-webpki",
"subtle",
"zeroize",
]
[[package]]
name = "rustls-native-certs"
version = "0.8.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4a073f5dc7469f984c52ad2752b63b0807745133b6de880b7b64c1ac4c48aec4"
dependencies = [
"openssl-probe",
"rustls-pki-types",
"schannel",
"security-framework",
]
[[package]]
name = "rustls-pemfile"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dce314e5fee3f39953d46bb63bb8a46d40c2f8fb7cc5a3b6cab2bde9721d6e50"
dependencies = [
"rustls-pki-types",
]
[[package]]
name = "rustls-pki-types"
version = "1.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "102f35f7de7b06d8c61f1cff6592f5253e0dea431c74867aad5dfd70f44663fb"
dependencies = [
"zeroize",
]
[[package]]
name = "rustls-webpki"
version = "0.103.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3a0760a887c14823fca5a1c970b01d47c328c696c0c64829d1df50e8f853fd5"
dependencies = [
"ring",
"rustls-pki-types",
"untrusted",
]
[[package]]
@@ -699,6 +945,38 @@ dependencies = [
"winapi-util",
]
[[package]]
name = "schannel"
version = "0.1.30"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9cb23c424d8ee841bb4d813677ca237e2a610067d594fec130f87d6093495ab5"
dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "security-framework"
version = "3.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b7f4bc775c73d9a02cde8bf7b2ec4c9d12743edf609006c7facc23998404cd1d"
dependencies = [
"bitflags",
"core-foundation",
"core-foundation-sys",
"libc",
"security-framework-sys",
]
[[package]]
name = "security-framework-sys"
version = "2.17.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ce2691df843ecc5d231c0b14ece2acc3efb62c0a398c7e1d875f3983ce020e3"
dependencies = [
"core-foundation-sys",
"libc",
]
[[package]]
name = "semver"
version = "1.0.28"
@@ -772,6 +1050,18 @@ dependencies = [
"digest",
]
[[package]]
name = "shlex"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
[[package]]
name = "simd-adler32"
version = "0.3.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a219298ac11a56ea9a6d2120044824d6f01aeb034955e7af7bc16858527deea"
[[package]]
name = "siphasher"
version = "1.0.4"
@@ -790,6 +1080,12 @@ version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f"
[[package]]
name = "subtle"
version = "2.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"
[[package]]
name = "syn"
version = "3.0.10"
@@ -808,10 +1104,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom",
"getrandom 0.4.3",
"once_cell",
"rustix",
"windows-sys",
"windows-sys 0.61.2",
]
[[package]]
@@ -834,6 +1130,25 @@ dependencies = [
"syn",
]
[[package]]
name = "time"
version = "0.3.55"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cdb87b95ec50ddfa440816d227a17b2ccbdda963a316a727fda0fc4334f7d134"
dependencies = [
"deranged",
"num-conv",
"powerfmt",
"serde_core",
"time-core",
]
[[package]]
name = "time-core"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9e1c906769ad99c88eaa54e728060edef082f8e358ff32030cb7c7d315e81109"
[[package]]
name = "tinyvec"
version = "1.13.3"
@@ -873,6 +1188,12 @@ version = "0.2.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "673aac59facbab8a9007c7f6108d11f63b603f7cabff99fabf650fea5c32b861"
[[package]]
name = "untrusted"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ecb6da28b8a351d773b68d5825ac39017e680750f980f3a1a85cd8dd28a47c1"
[[package]]
name = "utf8parse"
version = "0.2.2"
@@ -895,13 +1216,19 @@ dependencies = [
"winapi-util",
]
[[package]]
name = "wasi"
version = "0.11.1+wasi-snapshot-preview1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b"
[[package]]
name = "winapi-util"
version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"
dependencies = [
"windows-sys",
"windows-sys 0.61.2",
]
[[package]]
@@ -910,6 +1237,15 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-sys"
version = "0.52.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "282be5f36a8ce781fad8c8ae18fa3f9beff57ec1b52cb3de0789201425d9a33d"
dependencies = [
"windows-targets",
]
[[package]]
name = "windows-sys"
version = "0.61.2"
@@ -919,6 +1255,91 @@ dependencies = [
"windows-link",
]
[[package]]
name = "windows-targets"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973"
dependencies = [
"windows_aarch64_gnullvm",
"windows_aarch64_msvc",
"windows_i686_gnu",
"windows_i686_gnullvm",
"windows_i686_msvc",
"windows_x86_64_gnu",
"windows_x86_64_gnullvm",
"windows_x86_64_msvc",
]
[[package]]
name = "windows_aarch64_gnullvm"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3"
[[package]]
name = "windows_aarch64_msvc"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469"
[[package]]
name = "windows_i686_gnu"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b"
[[package]]
name = "windows_i686_gnullvm"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66"
[[package]]
name = "windows_i686_msvc"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66"
[[package]]
name = "windows_x86_64_gnu"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78"
[[package]]
name = "windows_x86_64_gnullvm"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d"
[[package]]
name = "windows_x86_64_msvc"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec"
[[package]]
name = "yasna"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e17bb3549cc1321ae1296b9cdc2698e2b6cb1992adfa19a8c72e5b7a738f44cd"
dependencies = [
"time",
]
[[package]]
name = "zeroize"
version = "1.9.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85ccff8a4805aba1eaa4cd4afcd49f7557dd2cf5fd79ffc37f8959466effc5aa"
[[package]]
name = "zlib-rs"
version = "0.6.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b268e58e7c693d7c271f93ffc4ba3b380412554231c85bf61ca7af91042a4112"
[[package]]
name = "zmij"
version = "1.0.23"
+7
View File
@@ -53,6 +53,13 @@ semver = "1"
hex = "0.4"
libloading = "0.8"
tempfile = "3"
flate2 = "1"
getrandom = "0.4"
rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12"] }
rustls-native-certs = "0.8"
rustls-pemfile = "2"
rcgen = "0.13"
regex = "1"
thiserror = "2"
clap = { version = "4", features = ["derive"] }
walkdir = "2"
+96 -44
View File
@@ -1,36 +1,49 @@
# Marqdo CLI
Marqdo is a **Markdown-native, independently interpreted programming language**:
standard CommonMark/GFM structures carry program control (headings scope,
`**strong**` executes, `*emph*` returns, ordered lists branch, bullet lists
loop, block quotes call), prose and executable nodes mix in one document, and
all side effects go through a capability-checked host boundary. The official
implementation is a Rust workspace; Python/Node are optional foreign kernels,
never the compiler target.
Marqdo is a **Markdown-native, independently interpreted programming language**.
A `.mq.md` file is ordinary CommonMark/GFM prose plus explicit executable
regions: a heading declares `@exec` or `@knowledge`, `**strong**`, `*emph*` and
inline `` `code` `` are interchangeable carriers of one statement, and keywords
inside that payload — `if`, `else`, `while`, `end`, `return`, `params` — decide
control flow. Lists, block quotes and emphasis are document presentation only.
The official implementation is a Rust workspace; Python/Node are optional
foreign kernels, never the compiler target.
This repository is the new, independent Marqdo toolchain (M0.5). It is **not**
a compatibility rewrite of `cflmy/marqdo`; the old syntax, package format and
plugin ABI v2 are not claimed to work here.
> **Status: stabilization candidate, not released.** The language, lock v3,
> Analysis v2, Trace v2, Tool v1, Plugin ABI v1 (native + worker) and pinned
> Foreign providers are implemented and covered by tests on Linux. The
> macOS/Windows `E0433` platform-cfg failure and the Linux strict
> probe→launch race are fixed in source at `8af7262`, but the native
> Windows/macOS and GitHub Linux lanes have not been re-run from this
> checkout. **`strict` OS isolation is available on Linux only when the
> runtime Bubblewrap probe passes**; it was verified against the full
> escape/resource matrix in a disposable Ubuntu 26.04 VM at `8af7262`
> (ADR-0021). macOS/Windows fail closed, strict L2 (Worker/Foreign) is not
> implemented, and the broker/worker are not a sandbox. The release decision
> is **DO NOT RELEASE** until the remote gates have same-SHA URLs. See
> **Breaking language revision landed.** The single semantic contract is
> [RFC-LANG-0002](docs/architecture/RFC-LANG-0002-unified-semantics.md), with
> normative summaries in [`spec/mapping.md`](spec/mapping.md) and
> [`spec/expression.ebnf`](spec/expression.ebnf). The frontend, expression
> parser, stdlib, examples and test corpus all implement it:
> `cargo test --workspace --locked` is green (507 passed, 0 failed, 2 ignored
> at `9c1723d`). Old syntax (`**strong**` = execute, `*emph*` = return,
> `1.` = branch, `-` = loop, `>` = call) is **not** an active language and
> `marqdo_semantics`/`--legacy` selectors do not exist. See
> [`docs/STATUS.md`](docs/STATUS.md) and
> [`docs/migration/README.md`](docs/migration/README.md).
> **Status: stabilization candidate, not released.** The language (RFC-LANG-0002),
> lock v3, Analysis v2, Trace v2, Tool v1, Plugin ABI v1 (native + worker) and
> pinned Foreign providers are implemented and covered by tests on Linux. The macOS/Windows `E0433` platform-cfg failure and the Linux
> strict probe→launch race are fixed in source at `8af7262`, but the native
> Windows/macOS and GitHub Linux lanes have not been re-run from this checkout.
> **`strict` OS isolation is available on Linux only when the runtime
> Bubblewrap probe passes**; it was verified against the full escape/resource
> matrix in a disposable Ubuntu 26.04 VM at `8af7262` (ADR-0021).
> macOS/Windows fail closed, strict L2 (Worker/Foreign) is not implemented, and
> the broker/worker are not a sandbox. The release decision is **DO NOT
> RELEASE** until the remote gates have same-SHA URLs. See
> [`docs/STATUS.md`](docs/STATUS.md), [`COMPATIBILITY.md`](COMPATIBILITY.md)
> and [`docs/security-gate.md`](docs/security-gate.md). The single current CI
> convergence plan is
> [`docs/NEXT_RELEASE_CI_CONVERGENCE.md`](docs/NEXT_RELEASE_CI_CONVERGENCE.md).
> [`docs/NEXT_RELEASE_CI_CONVERGENCE.md`](docs/NEXT_RELEASE_CI_CONVERGENCE.md)
> (historical baseline; refresh before use).
CI: the private remote [git.cflmy.cn/cflmy/marqdo-cli](https://git.cflmy.cn/cflmy/marqdo-cli)
uses Woodpecker at [ci.cflmy.cn](https://ci.cflmy.cn); [`.woodpecker.yml`](.woodpecker.yml)
uses Woodpecker at [ci.cflmy.cn](https://ci.cflmy.cn); [`.woodpecker/main.yml`](.woodpecker/main.yml)
runs fmt, clippy, workspace tests and a release build. The strict OS-isolation
lane still requires the disposable VM described in [docs/ci.md](docs/ci.md).
@@ -49,7 +62,7 @@ cargo build -p marqdo-cli
marqdo init demo
cd demo
marqdo check index.mq.md
marqdo run index.mq.md # executes ## main
marqdo run index.mq.md # executes ## @exec main
marqdo inspect index.mq.md --ast
marqdo inspect index.mq.md --signature
marqdo call index.mq.md main
@@ -58,19 +71,22 @@ marqdo call index.mq.md main
Example project from scratch (`index.mq.md`):
````markdown
# 数学
# @knowledge 数学
## 阶乘
## @exec 阶乘
1. `n <= 1`
`params(n: Int)`
*1*
`if n <= 1`
2. `n > 1`
`return 1`
**结果 = n * 阶乘(n=n-1)**
`else`
*结果*
`结果 = n * 阶乘(n=n-1)`
`return 结果`
`end`
````
```bash
@@ -81,14 +97,40 @@ marqdo call examples/factorial.mq.md 数学.阶乘 --arg n=5 # 120
The parser is frozen: `marqdo.md.v1` = CommonMark 0.31.2 + GFM tables,
strikethrough, task lists + `$...$`/`$$...$$` math (comrak `=1.0.0`). A `.mq.md`
file still renders normally in any compatible Markdown reader; rendering never
executes anything.
executes anything. The Markdown profile is unchanged by the semantic rewrite.
## Language in 60 seconds
* **Headings declare the mode.** `# @knowledge 笔记` is a document namespace;
`## @exec main` overrides to an executable function. Any H1–H6 may carry
`@exec`/`@knowledge`; unmarked headings inherit the nearest ancestor. The
document root is Knowledge. An Exec H2 is the only callable function; H3–H6
are sections inside it.
* **Closed carriers are interchangeable.** In an Exec section, `**x = 1**`,
`*x = 1*` and `` `x = 1` `` are the same assignment. Prefer inline code spans
for expressions containing `*` or `**`.
* **Statements are explicit.** `if <Bool>`, `else`, `while <Bool>`, `end`,
`return <expr>`, `params(n: Int)`, assignment, or an expression call. Each
carrier holds exactly one statement; every `if`/`while` is closed by `end`
inside the same heading body.
* **Containers do not compute.** `1.`, `-`, `>` and emphasis only organize the
document. `> print(1)` is prose; `> **print(1)**` contains a call.
* **Knowledge is inert.** Nothing in a `@knowledge` section executes, so it is
the place for normal emphasis, explanatory lists, tables and formula
resources.
* **No syntax switch.** There is no `marqdo_semantics` field or `--legacy`
flag; all `.mq.md` files use this syntax.
The full contract is [`spec/mapping.md`](spec/mapping.md) and
[`spec/expression.ebnf`](spec/expression.ebnf); the rationale and migration
rules are [`RFC-LANG-0002`](docs/architecture/RFC-LANG-0002-unified-semantics.md).
## What works (implemented and tested)
* **P0 contracts** — frozen Markdown profile, AST → program mapping, strict
expression grammar, evaluation semantics; 160 AST goldens, 73 semantic
goldens, 52 compile-error fixtures and 21 runtime-error fixtures
(`tests/`).
* **P0 contracts** — frozen `marqdo.md.v1` profile, RFC-LANG-0002 AST → program
mapping, the v2 statement grammar and evaluation semantics; 160 AST goldens,
73 semantic fixtures, 49 compile-error fixtures, 2 warning fixtures and 21
runtime-error fixtures, all green (`tests/`).
* **P1 interpreter** — native Rust tree-walk interpreter: values, scopes,
functions, static parameter inference, assignments, branches, loops,
returns, recursion, structured errors, fuel/stack/output budgets.
@@ -192,8 +234,9 @@ cargo clippy --workspace --all-targets -- -D warnings
`capabilities` are requests, never grants.
* **Reference validation is opt-in**: `check --refs` reports broken links;
plain `check` ignores them because links are static knowledge references.
Link-form calls cannot cross modules; cross-module calls use imported alias
expressions such as `geometry.圆面积(r=2)`.
Markdown links never call; calls are expressions inside executable payloads,
and cross-module calls use imported alias expressions such as
`geometry.圆面积(r=2)`.
* **`ref("#anchor")` is module-local** and errors with `ref.ambiguous` when a
heading range contains more than one formula/code block instead of picking
the first.
@@ -235,18 +278,26 @@ tests/ conformance/negative corpus (see tests/README.md)
## Documentation index
* [`AGENTS.md`](AGENTS.md) — operating rules for contributors and coding agents.
* [`docs/architecture/RFC-LANG-0002-unified-semantics.md`](docs/architecture/RFC-LANG-0002-unified-semantics.md)
— **the single language authority**: full new syntax, breaking changes,
architecture, migration and acceptance gates. Adopted by
[ADR-0052](docs/adr/0052-unified-markdown-semantics.md).
* [`spec/mapping.md`](spec/mapping.md) — normative AST→program mapping
(heading modes, unified carriers, control stack, diagnostics).
* [`spec/expression.ebnf`](spec/expression.ebnf) — normative statement and
expression grammar.
* [`docs/migration/README.md`](docs/migration/README.md) — migration rules and
the inventory of remaining work.
* [`docs/DEVELOPMENT_BLUEPRINT.md`](docs/DEVELOPMENT_BLUEPRINT.md) — architecture
baseline (Chinese; the M0.6 plan below supersedes it on command-surface scope,
extension registration, plugin addressing, package identity, repository
decoupling and release gates).
baseline (Chinese); historical sections are marked and the current language
contract is RFC-LANG-0002, not the blueprint's old mapping.
* [`docs/STABLE_RELEASE_IMPLEMENTATION.md`](docs/STABLE_RELEASE_IMPLEMENTATION.md)
— **the single authoritative next-phase plan**: PR order, G-A/G-B/G-C gates,
Windows/macOS fixes, protocol/environment stabilization, strict isolation
requirements and the release checklist.
— **the single authoritative next-phase plan** for release gates, PR order,
Windows/macOS fixes and the release checklist; it is a plan, not evidence.
* [`COMPATIBILITY.md`](COMPATIBILITY.md) — protocol versions, per-platform
feature evidence and explicit unsupported features.
feature evidence, the breaking language change and unsupported features.
* [`CHANGELOG.md`](CHANGELOG.md) — unreleased stabilization changes and the
`0.1.0` baseline.
`0.1.0` baseline, including the syntax breaking change.
* [`docs/security-gate.md`](docs/security-gate.md) — G-C evidence and limits;
[`docs/security-vm.md`](docs/security-vm.md) reproduces the disposable VM and
the self-hosted security runner.
@@ -281,6 +332,7 @@ tests/ conformance/negative corpus (see tests/README.md)
## Examples
```bash
marqdo run examples/hello.mq.md
marqdo call examples/factorial.mq.md 数学.阶乘 --arg n=5
+2 -28
View File
@@ -8,9 +8,7 @@
use std::path::Path;
use marqdo_expression::{BinaryOp, Expr, UnaryOp};
use marqdo_ir::{
CallTarget, Function, Module, ModuleId, PlaceTarget, Resource, SignatureKind, Stmt, UnitKind,
};
use marqdo_ir::{Function, Module, ModuleId, PlaceTarget, Resource, SignatureKind, Stmt, UnitKind};
use marqdo_span::Span;
use serde_json::{json, Value};
use sha2::{Digest, Sha256};
@@ -199,6 +197,7 @@ fn units_json(module: &Module) -> Vec<Value> {
UnitKind::Root => "root",
UnitKind::Namespace => "namespace",
UnitKind::Function => "function",
UnitKind::Knowledge => "knowledge",
UnitKind::Section => "section",
};
json!({
@@ -272,12 +271,6 @@ fn stmt_json(stmt: &Stmt) -> Value {
"value": expr_json(expr),
"span": span_json(*span),
}),
Stmt::Call { call, span } => json!({
"kind": "call_site",
"target": call_target_json(&call.target),
"args": call.args.iter().map(arg_json).collect::<Vec<_>>(),
"span": span_json(*span),
}),
Stmt::Branch { arms, span } => json!({
"kind": "branch",
"arms": arms.iter().map(|arm| json!({
@@ -377,19 +370,6 @@ fn arg_json(arg: &marqdo_expression::Arg) -> Value {
})
}
fn call_target_json(target: &CallTarget) -> Value {
match target {
CallTarget::Function { name } => json!({"kind":"function","name":name}),
CallTarget::Qualified {
module_alias,
function,
} => json!({"kind":"qualified","module_alias":module_alias,"function":function}),
CallTarget::Resolved { module, function } => {
json!({"kind":"resolved","module":module.to_string(),"function":function})
}
}
}
fn reference_edges(project: &LoadedProject, module_id: &marqdo_ir::ModuleId) -> Vec<Value> {
let Some(file) = project.files.get(module_id) else {
return Vec::new();
@@ -531,12 +511,6 @@ fn collect_calls(
}
Stmt::Eval { expr: value, .. } => expr_calls(project, module, function, value, calls),
Stmt::Return { value, .. } => expr_calls(project, module, function, value, calls),
Stmt::Call { call, span } => calls.push(json!({
"function": function.id.to_string(),
"span": span_json(*span),
"callee": call_target_json(&call.target),
"state": "resolved",
})),
Stmt::Branch { arms, .. } => {
for arm in arms {
expr_calls(project, module, function, &arm.condition, calls);
+247 -1024
View File
File diff suppressed because it is too large. Load diff
+68
View File
@@ -0,0 +1,68 @@
//! `marqdo check` command.
use super::*;
pub fn check(
file: Option<&Path>,
json: bool,
refs: bool,
plugins: &[PathBuf],
emit: Option<&Path>,
) -> i32 {
let mut project = match load_project(file, json) {
Ok(project) => project,
Err(code) => return code,
};
if let Err(error) = add_static_plugin_descriptors(&mut project, plugins) {
if json {
output::print_json(&json!({"success": false, "error": error}));
} else {
eprintln!("error: {error}");
}
return 2;
}
if refs {
let ref_diagnostics = validate_refs(&project);
project.diagnostics.extend(ref_diagnostics);
}
let errors = output::error_count(&project.diagnostics);
if let Some(path) = emit {
if errors > 0 {
let _ = std::fs::remove_file(path);
} else if let Err(error) = crate::analysis::write(&project, path) {
eprintln!("error: {error}");
return 2;
}
}
if json {
output::print_json(&json!({
"command": "check",
"success": errors == 0,
"errors": errors,
"diagnostics": output::diagnostics_json(&project.diagnostics),
}));
} else {
let text = output::render_text(&project.diagnostics);
eprint!("{text}");
if errors == 0 {
let checked = project
.program
.modules
.keys()
.filter(|id| !id.instance.starts_with("std:"))
.count();
println!("checked {checked} module(s): ok");
} else {
eprintln!(
"check failed: {} error(s), {} warning(s)",
errors,
project.diagnostics.len() - errors
);
}
}
if errors == 0 {
0
} else {
1
}
}
+46
View File
@@ -0,0 +1,46 @@
//! `marqdo init` command.
use super::*;
/// `marqdo init`.
pub fn init(dir: Option<&Path>) -> i32 {
let dir = dir.unwrap_or_else(|| Path::new("."));
if dir.join(marqdo_package::MANIFEST_FILE).exists() {
eprintln!(
"error: `{}` already contains {}",
dir.display(),
marqdo_package::MANIFEST_FILE
);
return 1;
}
let name = dir
.canonicalize()
.ok()
.and_then(|path| {
path.file_name()
.map(|name| name.to_string_lossy().to_string())
})
.unwrap_or_else(|| "example".to_string());
if let Err(error) = std::fs::create_dir_all(dir) {
eprintln!("error: cannot create {}: {error}", dir.display());
return 1;
}
let manifest = marqdo_package::Manifest::template(&name);
if let Err(error) = std::fs::write(dir.join(marqdo_package::MANIFEST_FILE), manifest.to_yaml())
{
eprintln!("error: cannot write manifest: {error}");
return 1;
}
let entry = dir.join("index.mq.md");
if let Err(error) = std::fs::write(&entry, INIT_TEMPLATE) {
eprintln!("error: cannot write {}: {error}", entry.display());
return 1;
}
let _ = std::fs::write(dir.join(".gitignore"), ".marqdo/\n");
println!("Initialized Marqdo project `{name}` in {}", dir.display());
println!("Next: marqdo check index.mq.md && marqdo run index.mq.md");
0
}
const INIT_TEMPLATE: &str =
"## @exec main\n\n`问候 = \"你好,Marqdo\"`\n\n`打印(问候)`\n\n`return 问候`\n";
+132
View File
@@ -0,0 +1,132 @@
//! `marqdo inspect` command.
use super::*;
/// Options for `marqdo inspect`.
#[derive(Debug, Clone, Default)]
pub struct InspectFlags {
/// Show the Markdown AST.
pub ast: bool,
/// Show heading units.
pub units: bool,
/// Show inferred/explicit signatures.
pub signature: bool,
/// Show executable IR.
pub ir: bool,
/// Show link reference edges.
pub refs: bool,
/// Show platform capability report.
pub platform: bool,
}
pub fn inspect(file: Option<&Path>, flags: &InspectFlags, json: bool) -> i32 {
let project = match load_project(file, json) {
Ok(project) => project,
Err(code) => return code,
};
let entry = project.entry_file();
let module = &entry.analysis.module;
let selected_any =
flags.ast || flags.units || flags.signature || flags.ir || flags.refs || flags.platform;
if json {
let mut object = serde_json::Map::new();
object.insert("command".into(), json!("inspect"));
object.insert("module".into(), json!(entry.module_id.to_string()));
if !selected_any {
object.insert("summary".into(), summary_json(&project));
}
if flags.ast {
object.insert(
"ast".into(),
serde_json::to_value(&entry.analysis.document.root).unwrap_or_default(),
);
}
if flags.units {
object.insert(
"units".into(),
serde_json::to_value(&entry.analysis.units.units).unwrap_or_default(),
);
}
if flags.signature {
object.insert("signatures".into(), signatures_json(module));
}
if flags.ir {
object.insert(
"ir".into(),
serde_json::to_value(&module.functions).unwrap_or_default(),
);
}
if flags.refs {
object.insert("refs".into(), json!(refs_json(&project)));
}
if flags.platform {
object.insert(
"platform".into(),
serde_json::to_value(PlatformCapabilities::detect()).unwrap_or_default(),
);
}
output::print_json(&serde_json::Value::Object(object));
return 0;
}
if !selected_any {
println!("module: {}", entry.module_id);
println!("path: {}", entry.path.display());
println!("project root: {}", project.root.display());
println!("functions:");
for function in &module.functions {
let params = function
.params
.iter()
.map(|param| param.name.as_str())
.collect::<Vec<_>>()
.join(", ");
let kind = match function.signature {
marqdo_ir::SignatureKind::Explicit => "explicit",
marqdo_ir::SignatureKind::Inferred => "inferred",
};
println!(" {} ({params}) [{kind}]", function.id.name);
}
println!(
"resources: {} formula/code block(s)",
module.resources.len()
);
return 0;
}
if flags.ast {
println!(
"{}",
serde_json::to_string_pretty(&entry.analysis.document.root).unwrap_or_default()
);
}
if flags.units {
println!(
"{}",
serde_json::to_string_pretty(&entry.analysis.units.units).unwrap_or_default()
);
}
if flags.signature {
println!(
"{}",
serde_json::to_string_pretty(&signatures_json(module)).unwrap_or_default()
);
}
if flags.ir {
println!(
"{}",
serde_json::to_string_pretty(&module.functions).unwrap_or_default()
);
}
if flags.refs {
println!(
"{}",
serde_json::to_string_pretty(&refs_json(&project)).unwrap_or_default()
);
}
if flags.platform {
println!(
"{}",
serde_json::to_string_pretty(&PlatformCapabilities::detect()).unwrap_or_default()
);
}
0
}
+252
View File
@@ -0,0 +1,252 @@
//! Package lock/sync/add/remove commands.
use super::*;
/// `marqdo lock`.
pub fn lock(json: bool) -> i32 {
let root = match discover_project_root() {
Ok(root) => root,
Err(error) => {
eprintln!("error: {error}");
return 2;
}
};
let manifest = match marqdo_package::Manifest::load(&root.join(marqdo_package::MANIFEST_FILE)) {
Ok(manifest) => manifest,
Err(error) => {
eprintln!("error: {error}");
return 2;
}
};
let graph =
match marqdo_package::PackageResolver::new(root.clone(), Some(manifest.clone()), None)
.with_fetch(true)
.with_unlocked(true)
.resolve_lock_graph()
{
Ok(graph) => graph,
Err(error) => {
eprintln!("error: {error}");
return 1;
}
};
let lockfile = graph.lock;
let path = root.join(marqdo_package::LOCK_FILE);
if let Err(error) = write_atomic(&path, lockfile.to_yaml().as_bytes()) {
eprintln!("error: cannot write {}: {error}", path.display());
return 1;
}
if json {
output::print_json(&json!({
"command": "lock",
"success": true,
"packages": lockfile.packages,
}));
} else {
println!(
"locked {} package(s) in {}",
lockfile.packages.len(),
path.display()
);
}
0
}
/// `marqdo sync [--locked]`.
pub fn sync(locked: bool, json: bool) -> i32 {
let root = match discover_project_root() {
Ok(root) => root,
Err(error) => {
eprintln!("error: {error}");
return 2;
}
};
let manifest = match marqdo_package::Manifest::load(&root.join(marqdo_package::MANIFEST_FILE)) {
Ok(manifest) => manifest,
Err(error) => {
eprintln!("error: {error}");
return 2;
}
};
let lock_path = root.join(marqdo_package::LOCK_FILE);
if !lock_path.is_file() {
eprintln!(
"error: {} is missing; run `marqdo lock` first",
lock_path.display()
);
return 1;
}
let lockfile = match marqdo_package::LockFile::load(&lock_path) {
Ok(lockfile) => lockfile,
Err(error) => {
eprintln!("error: {error}");
return 1;
}
};
if let Err(error) = lockfile.validate_root(&manifest) {
eprintln!("error: {error}");
return 1;
}
let resolver =
marqdo_package::PackageResolver::new(root, Some(manifest.clone()), Some(lockfile.clone()))
.with_fetch(!locked);
if let Err(error) = resolver.materialize_env(&lockfile) {
eprintln!("error: cannot materialize project environment: {error}");
return 1;
}
if json {
output::print_json(&json!({
"command": "sync",
"success": true,
"locked": locked,
"packages": lockfile.packages,
}));
} else {
println!(
"synchronized {} package(s){} into {}",
lockfile.packages.len(),
if locked { " (locked)" } else { "" },
resolver.env_root().display()
);
}
0
}
fn discover_project_root() -> Result<PathBuf, String> {
let cwd = std::env::current_dir().map_err(|error| error.to_string())?;
marqdo_package::discover_root(&cwd).ok_or_else(|| {
format!(
"no {} found in {} or any parent",
marqdo_package::MANIFEST_FILE,
cwd.display()
)
})
}
fn write_atomic(path: &Path, contents: &[u8]) -> Result<(), String> {
let file_name = path
.file_name()
.map(|name| name.to_string_lossy().to_string())
.unwrap_or_else(|| "marqdo.lock".to_string());
let temp = path.with_file_name(format!(
".{file_name}.tmp-{}-{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|duration| duration.as_nanos())
.unwrap_or(0)
));
let result = (|| -> Result<(), String> {
let mut file = std::fs::File::create(&temp)
.map_err(|error| format!("cannot create {}: {error}", temp.display()))?;
std::io::Write::write_all(&mut file, contents)
.map_err(|error| format!("cannot write {}: {error}", temp.display()))?;
file.sync_all()
.map_err(|error| format!("cannot sync {}: {error}", temp.display()))?;
std::fs::rename(&temp, path)
.map_err(|error| format!("cannot publish {}: {error}", path.display()))?;
Ok(())
})();
if result.is_err() {
let _ = std::fs::remove_file(&temp);
}
result
}
/// `marqdo add <name> --path <dir> [--git URL --rev SHA]`.
pub fn add(
name: &str,
path: Option<&Path>,
git: Option<&str>,
rev: Option<&str>,
json: bool,
) -> i32 {
let root = match discover_project_root() {
Ok(root) => root,
Err(error) => {
eprintln!("error: {error}");
return 2;
}
};
let manifest_path = root.join(marqdo_package::MANIFEST_FILE);
let mut manifest = match marqdo_package::Manifest::load(&manifest_path) {
Ok(manifest) => manifest,
Err(error) => {
eprintln!("error: {error}");
return 2;
}
};
let dependency = marqdo_package::Dependency {
path: path.map(|path| path.to_path_buf()),
git: git.map(str::to_string),
rev: rev.map(str::to_string),
};
if let Err(error) = dependency.validate(name) {
eprintln!("error: {error}");
return 2;
}
if manifest.dependencies.contains_key(name) {
eprintln!("error: dependency `{name}` already exists; remove it first");
return 1;
}
manifest.dependencies.insert(name.to_string(), dependency);
if let Err(error) = std::fs::write(&manifest_path, manifest.to_yaml()) {
eprintln!("error: cannot write {}: {error}", manifest_path.display());
return 1;
}
if json {
output::print_json(&json!({
"command": "add",
"success": true,
"dependency": name,
"manifest": manifest_path.display().to_string(),
}));
} else {
println!("added dependency `{name}` to {}", manifest_path.display());
println!(
"run `marqdo lock` to pin it in {}",
marqdo_package::LOCK_FILE
);
}
0
}
/// `marqdo remove <name>`.
pub fn remove(name: &str, json: bool) -> i32 {
let root = match discover_project_root() {
Ok(root) => root,
Err(error) => {
eprintln!("error: {error}");
return 2;
}
};
let manifest_path = root.join(marqdo_package::MANIFEST_FILE);
let mut manifest = match marqdo_package::Manifest::load(&manifest_path) {
Ok(manifest) => manifest,
Err(error) => {
eprintln!("error: {error}");
return 2;
}
};
if manifest.dependencies.remove(name).is_none() {
eprintln!("error: dependency `{name}` is not declared");
return 1;
}
if let Err(error) = std::fs::write(&manifest_path, manifest.to_yaml()) {
eprintln!("error: cannot write {}: {error}", manifest_path.display());
return 1;
}
// The lock is a graph, not a per-name list: regenerating it here would
// require resolving the new manifest. Leave it stale and tell the user.
if json {
output::print_json(&json!({
"command": "remove",
"success": true,
"dependency": name,
}));
} else {
println!("removed dependency `{name}`");
println!("run `marqdo lock` to refresh {}", marqdo_package::LOCK_FILE);
}
0
}
+312
View File
@@ -0,0 +1,312 @@
//! `marqdo run` execution assembly.
//!
//! Child module of `commands`: it may use the parent's project loader and
//! runtime helpers, but keeps the run session out of the command table.
use super::*;
use marqdo_sandbox::StrictChildState;
pub fn execute(
file: Option<&Path>,
function: Option<&str>,
named_args: &[(String, Value)],
settings: &SandboxSettings,
json: bool,
trace: bool,
) -> i32 {
let command_name = if function.is_some() { "call" } else { "run" };
let mut project = match load_project(file, json) {
Ok(project) => project,
Err(code) => return code,
};
if let Err(error) = add_static_plugin_descriptors(&mut project, &settings.plugins) {
if json {
output::print_json(&json!({"success": false, "error": error}));
} else {
eprintln!("error: {error}");
}
return 2;
}
if project.has_errors() {
if json {
output::print_json(&json!({
"command": command_name,
"success": false,
"diagnostics": output::diagnostics_json(&project.diagnostics),
}));
} else {
eprint!("{}", output::render_text(&project.diagnostics));
eprintln!("error: refusing to run a project with compile errors");
}
return 1;
}
let warnings: Vec<FileDiagnostic> = project
.diagnostics
.iter()
.filter(|item| !item.diagnostic.is_error())
.cloned()
.collect();
if !json && !warnings.is_empty() {
eprint!("{}", output::render_text(&warnings));
}
let mode = match settings.sandbox_mode() {
Ok(mode) => mode,
Err(error) => {
emit_error(json, command_name, "cli.invalid_mode", &error);
return 2;
}
};
if mode == SandboxMode::Strict {
if let Err(error) = settings.strict_preflight() {
emit_error(json, command_name, "strict.unsupported_grant", &error);
return 2;
}
match marqdo_sandbox::inspect_strict_child() {
// A verified supervisor child may run project code.
Ok(StrictChildState::VerifiedChild) => {}
// Not a child: enforce capability and launch under the supervisor.
Ok(StrictChildState::NotChild) => {
let platform = PlatformCapabilities::detect();
if let Err(error) = marqdo_sandbox::enforce(mode, &platform) {
emit_error(json, command_name, error.code(), &error.to_string());
return 1;
}
return launch_strict_process(&project.root, file, settings, json, command_name);
}
// A marker without a verified identity is never trusted, and the
// platform-neutral inspection keeps Linux-only symbols out of this
// module so macOS/Windows builds compile.
Err(reason) => {
let message = format!("strict isolation verification failed: {reason}");
emit_error(json, command_name, "strict.identity_invalid", &message);
return 1;
}
}
}
let platform = PlatformCapabilities::detect();
if let Err(error) = marqdo_sandbox::enforce(mode, &platform) {
emit_error(
json,
command_name,
"sandbox.unavailable",
&error.to_string(),
);
return 1;
}
let capabilities = match settings.capabilities() {
Ok(capabilities) => capabilities,
Err(error) => {
emit_error(json, command_name, "cli.invalid_grants", &error);
return 2;
}
};
let stdin = read_stdin();
let mut host = StdHost::new(mode, capabilities, project.root.clone()).with_stdin(stdin);
if trace || json || settings.trace_file.is_some() {
host = host
.with_trace()
.with_trace_budget(marqdo_host::TraceBudget {
max_events: settings
.trace_max_events
.unwrap_or(marqdo_host::trace::DEFAULT_MAX_TRACE_EVENTS),
max_bytes: settings
.trace_max_bytes
.unwrap_or(marqdo_host::trace::DEFAULT_MAX_TRACE_BYTES),
});
}
if let Some(fuel) = settings.fuel {
host.budget.fuel = fuel;
}
if let Some(stack) = settings.stack {
host.budget.stack_limit = stack;
}
if let Some(max_output) = settings.max_output {
host.budget.max_output_bytes = max_output;
}
if let Some(timeout) = settings.timeout {
host.budget.deadline = Some(Instant::now() + Duration::from_secs(timeout));
}
for kernel_path in &settings.kernels {
if let Err(error) = load_kernel_descriptor(&mut host, kernel_path) {
if json {
output::print_json(&json!({"success": false, "error": error}));
} else {
eprintln!("error: {error}");
}
return 2;
}
}
for plugin_path in &settings.plugins {
if let Err(message) =
load_plugin_into_host(&mut host, plugin_path, settings.trust_native_plugin)
{
if json {
output::print_json(&json!({"success": false, "error": message}));
} else {
eprintln!("error: {message}");
}
return 1;
}
}
// Expose the complete export names loaded from trusted manifests so the
// runtime resolves exactly those names instead of probing namespaces.
for (name, _signature, digest, package) in host.plugin_exports() {
if project
.program
.external_callables
.iter()
.any(|callable| callable.public_name == name)
{
continue;
}
project
.program
.external_callables
.push(marqdo_ir::callable::CallableDescriptor {
id: marqdo_ir::callable::CallableId(format!("plugin:{package}:{name}")),
public_name: name.clone(),
owner_package: package.clone(),
signature: marqdo_ir::callable::CallableSignature {
params: Vec::new(),
kind: marqdo_ir::SignatureKind::Explicit,
},
effects_requested: vec!["host".to_string()],
origin: marqdo_ir::callable::CallableOrigin::AbiPlugin {
package,
export: name,
},
implementation_digest: Some(digest),
source_span: None,
});
}
let function_name = function.unwrap_or("main").to_string();
let entry_module = &project.program.modules[&project.entry];
if entry_module.function(&function_name).is_none() {
let message = if function.is_none() {
format!(
"{} has no callable `main`; declare an Exec H2 (`## @exec main`) or pass an \
explicit function name",
project.entry_file().path.display()
)
} else {
format!(
"no function named `{function_name}` in {}",
project.entry_file().path.display()
)
};
if json {
emit_error(true, command_name, "sem.no_entry_function", &message);
} else {
eprintln!("error[sem.no_entry_function]: {message}");
}
return 1;
}
let runtime_args: Vec<EvaluatedArg> = named_args
.iter()
.map(|(name, value)| EvaluatedArg::named(name.clone(), value.clone(), Span::default()))
.collect();
let memory_limits = MemoryLimits {
max_total_bytes: settings
.max_memory
.unwrap_or(marqdo_runtime::DEFAULT_MAX_TOTAL_BYTES),
max_region_bytes: settings
.max_region_memory
.unwrap_or(marqdo_runtime::DEFAULT_MAX_REGION_BYTES),
};
let result = run_with_limits(
&project.program,
&mut host,
&project.entry,
&function_name,
runtime_args,
memory_limits,
);
if let Some(path) = &settings.trace_file {
let status = match &result {
Ok(_) => "ok",
Err(error) => error.code(),
};
let entry = format!("{}::{}", project.entry, function_name);
if let Err(error) = crate::trace::write(&project, &host, &entry, status, path) {
emit_error(json, command_name, "cli.trace_failed", &error);
return 1;
}
}
let stdout = host.take_output();
let stdout_text = String::from_utf8_lossy(&stdout).to_string();
let stderr = host.take_error();
let stderr_text = String::from_utf8_lossy(&stderr).to_string();
let trace_entries = host.trace().to_vec();
match result {
Ok(outcome) => {
if json {
output::print_json(&json!({
"command": command_name,
"success": true,
"value": marqdo_value::to_wire(&outcome.value),
"stdout": stdout_text,
"stderr": stderr_text,
"diagnostics": output::diagnostics_json(&project.diagnostics),
"trace": trace_entries,
}));
} else {
use std::io::Write;
if !stderr.is_empty() {
let _ = std::io::stderr().write_all(&stderr);
}
let _ = std::io::stdout().write_all(&stdout);
if function.is_some() && outcome.value != Value::Unit {
println!("{}", outcome.value.display());
}
if trace {
for entry in &trace_entries {
eprintln!(
"trace: {} {} {}us {}",
entry.op, entry.module, entry.duration_micros, entry.result
);
}
}
}
0
}
Err(error) => {
if json {
output::print_json(&json!({
"command": command_name,
"success": false,
"stdout": stdout_text,
"stderr": stderr_text,
"error": runtime_error_json(&error, &project),
"trace": trace_entries,
}));
} else {
use std::io::Write;
if !stderr.is_empty() {
let _ = std::io::stderr().write_all(&stderr);
}
let _ = std::io::stdout().write_all(&stdout);
eprint!("{}", render_runtime_error(&error, &project));
}
1
}
}
}
/// Read piped stdin once at the composition boundary.
///
/// A terminal stdin is not read, so `marqdo run` never blocks waiting for an
/// interactive EOF. Non-terminal input is a finite pipe/file and is copied into
/// the run-scoped `stdio/stdin` resource.
fn read_stdin() -> Vec<u8> {
use std::io::{IsTerminal, Read};
let mut stdin = std::io::stdin();
if stdin.is_terminal() {
return Vec::new();
}
let mut bytes = Vec::new();
if let Err(error) = stdin.read_to_end(&mut bytes) {
eprintln!("error: cannot read stdin: {error}");
return Vec::new();
}
bytes
}
+152
View File
@@ -0,0 +1,152 @@
//! Sandbox mode parsing and capability construction.
//!
//! This module owns the CLI-to-`Capabilities` mapping and strict preflight;
//! command implementations only consume the resulting grant set.
use std::path::PathBuf;
use marqdo_host::Capabilities;
use marqdo_sandbox::SandboxMode;
/// Effective sandbox settings from the command line.
#[derive(Debug, Clone, Default)]
pub struct SandboxSettings {
/// Mode name.
pub mode: String,
/// Read grants.
pub allow_read: Vec<String>,
/// Write grants.
pub allow_write: Vec<String>,
/// Network grants (IP literals, ADR-0034).
pub allow_net: Vec<String>,
/// Environment variable grants (ADR-0043).
pub allow_env: Vec<String>,
/// Process program grants (ADR-0044).
pub allow_process: Vec<String>,
/// OS entropy grant (ADR-0046).
pub allow_random: bool,
/// Foreign execution grant.
pub allow_foreign: bool,
/// Plugin grants.
pub allow_plugin: Vec<String>,
/// Fuel override.
pub fuel: Option<u64>,
/// Stack depth override.
pub stack: Option<usize>,
/// Output limit override.
pub max_output: Option<usize>,
/// Total committed addressable-memory budget override (bytes).
pub max_memory: Option<usize>,
/// Per-region committed addressable-memory budget override (bytes).
pub max_region_memory: Option<usize>,
/// Wall-clock timeout in seconds.
pub timeout: Option<u64>,
/// Explicitly trusted plugin manifests to load.
pub plugins: Vec<PathBuf>,
/// Explicit trust decision for in-process native plugin loading.
pub trust_native_plugin: bool,
/// Optional Trace v2 output path.
pub trace_file: Option<PathBuf>,
/// Trace event budget override (0 = unlimited).
pub trace_max_events: Option<usize>,
/// Trace approximate byte budget override (0 = unlimited).
pub trace_max_bytes: Option<usize>,
/// External foreign-kernel descriptor paths.
pub kernels: Vec<PathBuf>,
}
impl SandboxSettings {
/// Parse the mode string.
pub fn sandbox_mode(&self) -> Result<SandboxMode, String> {
SandboxMode::parse(&self.mode).ok_or_else(|| {
format!(
"unknown sandbox mode `{}` (expected pure, project or strict)",
self.mode
)
})
}
/// Reject strict-mode requests that this L1 isolation level does not
/// support. Strict currently covers pure Marqdo and native Math only;
/// granting Foreign, ABI plugins, tools or trace files would require
/// extending the supervisor contract first.
pub fn strict_preflight(&self) -> Result<(), String> {
let requested = self.allow_foreign
|| !self.allow_read.is_empty()
|| !self.allow_net.is_empty()
|| !self.allow_write.is_empty()
|| !self.allow_env.is_empty()
|| !self.allow_process.is_empty()
|| self.allow_random
|| !self.allow_plugin.is_empty()
|| !self.plugins.is_empty()
|| self.trust_native_plugin
|| !self.kernels.is_empty()
|| self.trace_file.is_some();
if requested {
return Err(
"strict mode currently supports pure Marqdo and native Math only; \
--allow-read/--allow-write/--allow-env/--allow-process/--allow-random/\
--allow-foreign/--allow-plugin/--plugin/--trust-native-plugin/--kernel/--trace-file are
rejected. Use --mode project for those capabilities."
.to_string(),
);
}
Ok(())
}
/// Build the effective capability set.
pub fn capabilities(&self) -> Result<Capabilities, String> {
let mode = self.sandbox_mode()?;
match mode {
SandboxMode::Pure => {
if !self.allow_read.is_empty()
|| !self.allow_net.is_empty()
|| !self.allow_write.is_empty()
|| !self.allow_env.is_empty()
|| !self.allow_process.is_empty()
|| self.allow_random
|| self.allow_foreign
|| !self.allow_plugin.is_empty()
{
return Err(
"pure mode prohibits external I/O grants; use --mode project".to_string(),
);
}
Ok(Capabilities::pure())
}
SandboxMode::Project => {
let mut capabilities = Capabilities::project();
for pattern in &self.allow_read {
capabilities = capabilities.allow_read(pattern.clone());
}
for pattern in &self.allow_write {
capabilities = capabilities.allow_write(pattern.clone());
}
for host in &self.allow_net {
capabilities = capabilities.allow_net(host.clone());
}
for name in &self.allow_env {
capabilities = capabilities.allow_env(name.clone());
}
for program in &self.allow_process {
capabilities = capabilities.allow_process(program.clone());
}
if self.allow_random {
capabilities = capabilities.allow_random();
}
if self.allow_foreign {
capabilities = capabilities.allow_foreign();
}
for plugin in &self.allow_plugin {
capabilities = capabilities.allow_plugin(plugin.clone());
}
Ok(capabilities)
}
SandboxMode::Strict => {
self.strict_preflight()?;
Ok(Capabilities::project())
}
}
}
}
+138
View File
@@ -0,0 +1,138 @@
//! CLI formatter and plugin/tool inspection commands.
use super::*;
/// `marqdo fmt`.
pub fn fmt(file: Option<&Path>, check_only: bool) -> i32 {
let file = file.unwrap_or_else(|| Path::new("index.mq.md"));
let Ok(source) = std::fs::read_to_string(file) else {
eprintln!("error: cannot read `{}`", file.display());
return 2;
};
let mut formatted = source.replace("\r\n", "\n");
while formatted.ends_with("\n\n") {
formatted.pop();
}
if !formatted.ends_with('\n') {
formatted.push('\n');
}
if formatted == source {
return 0;
}
if check_only {
eprintln!(
"error: `{}` is not formatted (trailing whitespace/newlines or CRLF)",
file.display()
);
return 1;
}
if let Err(error) = std::fs::write(file, formatted) {
eprintln!("error: cannot write `{}`: {error}", file.display());
return 1;
}
0
}
/// `marqdo plugin inspect`.
pub fn plugin_inspect(target: &Path, json: bool) -> i32 {
let manifest_path = if target.is_dir() {
target.join("plugin.yaml")
} else {
target.to_path_buf()
};
let base_dir = manifest_path
.parent()
.map(|parent| parent.to_path_buf())
.unwrap_or_else(|| PathBuf::from("."));
let manifest = match marqdo_plugin_abi::PluginManifest::load(&manifest_path) {
Ok(manifest) => manifest,
Err(error) => {
report_plugin_error(&error, json);
return 1;
}
};
if let Err(error) = manifest.validate() {
report_plugin_error(&error, json);
return 1;
}
let host_target = marqdo_plugin_abi::HostTarget::current();
let artifact = match manifest.verify_artifact(&base_dir, &host_target) {
Ok((path, artifact)) => {
let abi = match manifest.abi_requirement() {
Ok(requirement) => {
if requirement.is_compatible_with(marqdo_plugin_abi::AbiVersion::current()) {
"compatible"
} else {
"incompatible"
}
}
Err(error) => {
report_plugin_error(&error, json);
return 1;
}
};
(path, artifact, abi)
}
Err(error) => {
report_plugin_error(&error, json);
return 1;
}
};
let (artifact_path, artifact_info, abi) = artifact;
if json {
output::print_json(&json!({
"command": "plugin.inspect",
"success": true,
"plugin": manifest.plugin.name,
"version": manifest.plugin.version,
"abi": manifest.plugin.abi,
"abi_compatibility": abi,
"target": host_target.triple().as_str(),
"artifact": {
"file": artifact_info.file,
"path": artifact_path.display().to_string(),
"sha256": artifact_info.sha256,
"verified": true,
},
"exports": manifest.plugin.exports.iter().map(|export| json!({
"name": export.name,
"signature": export.signature,
})).collect::<Vec<_>>(),
"loaded": false,
}));
} else {
println!(
"plugin: {} {}",
manifest.plugin.name, manifest.plugin.version
);
println!("abi: {} ({abi})", manifest.plugin.abi);
println!("target: {}", host_target.triple().as_str());
println!(
"artifact: {} (sha256 verified: {})",
artifact_path.display(),
artifact_info.sha256
);
println!("exports:");
for export in &manifest.plugin.exports {
println!(" {} {}", export.name, export.signature);
}
println!("loaded: false (inspection never executes plugin code)");
}
0
}
fn report_plugin_error(error: &marqdo_plugin_abi::PluginError, json: bool) {
if json {
output::print_json(&json!({
"command": "plugin.inspect",
"success": false,
"error": {
"kind": error.kind,
"code": error.code,
"message": error.message,
}
}));
} else {
eprintln!("error[{}]: {}", error.code, error.message);
}
}
-1
View File
@@ -540,7 +540,6 @@ fn stmts_use_print(stmts: &[Stmt]) -> bool {
target_uses || expr_uses_print(value)
}
Stmt::Eval { expr, .. } | Stmt::Return { value: expr, .. } => expr_uses_print(expr),
Stmt::Call { call, .. } => call.args.iter().any(|arg| expr_uses_print(&arg.value)),
Stmt::Branch { arms, .. } => arms
.iter()
.any(|arm| expr_uses_print(&arm.condition) || stmts_use_print(&arm.body)),
+38 -6
View File
@@ -23,7 +23,7 @@ use clap::{Args, Parser, Subcommand};
name = "marqdo",
version,
about = "Markdown-native programming language (independent Rust interpreter)",
long_about = "Marqdo executes Markdown documents whose AST structures carry program meaning. `check` and `inspect` never execute project code; `run` and `call` execute through the capability host."
long_about = "Marqdo executes .mq.md documents: headings declare @exec/@knowledge modes, Strong/Emph/inline Code carry one statement each, and if/else/while/end/return/params decide control flow. `check` and `inspect` never execute project code; `run` and `call` execute through the capability host."
)]
struct Cli {
#[command(subcommand)]
@@ -86,12 +86,21 @@ struct SandboxArgs {
/// Grant read access to a project-relative glob (repeatable).
#[arg(long = "allow-read", value_name = "GLOB")]
allow_read: Vec<String>,
/// Grant network access to an IP literal host (repeatable, ADR-0034).
#[arg(long = "allow-net", value_name = "IP")]
/// Grant network access to a host name or IP literal (repeatable, ADR-0034/0047).
#[arg(long = "allow-net", value_name = "HOST")]
allow_net: Vec<String>,
/// Grant write access to a project-relative glob (repeatable).
#[arg(long = "allow-write", value_name = "GLOB")]
allow_write: Vec<String>,
/// Grant read access to an environment variable (repeatable, ADR-0043).
#[arg(long = "allow-env", value_name = "NAME")]
allow_env: Vec<String>,
/// Grant execution of an exact program path (repeatable, ADR-0044).
#[arg(long = "allow-process", value_name = "PROGRAM")]
allow_process: Vec<String>,
/// Grant OS entropy reads (ADR-0046).
#[arg(long = "allow-random")]
allow_random: bool,
/// Grant explicit foreign kernel execution.
#[arg(long = "allow-foreign")]
allow_foreign: bool,
@@ -143,6 +152,9 @@ impl SandboxArgs {
allow_read: self.allow_read.clone(),
allow_net: self.allow_net.clone(),
allow_write: self.allow_write.clone(),
allow_env: self.allow_env.clone(),
allow_process: self.allow_process.clone(),
allow_random: self.allow_random,
allow_foreign: self.allow_foreign,
allow_plugin: self.allow_plugin.clone(),
fuel: self.fuel,
@@ -355,8 +367,29 @@ struct SyncArgs {
}
fn main() {
// The interpreter and the bundled pure stdlib recurse on the parse/execute
// path. Windows gives the main thread 1 MiB by default (Linux/macOS use
// 8 MiB), which overflowed on the regex self-test. Run the CLI on an
// explicit 8 MiB thread on every platform so behavior does not depend on
// the host's main-thread default.
const STACK_BYTES: usize = 8 * 1024 * 1024;
let code = match std::thread::Builder::new()
.name("marqdo-main".to_string())
.stack_size(STACK_BYTES)
.spawn(dispatch)
{
Ok(handle) => handle.join().unwrap_or(101),
Err(error) => {
eprintln!("error: cannot start the marqdo main thread: {error}");
1
}
};
std::process::exit(code);
}
fn dispatch() -> i32 {
let cli = Cli::parse();
let code = match cli.command {
match cli.command {
Command::Init(args) => commands::init(args.dir.as_deref()),
Command::Check(args) => commands::check(
args.file.as_deref(),
@@ -421,6 +454,5 @@ fn main() {
Command::PluginWorker(args) => {
marqdo_plugin_abi::worker::serve(&args.manifest, &args.base_dir)
}
};
std::process::exit(code);
}
}
+1 -1
View File
@@ -258,7 +258,7 @@ mod tests {
std::fs::copy(entry.path(), dir.path().join(entry.file_name())).unwrap();
}
}
std::fs::write(dir.path().join("io.mq.md"), "## tampered\n").unwrap();
std::fs::write(dir.path().join("io.mq.md"), "## @knowledge tampered\n").unwrap();
let error = StdlibBundle::load(dir.path()).unwrap_err();
assert!(error.contains("digest mismatch"), "{error}");
}
+17 -16
View File
@@ -49,21 +49,21 @@ fn write(dir: &Path, name: &str, source: &str) -> PathBuf {
path
}
const PROGRAM: &str = r#"# 数学
const PROGRAM: &str = r#"# @knowledge 数学
## 加一
## @exec 加一
`(n)`
`params(n)`
*n + 1*
`return n + 1`
## main
## @exec main
**结果 = 加一(41)**
`结果 = 加一(41)`
> 打印("secret-value")
`打印("secret-value")`
*结果*
`return 结果`
"#;
#[test]
@@ -143,7 +143,7 @@ fn analysis_is_deterministic_and_has_stable_semantic_nodes() {
fn analysis_v2_identity_is_location_independent() {
let first = TempDir::new("analysis-identity-a");
let second = TempDir::new("analysis-identity-b");
let source = "## main\n\n**x = 1**\n\n*x*\n";
let source = "## @exec main\n\n`x = 1`\n\n`return x`\n";
for dir in [&first, &second] {
write(
dir.path(),
@@ -184,7 +184,8 @@ fn analysis_v2_identity_is_location_independent() {
#[test]
fn analysis_v2_projects_memory_place_targets_additively() {
let dir = TempDir::new("analysis-place");
let source = "## main\n\n**内存 = memory.new(1, 2)**\n\n**`内存[0] = 7`**\n\n*内存[0]*\n";
let source =
"## @exec main\n\n`内存 = memory.new(1, 2)`\n\n`内存[0] = 7`\n\n`return 内存[0]`\n";
write(dir.path(), "index.mq.md", source);
let target = dir.path().join("analysis.json");
let output = cli(
@@ -212,9 +213,9 @@ fn analysis_v2_projects_memory_place_targets_additively() {
#[test]
fn analysis_v2_projects_addressable_formula_and_code_resources() {
let dir = TempDir::new("analysis-resources");
let source = r#"## calc
let source = r#"## @exec calc
`(x)`
`params(x)`
$$A = x + 1$$
@@ -222,7 +223,7 @@ $$A = x + 1$$
print(1)
```
*x*
`return x`
"#;
write(
dir.path(),
@@ -275,7 +276,7 @@ fn trace_v2_budget_truncates_long_loops_without_failing_the_run() {
write(
dir.path(),
"index.mq.md",
"## main\n\n**i = 0**\n\n- `i < 500`\n\n **i = i + 1**\n\n*i*\n",
"## @exec main\n\n`i = 0`\n\n`while i < 500`\n\n`i = i + 1`\n\n`end`\n\n`return i`\n",
);
let trace_path = dir.path().join("trace.json");
let output = cli(
@@ -318,7 +319,7 @@ fn trace_v2_budget_truncates_long_loops_without_failing_the_run() {
#[test]
fn failed_check_removes_stale_analysis() {
let dir = TempDir::new("analysis-error");
write(dir.path(), "bad.mq.md", "## main\n\n1. 没有条件\n");
write(dir.path(), "bad.mq.md", "## @exec main\n\n`return x +`\n");
let target = dir.path().join("analysis.json");
std::fs::write(&target, "stale").unwrap();
let output = cli(
@@ -376,7 +377,7 @@ fn trace_records_calls_effects_and_status_without_secrets() {
}
// Error runs still write a terminal trace.
write(dir.path(), "bad.mq.md", "## main\n\n**x = 1 // 0**\n");
write(dir.path(), "bad.mq.md", "## @exec main\n\n`x = 1 // 0`\n");
let error_trace = dir.path().join("error-trace.json");
let output = cli(
dir.path(),
+274
View File
@@ -0,0 +1,274 @@
//! Architecture boundary gate (review RFC-CM P0/P1).
//!
//! These checks pin the documented layering: the runtime depends on the host
//! platform layer, the bundled stdlib stays in this repository, `std:` import
//! targets resolve only inside the bundle, and the host keeps the documented
//! stdio/file/TCP resource endpoints. They are source and digest checks, not an
//! OS-sandbox proof.
use std::path::{Path, PathBuf};
use sha2::{Digest, Sha256};
fn repo_root() -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR"))
.parent()
.unwrap()
.parent()
.unwrap()
.to_path_buf()
}
fn read(path: &Path) -> String {
std::fs::read_to_string(path).unwrap_or_else(|error| panic!("read {}: {error}", path.display()))
}
const WORKSPACE_CRATES: &[&str] = &[
"marqdo-span",
"marqdo-value",
"marqdo-expression",
"marqdo-markdown",
"marqdo-ir",
"marqdo-frontend",
"marqdo-math",
"marqdo-foreign",
"marqdo-plugin-abi",
"marqdo-provider-worker",
"marqdo-plugin-sdk",
"marqdo-host",
"marqdo-runtime",
"marqdo-package",
"marqdo-sandbox",
"cli",
];
#[test]
fn workspace_crates_match_the_documented_layers() {
let manifest = read(&repo_root().join("Cargo.toml"));
let members = manifest
.split("members = [")
.nth(1)
.and_then(|rest| rest.split(']').next())
.expect("workspace members");
let names: Vec<&str> = members
.split(',')
.map(|item| item.trim().trim_matches('"'))
.filter(|item| !item.is_empty())
.collect();
let expected: Vec<String> = WORKSPACE_CRATES
.iter()
.map(|name| format!("crates/{name}"))
.collect();
assert_eq!(
names,
expected.iter().map(String::as_str).collect::<Vec<_>>(),
"workspace members drifted; update docs/architecture/CURRENT.md with the layer"
);
assert!(
!names.contains(&"crates/marqdo-stdlib"),
"the stdlib must stay in this repository (ADR-0039), not become a crate"
);
}
#[test]
fn runtime_depends_on_the_host_platform_layer_without_a_cycle() {
let runtime = read(&repo_root().join("crates/marqdo-runtime/Cargo.toml"));
assert!(
runtime
.lines()
.any(|line| line.trim() == "marqdo-host = { workspace = true }"),
"runtime must keep the documented marqdo-runtime -> marqdo-host direction"
);
let host = read(&repo_root().join("crates/marqdo-host/Cargo.toml"));
assert!(
!host.contains("marqdo-runtime"),
"marqdo-host must not depend back on marqdo-runtime"
);
}
#[test]
fn bundled_stdlib_stays_in_this_repository_and_matches_its_digest() {
let root = repo_root().join("stdlib");
let manifest: serde_json::Value =
serde_json::from_slice(&std::fs::read(root.join("stdlib.json")).unwrap()).unwrap();
assert_eq!(manifest["name"], "marqdo-stdlib");
assert_eq!(manifest["version"], "0.1.0");
let declared = manifest["digest"].as_str().expect("digest string");
let actual = format!("sha256:{}", tree_digest(&root).unwrap());
assert_eq!(
actual, declared,
"stdlib digest mismatch; regenerate it with tools/pack-stdlib.py and commit the digest"
);
}
#[test]
fn stdlib_imports_resolve_inside_the_bundle() {
let root = repo_root().join("stdlib");
for entry in walk_markdown(&root) {
let is_test = entry.starts_with(root.join("tests"));
let text = read(&entry);
for target in import_targets(&text) {
if let Some(path) = target.strip_prefix("std:") {
assert!(
is_test,
"{} imports `{}`; a product stdlib module uses a relative bundle path",
entry.display(),
target
);
let resolved = root.join(path).with_extension("mq.md");
assert!(
resolved.exists(),
"{} imports `{}`, which does not resolve inside the bundle",
entry.display(),
target
);
continue;
}
let resolved = entry.parent().unwrap().join(&target);
assert!(
resolved.starts_with(&root),
"{} imports `{}` outside the bundle",
entry.display(),
target
);
assert!(
resolved.exists(),
"{} imports missing `{}`",
entry.display(),
target
);
}
}
}
#[test]
fn host_keeps_the_documented_resource_endpoints() {
let resource = read(&repo_root().join("crates/marqdo-host/src/resource.rs"));
for endpoint in ["stdin", "stdout", "stderr", "file", "tcp"] {
let literal = format!("\"{endpoint}\"");
assert!(
resource.contains(&literal),
"host resource layer no longer mentions the documented `{endpoint}` endpoint"
);
}
}
fn import_targets(text: &str) -> Vec<String> {
let mut targets = Vec::new();
let mut in_import = false;
for line in text.lines() {
if line.trim() == "import:" {
in_import = true;
continue;
}
if in_import {
let trimmed = line.trim();
if trimmed.is_empty() {
in_import = false;
continue;
}
if !line.starts_with(' ') && !line.starts_with('\t') {
in_import = false;
continue;
}
if let Some((_, value)) = trimmed.split_once(':') {
let value = value.trim().trim_matches('"');
if !value.is_empty() {
targets.push(value.to_string());
}
}
}
}
targets
}
fn walk_markdown(dir: &Path) -> Vec<PathBuf> {
let mut out = Vec::new();
for entry in std::fs::read_dir(dir).unwrap() {
let path = entry.unwrap().path();
if path.is_dir() {
out.extend(walk_markdown(&path));
} else if path
.file_name()
.is_some_and(|name| name.to_string_lossy().ends_with(".mq.md"))
{
out.push(path);
}
}
out.sort();
out
}
fn tree_digest(root: &Path) -> Result<String, String> {
let mut files: Vec<(String, PathBuf)> = walk_markdown(root)
.into_iter()
.map(|path| {
let relative = path
.strip_prefix(root)
.map_err(|error| error.to_string())?
.to_string_lossy()
.replace('\\', "/");
Ok((relative, path))
})
.collect::<Result<_, String>>()?;
files.sort();
let mut hasher = Sha256::new();
for (relative, path) in files {
hasher.update(relative.as_bytes());
hasher.update(b"\0");
hasher.update(std::fs::read(&path).map_err(|error| error.to_string())?);
hasher.update(b"\0");
}
Ok(hex::encode(hasher.finalize()))
}
/// The CLI must consume the platform-neutral strict-child API. Direct use of
/// the Linux-only `marker_present`/`verify_active` symbols reintroduces the
/// macOS/Windows `E0425` build break (RFC-CI-0001 B01).
#[test]
fn cli_uses_only_the_platform_neutral_strict_child_api() {
let root = repo_root();
let run = read(&root.join("crates/cli/src/commands/run.rs"));
assert!(
run.contains("inspect_strict_child"),
"run.rs must use marqdo_sandbox::inspect_strict_child"
);
for source in walk_rs(&root.join("crates/cli/src")) {
let text = read(&source);
for forbidden in ["marker_present", "verify_active"] {
assert!(
!text.contains(forbidden),
"{} references Linux-only `{forbidden}`; use StrictChildState instead",
source.display()
);
}
}
let sandbox = read(&root.join("crates/marqdo-sandbox/src/lib.rs"));
assert!(
sandbox.contains("pub fn inspect_strict_child"),
"the sandbox crate must expose inspect_strict_child"
);
assert!(
!sandbox.contains("pub use linux::{marker_present"),
"the Linux-only marker functions must not be re-exported from the crate root"
);
}
fn walk_rs(root: &Path) -> Vec<PathBuf> {
let mut files = Vec::new();
let mut stack = vec![root.to_path_buf()];
while let Some(directory) = stack.pop() {
let entries = std::fs::read_dir(&directory)
.unwrap_or_else(|error| panic!("read_dir {}: {error}", directory.display()));
for entry in entries {
let entry = entry.expect("directory entry");
let path = entry.path();
if path.is_dir() {
stack.push(path);
} else if path.extension().and_then(|value| value.to_str()) == Some("rs") {
files.push(path);
}
}
}
files
}
+40 -16
View File
@@ -21,7 +21,6 @@
use std::fs;
use std::path::{Path, PathBuf};
use std::process::Command;
use marqdo_host::{Capabilities, Host, StdHost};
use marqdo_ir::{ModuleId, Program, ResolvedImport};
@@ -68,11 +67,10 @@ fn expected_path(fixture: &Path) -> PathBuf {
}
fn python_available() -> bool {
Command::new("python3")
.arg("--version")
.output()
.map(|output| output.status.success())
.unwrap_or(false)
// Ask the same platform-aware resolver the runtime uses. On Windows a
// PATH entry is `python3.exe`, not the extension-less `python3` token, so a
// raw `Command::new("python3")` probe can disagree with execution.
marqdo_foreign::detect("python").is_some()
}
/// Unique scratch directory for the host project root. Program fixtures are
@@ -266,6 +264,26 @@ fn semantic_goldens_match_end_to_end_execution() {
);
}
fn normalise_relative(path: &Path) -> String {
let mut parts: Vec<std::ffi::OsString> = Vec::new();
for component in path.components() {
match component {
std::path::Component::Normal(part) => parts.push(part.to_os_string()),
std::path::Component::ParentDir => {
parts.pop();
}
std::path::Component::CurDir
| std::path::Component::RootDir
| std::path::Component::Prefix(_) => {}
}
}
parts
.iter()
.map(|part| part.to_string_lossy().to_string())
.collect::<Vec<_>>()
.join("/")
}
/// Load the bundled stdlib print modules for direct-runtime fixtures.
///
/// The CLI loader normally injects these for Compat-B. The semantic harness
@@ -294,13 +312,23 @@ fn load_stdlib_module(
program: &mut Program,
loaded: &mut std::collections::BTreeSet<String>,
) -> Result<ModuleId, String> {
if !loaded.insert(relative.to_string()) {
let root = root.canonicalize().unwrap_or_else(|_| root.to_path_buf());
let relative = normalise_relative(Path::new(relative));
if loaded.contains(&relative) {
if let Some((id, _)) = program
.modules
.iter()
.find(|(id, _)| id.relative_path == relative)
{
return Ok(id.clone());
}
return Err(format!("stdlib import cycle at `{relative}`"));
}
let path = root.join(relative);
loaded.insert(relative.clone());
let path = root.join(&relative);
let source = fs::read_to_string(&path)
.map_err(|error| format!("cannot read `{}`: {error}", path.display()))?;
let module_id = ModuleId::new("std:test", relative.replace('\\', "/"));
let module_id = ModuleId::new("std:test", relative.clone());
let analysis = marqdo_frontend::analyze_source(&module_id.to_string(), &source);
if analysis.has_errors() {
return Err(format!("stdlib module `{relative}` has diagnostics"));
@@ -314,18 +342,14 @@ fn load_stdlib_module(
for (alias, target) in imports {
let target = target.strip_prefix("std:").unwrap_or(&target);
let child = if target.starts_with("./") || target.starts_with("../") {
let base = Path::new(relative)
let base = Path::new(&relative)
.parent()
.unwrap_or_else(|| Path::new(""));
base.join(target)
.components()
.map(|component| component.as_os_str().to_string_lossy().to_string())
.collect::<Vec<_>>()
.join("/")
normalise_relative(&base.join(target))
} else {
target.to_string()
};
let to = load_stdlib_module(root, &child, program, loaded)?;
let to = load_stdlib_module(&root, &child, program, loaded)?;
program.resolved_imports.push(ResolvedImport {
from: module_id.clone(),
alias,
+204
View File
@@ -0,0 +1,204 @@
//! Documentation consistency gate (review RFC-CM P0).
//!
//! The current architecture lives in one file, the status file must link it,
//! relative links must resolve, and a removed "running status" claim must not
//! come back. This is a source check, not a release proof.
use std::collections::BTreeSet;
use std::path::{Path, PathBuf};
fn repo_root() -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR"))
.parent()
.unwrap()
.parent()
.unwrap()
.to_path_buf()
}
fn read(path: &Path) -> String {
std::fs::read_to_string(path).unwrap_or_else(|error| {
panic!("read {}: {error}", path.display());
})
}
#[test]
fn current_architecture_has_one_overview_and_four_state_dimensions() {
let current = read(&repo_root().join("docs/architecture/CURRENT.md"));
for heading in [
"## 1. Four state dimensions",
"## 2. Responsibility layers",
"## 3. Dependency direction and public contracts",
"## 4. Extension path classification",
"## 5. Gap and closure backlog",
"## 6. Documentation governance",
] {
assert!(current.contains(heading), "CURRENT.md is missing {heading}");
}
for dimension in [
"Implemented",
"Tested locally",
"Bundled stdlib released",
"Multi-platform verified",
] {
assert!(
current.contains(dimension),
"CURRENT.md is missing the `{dimension}` state dimension"
);
}
let baseline = current
.lines()
.find(|line| line.starts_with("- Baseline:"))
.expect("CURRENT.md has no Baseline line");
let sha = baseline
.split('`')
.nth(1)
.expect("Baseline line has no backtick-quoted SHA");
assert_eq!(sha.len(), 40, "Baseline SHA is not 40 hex digits: {sha}");
assert!(
sha.chars().all(|c| c.is_ascii_hexdigit()),
"Baseline SHA is not hex: {sha}"
);
}
#[test]
fn index_and_status_point_at_the_current_overview_and_history() {
let root = repo_root();
let index = read(&root.join("docs/architecture/README.md"));
for target in ["CURRENT.md", "../STATUS.md", "../history/README.md"] {
assert!(
index.contains(target),
"architecture index does not link {target}"
);
}
// The index is an index, not a second running-status page.
assert!(
!index.contains("P5 providers") && !index.contains("P5 Provider"),
"architecture index must not duplicate the current P5 status"
);
let status = read(&root.join("docs/STATUS.md"));
assert!(
status.contains("architecture/CURRENT.md"),
"STATUS.md does not link the current overview"
);
assert!(
!status.contains("is still `c9f01aa8"),
"STATUS.md still claims the stale `c9f01aa8` remote baseline"
);
}
#[test]
fn current_relative_markdown_links_resolve() {
let root = repo_root();
for relative in [
"docs/architecture/CURRENT.md",
"docs/architecture/README.md",
"docs/STATUS.md",
"docs/history/README.md",
] {
let path = root.join(relative);
let text = read(&path);
let mut checked = BTreeSet::new();
for target in markdown_targets(&text) {
if target.starts_with("http://")
|| target.starts_with("https://")
|| target.starts_with('#')
{
continue;
}
let file = target.split('#').next().unwrap();
if file.is_empty() || !checked.insert(file.to_string()) {
continue;
}
let resolved = path.parent().unwrap().join(file);
assert!(
resolved.exists(),
"{} links missing `{target}` (resolved {})",
path.display(),
resolved.display()
);
}
}
}
/// CI/security status drift is a defect: the current architecture table must
/// not simultaneously claim a network endpoint is implemented and absent, and
/// the CI doc must not claim the GitHub-hosted Windows/macOS matrix does not
/// exist (it does; Woodpecker has no such agent).
#[test]
fn ci_and_architecture_status_do_not_contradict_the_workflows() {
let root = repo_root();
let current = read(&root.join("docs/architecture/CURRENT.md"));
assert!(
!current.contains("no DNS/TLS/server socket/UDP"),
"CURRENT.md repeats the stale TCP row that contradicts the network endpoint row"
);
assert!(
!current.contains("file metadata / directory / rename / remove / mkdir"),
"CURRENT.md repeats the stale not-implemented filesystem row"
);
let ci = read(&root.join("docs/ci.md"));
assert!(
!ci.contains("no runners are configured"),
"docs/ci.md denies the existing GitHub-hosted Windows/macOS matrix"
);
assert!(
ci.contains("windows-latest") && ci.contains("macos-latest"),
"docs/ci.md must name the native matrix it cannot replace with Woodpecker"
);
let historical = read(&root.join("docs/NEXT_RELEASE_CI_CONVERGENCE.md"));
assert!(
historical.contains("Historical / Superseded baseline"),
"the old convergence plan must keep its historical banner"
);
}
#[test]
fn evidence_files_are_sha_pinned() {
let evidence = repo_root().join("docs/evidence");
for entry in std::fs::read_dir(&evidence).expect("read evidence dir") {
let path = entry.unwrap().path();
if path.extension().and_then(|ext| ext.to_str()) != Some("md") {
continue;
}
let text = read(&path);
assert!(
has_hex_sha(&text),
"{} has no 7–40 hex commit/security SHA",
path.display()
);
}
}
fn markdown_targets(text: &str) -> Vec<String> {
let bytes = text.as_bytes();
let mut out = Vec::new();
let mut i = 0;
while i + 1 < bytes.len() {
if bytes[i] == b']' && bytes[i + 1] == b'(' {
let start = i + 2;
let mut end = start;
while end < bytes.len() && bytes[end] != b')' && bytes[end] != b'\n' {
end += 1;
}
if end < bytes.len() && bytes[end] == b')' {
out.push(text[start..end].to_string());
i = end;
}
}
i += 1;
}
out
}
fn has_hex_sha(text: &str) -> bool {
for word in text.split(|c: char| !c.is_ascii_hexdigit()) {
if (7..=40).contains(&word.len()) {
return true;
}
}
false
}
+19 -17
View File
@@ -50,39 +50,41 @@ import:
编码: "std:text"
---
## main
## @exec main
**流 = resource.open("tcp", "{address}", ["read", "write"])**
`流 = resource.open("tcp", "{address}", ["read", "write"])`
**请求 = http.构建请求("GET", "/", [["Host", "example.test"]], "")**
`请求 = http.构建请求("GET", "/", [["Host", "example.test"]], "")`
**请求内存 = memory.new(1, 1)**
`请求内存 = memory.new(1, 1)`
**请求长度 = 编码.utf8编码(请求, 请求内存, 0)**
`请求长度 = 编码.utf8编码(请求, 请求内存, 0)`
**写入 = resource.write(流, 请求内存, 0, 请求长度)**
`写入 = resource.write(流, 请求内存, 0, 请求长度)`
**接收 = memory.new(1, 1)**
`接收 = memory.new(1, 1)`
**读出 = resource.read(流, 接收, 0, 4096)**
`读出 = resource.read(流, 接收, 0, 4096)`
**数量 = 读出.value.count**
`数量 = 读出.value.count`
**响应文本 = ""**
`响应文本 = ""`
**下标 = 0**
`下标 = 0`
- `下标 < 数量`
`while 下标 < 数量`
**响应文本 = 响应文本 + text.from_codepoint(接收[下标])**
`响应文本 = 响应文本 + text.from_codepoint(接收[下标])`
**下标 = 下标 + 1**
`下标 = 下标 + 1`
**关闭 = resource.close(流)**
`end`
**响应 = http.解析响应(响应文本)**
`关闭 = resource.close(流)`
*[写入.ok, 读出.ok, 响应.状态, 响应.头["Content-Type"], 响应.体, 关闭.ok]*
`响应 = http.解析响应(响应文本)`
`return [写入.ok, 读出.ok, 响应.状态, 响应.头["Content-Type"], 响应.体, 关闭.ok]`
"#
);
std::fs::write(dir.path().join("index.mq.md"), program).unwrap();
+175 -47
View File
@@ -79,6 +79,16 @@ fn cli(dir: &Path, args: &[&str]) -> Output {
.unwrap_or_else(|error| panic!("cannot run marqdo {args:?}: {error}"))
}
/// Run the CLI in `dir` with one extra environment variable.
fn cli_env(dir: &Path, args: &[&str], key: &str, value: &str) -> Output {
Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(args)
.current_dir(dir)
.env(key, value)
.output()
.unwrap_or_else(|error| panic!("cannot run marqdo {args:?}: {error}"))
}
fn stdout_text(output: &Output) -> String {
String::from_utf8_lossy(&output.stdout).to_string()
}
@@ -170,9 +180,12 @@ fn init_creates_a_runnable_project() {
#[test]
fn check_exit_codes_and_diagnostics() {
let dir = TempDir::new("check");
dir.write("good.mq.md", "## main\n\n*1 + 1*\n");
dir.write("bad.mq.md", "## f\n\n**a *b* c**\n");
dir.write("warning.mq.md", "**x = 1**\n\n## main\n\n*1*\n");
dir.write("good.mq.md", "## @exec main\n\n`return 1 + 1`\n");
dir.write("bad.mq.md", "## @exec f\n\n**a *b* c**\n");
dir.write(
"warning.mq.md",
"**x = 1**\n\n## @exec main\n\n`return 1`\n",
);
let good = cli(dir.path(), &["check", "good.mq.md"]);
assert_exit(&good, 0, "clean document");
@@ -181,7 +194,7 @@ fn check_exit_codes_and_diagnostics() {
let bad = cli(dir.path(), &["check", "bad.mq.md"]);
assert_exit(&bad, 1, "invalid executable shape is a hard error");
let bad_stderr = stderr_text(&bad);
assert!(bad_stderr.contains("error[markdown.invalid_construct]"));
assert!(bad_stderr.contains("error[sem.payload_nested]"));
assert!(bad_stderr.contains("check failed"));
assert!(stdout_text(&bad).is_empty());
@@ -192,20 +205,36 @@ fn check_exit_codes_and_diagnostics() {
assert_eq!(value["success"], false);
assert_eq!(value["errors"], 1);
let diagnostics = value["diagnostics"].as_array().expect("diagnostics array");
assert_eq!(diagnostics.len(), 1);
assert_eq!(diagnostics[0]["code"], "markdown.invalid_construct");
assert_eq!(diagnostics[0]["severity"], "error");
assert_eq!(diagnostics[0]["line"], 3);
let nested = diagnostics
.iter()
.find(|diagnostic| diagnostic["code"] == "sem.payload_nested")
.expect("sem.payload_nested diagnostic");
assert_eq!(nested["severity"], "error");
assert_eq!(nested["line"], 3);
assert!(
diagnostics
.iter()
.all(|diagnostic| diagnostic["severity"] == "error"
|| diagnostic["code"] == "sem.inferred_signature"),
"the fixture may carry the inferred-signature warning but no other diagnostic"
);
assert!(stderr_text(&bad_json).is_empty());
let warning = cli(dir.path(), &["check", "warning.mq.md"]);
assert_exit(&warning, 0, "a warning does not fail check");
assert!(stdout_text(&warning).contains("checked 1 module(s): ok"));
assert!(stderr_text(&warning).contains("warning[markdown.inert_executable]"));
let warning_json = cli(dir.path(), &["check", "warning.mq.md", "--json"]);
let value = json_stdout(&warning_json);
// RFC-LANG-0002: a Strong at the Knowledge root is inert prose, not an
// executable shape, and the old `markdown.inert_executable` warning no
// longer exists.
let inert = cli(dir.path(), &["check", "warning.mq.md"]);
assert_exit(&inert, 0, "inert Knowledge prose does not fail check");
assert!(stdout_text(&inert).contains("checked 1 module(s): ok"));
assert!(!stderr_text(&inert).contains("warning["));
let inert_json = cli(dir.path(), &["check", "warning.mq.md", "--json"]);
let value = json_stdout(&inert_json);
assert_eq!(value["success"], true);
assert_eq!(value["diagnostics"][0]["severity"], "warning");
assert_eq!(
value["diagnostics"].as_array().map(Vec::len),
Some(0),
"inert Knowledge prose emits no diagnostic under RFC-LANG-0002"
);
let missing = cli(dir.path(), &["check", "missing.mq.md"]);
assert_exit(&missing, 2, "missing file is a usage/IO error");
@@ -217,7 +246,7 @@ fn run_json_embeds_program_stdout() {
let dir = TempDir::new("run-json");
dir.write(
"print.mq.md",
"## main\n\n> 打印(\"你好\")\n\n> 打印(42)\n\n*None*\n",
"## @exec main\n\n`打印(\"你好\")`\n\n`打印(42)`\n\n`return None`\n",
);
let output = cli(dir.path(), &["run", "print.mq.md", "--json"]);
assert_exit(&output, 0, "run --json must succeed");
@@ -240,11 +269,11 @@ fn run_json_embeds_program_stdout() {
#[test]
fn run_refuses_documents_with_compile_errors() {
let dir = TempDir::new("run-errors");
dir.write("bad.mq.md", "## f\n\n**a *b* c**\n");
dir.write("bad.mq.md", "## @exec f\n\n**a *b* c**\n");
let output = cli(dir.path(), &["run", "bad.mq.md"]);
assert_exit(&output, 1, "compile errors must prevent execution");
let stderr = stderr_text(&output);
assert!(stderr.contains("markdown.invalid_construct"));
assert!(stderr.contains("sem.payload_nested"));
assert!(stderr.contains("refusing to run"));
assert!(stdout_text(&output).is_empty());
}
@@ -449,6 +478,90 @@ fn strict_mode_enforces_isolation_or_refuses_closed() {
.contains("strict mode"));
}
/// A forged `MARQDO_STRICT_ACTIVE` marker must never be trusted and every
/// `--json` refusal must still contain exactly one JSON document. This runs on
/// every platform; Linux rejects the marker during identity verification and
/// non-Linux builds reject it as unsupported.
#[test]
fn forged_strict_marker_is_rejected_with_one_json_document() {
let root = repo_root();
let hello = root.join("examples").join("hello.mq.md");
let hello = hello.to_str().unwrap();
let json = cli_env(
&root,
&["run", hello, "--mode", "strict", "--json"],
"MARQDO_STRICT_ACTIVE",
"1",
);
assert_exit(&json, 1, "forged strict marker must fail closed");
let value = json_stdout(&json);
assert_eq!(value["command"], "run");
assert_eq!(value["success"], false);
assert_eq!(value["code"], "strict.identity_invalid");
let human = cli_env(
&root,
&["run", hello, "--mode", "strict"],
"MARQDO_STRICT_ACTIVE",
"1",
);
assert_exit(&human, 1, "forged strict marker must fail closed");
assert!(
stdout_text(&human).is_empty(),
"project output appeared for a rejected marker: {}",
stdout_text(&human)
);
assert!(stderr_text(&human).contains("strict isolation verification failed"));
}
/// CLI usage errors in `--json` mode keep the same one-document contract as
/// successful and runtime-failure paths.
#[test]
fn json_mode_reports_cli_errors_as_one_document() {
let root = repo_root();
let hello = root.join("examples").join("hello.mq.md");
let hello = hello.to_str().unwrap();
let mode = cli(&root, &["run", hello, "--mode", "bogus", "--json"]);
assert_exit(&mode, 2, "unknown mode");
let value = json_stdout(&mode);
assert_eq!(value["success"], false);
assert_eq!(value["code"], "cli.invalid_mode");
let grants = cli(
&root,
&[
"run",
hello,
"--mode",
"pure",
"--allow-read",
"x",
"--json",
],
);
assert_exit(&grants, 2, "pure mode must reject grants");
let value = json_stdout(&grants);
assert_eq!(value["success"], false);
assert_eq!(value["code"], "cli.invalid_grants");
let strict = cli(
&root,
&[
"run",
hello,
"--mode",
"strict",
"--allow-foreign",
"--json",
],
);
assert_exit(&strict, 2, "strict L1 must reject foreign grants");
let value = json_stdout(&strict);
assert_eq!(value["success"], false);
assert_eq!(value["code"], "strict.unsupported_grant");
}
#[test]
fn inspect_outputs_ast_units_signatures_ir_and_refs() {
let root = repo_root();
@@ -501,7 +614,7 @@ fn inspect_outputs_ast_units_signatures_ir_and_refs() {
let dir = TempDir::new("refs");
dir.write(
"broken.mq.md",
"## main\n\n见[断链](#不存在的标题)。\n\n*1*\n",
"## @exec main\n\n见[断链](#不存在的标题)。\n\n`return 1`\n",
);
let broken = cli(dir.path(), &["inspect", "broken.mq.md", "--refs", "--json"]);
assert_exit(&broken, 0, "inspect --refs never executes the document");
@@ -515,7 +628,7 @@ fn inspect_outputs_ast_units_signatures_ir_and_refs() {
#[test]
fn fmt_normalises_line_endings_and_trailing_newlines() {
let dir = TempDir::new("fmt");
dir.write("doc.mq.md", "## main\r\n\r\n*1*\r\n\r\n\r\n");
dir.write("doc.mq.md", "## @exec main\r\n\r\n`return 1`\r\n\r\n\r\n");
let before = cli(dir.path(), &["fmt", "--check", "doc.mq.md"]);
assert_exit(&before, 1, "unformatted document is detected");
assert!(stderr_text(&before).contains("is not formatted"));
@@ -523,7 +636,7 @@ fn fmt_normalises_line_endings_and_trailing_newlines() {
let format = cli(dir.path(), &["fmt", "doc.mq.md"]);
assert_exit(&format, 0, "fmt must rewrite the document");
let formatted = dir.read("doc.mq.md");
assert_eq!(formatted, "## main\n\n*1*\n");
assert_eq!(formatted, "## @exec main\n\n`return 1`\n");
assert!(!formatted.contains('\r'));
let after = cli(dir.path(), &["fmt", "--check", "doc.mq.md"]);
@@ -535,11 +648,11 @@ fn multi_module_imports_and_cycles() {
let dir = TempDir::new("modules");
dir.write(
"geometry.mq.md",
"## 圆面积\n\n`(r: Number)`\n\n*math.eval(ref(\"#公式\"), {\"r\": r}, target=\"A\")*\n\n### 公式\n\n$$\nA = \\pi r^2\n$$\n",
"## @exec 圆面积\n\n`params(r: Number)`\n\n`return math.eval(ref(\"#公式\"), {\"r\": r}, target=\"A\")`\n\n### @knowledge 公式\n\n$$\nA = \\pi r^2\n$$\n",
);
dir.write(
"index.mq.md",
"---\nimport:\n geometry: \"./geometry.mq.md\"\n---\n\n## main\n\n**面积 = geometry.圆面积(r=2)**\n\n> 打印(面积)\n",
"---\nimport:\n geometry: \"./geometry.mq.md\"\n---\n\n## @exec main\n\n`面积 = geometry.圆面积(r=2)`\n\n`打印(面积)`\n",
);
let run = cli(dir.path(), &["run", "index.mq.md"]);
assert_exit(&run, 0, "multi-module import must run");
@@ -552,11 +665,11 @@ fn multi_module_imports_and_cycles() {
let cycle = TempDir::new("cycle");
cycle.write(
"a.mq.md",
"---\nimport:\n b: \"./b.mq.md\"\n---\n\n## main\n\n*1*\n",
"---\nimport:\n b: \"./b.mq.md\"\n---\n\n## @exec main\n\n`return 1`\n",
);
cycle.write(
"b.mq.md",
"---\nimport:\n a: \"./a.mq.md\"\n---\n\n## f\n\n*1*\n",
"---\nimport:\n a: \"./a.mq.md\"\n---\n\n## @exec f\n\n`return 1`\n",
);
let output = cli(cycle.path(), &["check", "a.mq.md"]);
assert_exit(&output, 1, "import cycle is an error");
@@ -568,7 +681,7 @@ fn multi_module_imports_and_cycles() {
let missing = TempDir::new("missing-import");
missing.write(
"index.mq.md",
"---\nimport:\n gone: \"./gone.mq.md\"\n---\n\n## main\n\n*1*\n",
"---\nimport:\n gone: \"./gone.mq.md\"\n---\n\n## @exec main\n\n`return 1`\n",
);
let output = cli(missing.path(), &["check", "index.mq.md"]);
assert_exit(&output, 1, "missing import target is an error");
@@ -582,8 +695,8 @@ fn lock_and_sync_locked_detect_dependency_tampering() {
"marqdo.yaml",
"project:\n name: locktest\n version: 0.1.0\ndependencies:\n dep:\n path: \"./dep\"\n",
);
dir.write("index.mq.md", "## main\n\n*1*\n");
dir.write("dep/index.mq.md", "## main\n\n*1*\n");
dir.write("index.mq.md", "## @exec main\n\n`return 1`\n");
dir.write("dep/index.mq.md", "## @exec main\n\n`return 1`\n");
let lock = cli(dir.path(), &["lock"]);
assert_exit(&lock, 0, "lock must succeed");
@@ -596,7 +709,7 @@ fn lock_and_sync_locked_detect_dependency_tampering() {
assert!(stdout_text(&sync).contains("synchronized 1 package(s)"));
// Tamper with the dependency contents: `sync --locked` must fail.
dir.write("dep/index.mq.md", "## main\n\n*2*\n");
dir.write("dep/index.mq.md", "## @exec main\n\n`return 2`\n");
let tampered = cli(dir.path(), &["sync", "--locked"]);
assert_exit(&tampered, 1, "tampered dependency must fail sync --locked");
assert!(stderr_text(&tampered).contains("lockfile is out of date"));
@@ -711,7 +824,7 @@ fn plugin_inspect_and_explicit_trusted_invocation() {
let dir = TempDir::new("plugin-run");
dir.write(
"doc.mq.md",
"## main\n\n**结果 = solver.add(a=20, b=22)**\n\n> 打印(结果)\n",
"## @exec main\n\n`结果 = solver.add(a=20, b=22)`\n\n`打印(结果)`\n",
);
let plugin_path = plugin_dir.to_str().unwrap();
let doc = dir.path().join("doc.mq.md");
@@ -756,7 +869,7 @@ fn plugin_inspect_and_explicit_trusted_invocation() {
// Unknown plugin namespace is a hard runtime symbol error.
dir.write(
"unknown.mq.md",
"## main\n\n**结果 = other.add(a=1, b=1)**\n",
"## @exec main\n\n`结果 = other.add(a=1, b=1)`\n",
);
let unknown = cli(
dir.path(),
@@ -775,7 +888,7 @@ fn plugin_inspect_and_explicit_trusted_invocation() {
// The export signature requires both named arguments: with a declared
// plugin manifest this is rejected statically, before any library load.
dir.write("args.mq.md", "## main\n\n**结果 = solver.add(a=1)**\n");
dir.write("args.mq.md", "## @exec main\n\n`结果 = solver.add(a=1)`\n");
let bad_args = cli(
dir.path(),
&[
@@ -798,11 +911,11 @@ fn runtime_error_in_imported_module_reports_that_module() {
let dir = TempDir::new("cross-module-error");
dir.write(
"index.mq.md",
"---\nimport:\n physics: \"./physics.mq.md\"\n---\n\n## main\n\n> 打印(physics.boom())\n",
"---\nimport:\n physics: \"./physics.mq.md\"\n---\n\n## @exec main\n\n`打印(physics.boom())`\n",
);
dir.write(
"physics.mq.md",
"## boom\n\n说明文字。\n\n**x = 1 / 0**\n\n*x*\n",
"## @exec boom\n\n说明文字。\n\n`x = 1 / 0`\n\n`return x`\n",
);
let output = cli(dir.path(), &["run", "index.mq.md", "--json"]);
@@ -957,7 +1070,7 @@ fn plugin_abi_major_mismatch_is_rejected_before_execution() {
let dir = TempDir::new("plugin-abi-run");
dir.write(
"doc.mq.md",
"## main\n\n**结果 = solver.add(a=20, b=22)**\n\n> 打印(结果)\n",
"## @exec main\n\n`结果 = solver.add(a=20, b=22)`\n\n`打印(结果)`\n",
);
let output = cli(
dir.path(),
@@ -1019,7 +1132,7 @@ fn check_never_spawns_a_foreign_kernel() {
);
dir.write(
"doc.mq.md",
"## 代码\n\n```mocklang\nignored\n```\n\n## main\n\n**值 = foreign.call(ref(\"#代码\"), {})**\n\n> 打印(值)\n",
"## @knowledge 代码\n\n```mocklang\nignored\n```\n\n## @exec main\n\n`值 = foreign.call(ref(\"#代码\"), {})`\n\n`打印(值)`\n",
);
let checked = cli(dir.path(), &["check", "doc.mq.md"]);
@@ -1051,7 +1164,7 @@ fn power_branch_loop_and_inert_code_end_to_end() {
let dir = TempDir::new("power");
dir.write(
"branch.mq.md",
"## main\n\n**n = 2**\n\n1. `n ** 2 == 4`,正确分支。\n\n > print(\"乘方正确\")\n\n2. `n ** 2 != 4`,另一分支。\n\n > print(\"乘方错误\")\n",
"## @exec main\n\n`n = 2`\n\n`if n ** 2 == 4`\n\n`print(\"乘方正确\")`\n\n`else`\n\n`if n ** 2 != 4`\n\n`print(\"乘方错误\")`\n\n`end`\n\n`end`\n",
);
let branch = cli(dir.path(), &["run", "branch.mq.md", "--json"]);
assert_exit(&branch, 0, "power branch run");
@@ -1062,7 +1175,7 @@ fn power_branch_loop_and_inert_code_end_to_end() {
dir.write(
"loop.mq.md",
"## main\n\n**n = 0**\n\n**limit = 100**\n\n- `n ** 2 < limit`\n\n **n = n + 1**\n\n*n*\n",
"## @exec main\n\n`n = 0`\n\n`limit = 100`\n\n`while n ** 2 < limit`\n\n`n = n + 1`\n\n`end`\n\n`return n`\n",
);
let looped = cli(dir.path(), &["run", "loop.mq.md", "--json"]);
assert_exit(&looped, 0, "power loop run");
@@ -1074,7 +1187,7 @@ fn power_branch_loop_and_inert_code_end_to_end() {
// diagnostics and never changes the returned value.
dir.write(
"inert.mq.md",
"## main\n\n```text\n**n = 2 ** 3**\n```\n\n正文里的 `2 ** 3` 不执行。\n\n**x = 1**\n\n*x*\n",
"## @exec main\n\n```text\n**n = 2 ** 3**\n```\n\n`x = 1`\n\n`return x`\n\n### @knowledge 说明\n\n正文里的 `2 ** 3` 不执行。\n",
);
let inert = cli(dir.path(), &["run", "inert.mq.md", "--json"]);
assert_exit(&inert, 0, "inert code run");
@@ -1085,7 +1198,7 @@ fn power_branch_loop_and_inert_code_end_to_end() {
#[test]
fn condition_parse_diagnostic_points_inside_the_code_span() {
let dir = TempDir::new("condition-span");
let source = "## main\n\n**n = 2**\n\n1. `n ** 2 ==`\n\n > print(1)\n";
let source = "## @exec main\n\n`n = 2`\n\n`if n ** 2 ==`\n\n`print(1)`\n\n`end`\n";
dir.write("bad.mq.md", source);
let output = cli(dir.path(), &["check", "bad.mq.md", "--json"]);
assert_exit(&output, 1, "condition parse error");
@@ -1094,12 +1207,27 @@ fn condition_parse_diagnostic_points_inside_the_code_span() {
.as_array()
.expect("diagnostics array")
.iter()
.find(|diagnostic| diagnostic["code"] == "expr.parse")
.expect("expr.parse diagnostic");
.find(|diagnostic| diagnostic["code"] == "sem.syntax")
.expect("sem.syntax diagnostic");
assert_eq!(diagnostic["line"], 5);
// 1-based column 14 is the closing backtick: the empty end-of-input span
// must not be shifted by the delimiter.
assert_eq!(diagnostic["column"], 14);
let line = source.lines().nth(4).expect("line 5");
assert_eq!(&line[13..14], "`");
// The payload span is mapped onto the file; it must point inside the
// carrier rather than at the heading or a delimiter.
let column = diagnostic["column"].as_u64().expect("column");
assert!((11..=14).contains(&column), "unexpected column {column}");
}
#[test]
fn run_reports_sem_no_entry_function_without_exec_main() {
let dir = TempDir::new("no-entry");
dir.write(
"index.mq.md",
"# @knowledge 文档\n\n这里没有可调用的函数。\n",
);
let output = cli(dir.path(), &["run", "index.mq.md"]);
assert_exit(&output, 1, "missing Exec H2 main is an error");
let stderr = stderr_text(&output);
assert!(
stderr.contains("sem.no_entry_function"),
"stderr must carry the stable code: {stderr}"
);
}
+2 -2
View File
@@ -129,7 +129,7 @@ fn foreign_and_plugin_shortcuts_are_unavailable_in_the_pure_lane() {
let foreign = dir.path().join("foreign.mq.md");
std::fs::write(
&foreign,
"## 代码\n\n```python\nprint(1)\n```\n\n## main\n\n*foreign.call(ref(\"#代码\"), None)*\n",
"## @knowledge 代码\n\n```python\nprint(1)\n```\n\n## @exec main\n\n`return foreign.call(ref(\"#代码\"), None)`\n",
)
.unwrap();
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
@@ -145,7 +145,7 @@ fn foreign_and_plugin_shortcuts_are_unavailable_in_the_pure_lane() {
let plugin = dir.path().join("plugin.mq.md");
std::fs::write(
&plugin,
"## main\n\n> plugin.invoke(\"missing.export\", {})\n\n*None*\n",
"## @exec main\n\n`plugin.invoke(\"missing.export\", {})`\n\n`return None`\n",
)
.unwrap();
let output = Command::new(env!("CARGO_BIN_EXE_marqdo"))
+46 -19
View File
@@ -82,20 +82,23 @@ fn transitive_path_dependency_resolves_from_its_own_manifest() {
);
write(
&root.join("deps/a/index.mq.md"),
"---\nimport:\n b: \"@b\"\n---\n\n## outer\n\n*b.inner()*\n",
"---\nimport:\n b: \"@b\"\n---\n\n## @exec outer\n\n`return b.inner()`\n",
);
write(
&root.join("deps/a/b/marqdo.yaml"),
"project:\n name: b\n version: 0.1.0\n",
);
write(&root.join("deps/a/b/index.mq.md"), "## inner\n\n*42*\n");
write(
&root.join("deps/a/b/index.mq.md"),
"## @exec inner\n\n`return 42`\n",
);
write(
&root.join("marqdo.yaml"),
"project:\n name: root\n version: 0.1.0\ndependencies:\n a:\n path: ./deps/a\n",
);
write(
&root.join("index.mq.md"),
"---\nimport:\n a: \"@a\"\n---\n\n## main\n\n> 打印(a.outer())\n",
"---\nimport:\n a: \"@a\"\n---\n\n## @exec main\n\n`打印(a.outer())`\n",
);
let lock = cli(root, &["lock"]);
assert_eq!(
@@ -133,7 +136,7 @@ fn pinned_git_dependency_locks_syncs_and_runs() {
&dep.join("marqdo.yaml"),
"project:\n name: remote\n version: 0.3.0\n",
);
write(&dep.join("index.mq.md"), "## hello\n\n*7*\n");
write(&dep.join("index.mq.md"), "## @exec hello\n\n`return 7`\n");
git(&dep, &["init", "-q"]);
git(&dep, &["add", "."]);
git(&dep, &["commit", "-q", "-m", "initial"]);
@@ -156,7 +159,7 @@ fn pinned_git_dependency_locks_syncs_and_runs() {
);
write(
&root.join("index.mq.md"),
"---\nimport:\n remote: \"@remote\"\n---\n\n## main\n\n> 打印(remote.hello())\n",
"---\nimport:\n remote: \"@remote\"\n---\n\n## @exec main\n\n`打印(remote.hello())`\n",
);
let lock = cli(&root, &["lock"]);
@@ -206,7 +209,7 @@ fn check_does_not_fetch_a_missing_git_revision() {
);
write(
&root.join("index.mq.md"),
"---\nimport:\n remote: \"@remote\"\n---\n\n## main\n\n*1*\n",
"---\nimport:\n remote: \"@remote\"\n---\n\n## @exec main\n\n`return 1`\n",
);
let output = cli(root, &["check", "index.mq.md"]);
assert_eq!(output.status.code(), Some(1));
@@ -223,10 +226,13 @@ fn run_and_check_require_a_lock_when_dependencies_exist() {
&root.join("marqdo.yaml"),
"project:\n name: root\n version: 0.1.0\ndependencies:\n dep:\n path: ./dep\n",
);
write(&root.join("dep/index.mq.md"), "## value\n\n*1*\n");
write(
&root.join("dep/index.mq.md"),
"## @exec value\n\n`return 1`\n",
);
write(
&root.join("index.mq.md"),
"---\nimport:\n dep: \"@dep\"\n---\n\n## main\n\n*1*\n",
"---\nimport:\n dep: \"@dep\"\n---\n\n## @exec main\n\n`return 1`\n",
);
for command in ["check", "run"] {
let output = cli(root, &[command, "index.mq.md"]);
@@ -244,14 +250,20 @@ fn sync_is_strict_and_never_rewrites_the_lock() {
&root.join("marqdo.yaml"),
"project:\n name: root\n version: 0.1.0\ndependencies:\n dep:\n path: ./dep\n",
);
write(&root.join("dep/index.mq.md"), "## value\n\n*1*\n");
write(&root.join("index.mq.md"), "## main\n\n*1*\n");
write(
&root.join("dep/index.mq.md"),
"## @exec value\n\n`return 1`\n",
);
write(&root.join("index.mq.md"), "## @exec main\n\n`return 1`\n");
let lock = cli(root, &["lock"]);
assert_eq!(lock.status.code(), Some(0));
let original = std::fs::read(root.join("marqdo.lock")).unwrap();
write(&root.join("dep/index.mq.md"), "## value\n\n*2*\n");
write(
&root.join("dep/index.mq.md"),
"## @exec value\n\n`return 2`\n",
);
let stale = cli(root, &["sync"]);
assert_eq!(stale.status.code(), Some(1), "plain sync must be strict");
let stderr = String::from_utf8_lossy(&stale.stderr);
@@ -285,13 +297,19 @@ fn lock_records_transitive_closure_and_sync_materializes_it() {
&root.join("deps/a/marqdo.yaml"),
"project:\n name: a\n version: 0.1.0\ndependencies:\n b:\n path: ./b\n",
);
write(&root.join("deps/a/index.mq.md"), "## a\n\n*1*\n");
write(
&root.join("deps/a/index.mq.md"),
"## @exec a\n\n`return 1`\n",
);
write(
&root.join("deps/a/b/marqdo.yaml"),
"project:\n name: b\n version: 0.1.0\n",
);
write(&root.join("deps/a/b/index.mq.md"), "## b\n\n*2*\n");
write(&root.join("index.mq.md"), "## main\n\n*1*\n");
write(
&root.join("deps/a/b/index.mq.md"),
"## @exec b\n\n`return 2`\n",
);
write(&root.join("index.mq.md"), "## @exec main\n\n`return 1`\n");
let lock = cli(root, &["lock"]);
assert_eq!(lock.status.code(), Some(0));
@@ -326,10 +344,13 @@ fn deleted_lock_edge_fails_check_and_run() {
&root.join("marqdo.yaml"),
"project:\n name: root\n version: 0.1.0\ndependencies:\n dep:\n path: ./dep\n",
);
write(&root.join("dep/index.mq.md"), "## value\n\n*1*\n");
write(
&root.join("dep/index.mq.md"),
"## @exec value\n\n`return 1`\n",
);
write(
&root.join("index.mq.md"),
"---\nimport:\n dep: \"@dep\"\n---\n\n## main\n\n*1*\n",
"---\nimport:\n dep: \"@dep\"\n---\n\n## @exec main\n\n`return 1`\n",
);
assert_eq!(cli(root, &["lock"]).status.code(), Some(0));
let lock = std::fs::read_to_string(root.join("marqdo.lock")).unwrap();
@@ -362,10 +383,13 @@ fn tampered_path_dependency_content_is_rejected_by_locked_sync() {
&root.join("dep/marqdo.yaml"),
"project:\n name: dep\n version: 0.1.0\n",
);
write(&root.join("dep/index.mq.md"), "## value\n\n*1*\n");
write(
&root.join("dep/index.mq.md"),
"## @exec value\n\n`return 1`\n",
);
write(
&root.join("index.mq.md"),
"---\nimport:\n dep: \"@dep\"\n---\n\n## main\n\n*1*\n",
"---\nimport:\n dep: \"@dep\"\n---\n\n## @exec main\n\n`return 1`\n",
);
assert_eq!(cli(root, &["lock"]).status.code(), Some(0));
let lock_before = std::fs::read(root.join("marqdo.lock")).unwrap();
@@ -375,7 +399,10 @@ fn tampered_path_dependency_content_is_rejected_by_locked_sync() {
// Content change after the lock is a tamper/staleness signal. `run` and
// `check` must never silently accept it, re-lock, or materialize a new
// environment.
write(&root.join("dep/index.mq.md"), "## value\n\n*2*\n");
write(
&root.join("dep/index.mq.md"),
"## @exec value\n\n`return 2`\n",
);
for args in [
vec!["sync", "--locked"],
vec!["run", "index.mq.md"],
+2 -2
View File
@@ -112,7 +112,7 @@ fn write_manifest(dir: &Path, sha: &str) {
.unwrap();
std::fs::write(
dir.join("doc.mq.md"),
"## main\n\n**结果 = solver.add(a=20, b=22)**\n\n> 打印(结果)\n",
"## @exec main\n\n`结果 = solver.add(a=20, b=22)`\n\n`打印(结果)`\n",
)
.unwrap();
}
@@ -227,7 +227,7 @@ fn write_capability_manifest(dir: &Path, sha: &str) {
.unwrap();
std::fs::write(
dir.join("doc.mq.md"),
"## main\n\n**结果 = capability.read()**\n\n> 打印(结果)\n",
"## @exec main\n\n`结果 = capability.read()`\n\n`打印(结果)`\n",
)
.unwrap();
std::fs::create_dir_all(dir.join("data")).unwrap();
+3 -3
View File
@@ -44,7 +44,7 @@ fn print_is_routed_through_the_stdlib_and_not_hostop_print() {
write(
dir.path(),
"index.mq.md",
"## main\n\n> 打印([\"x\", 1])\n\n*None*\n",
"## @exec main\n\n`打印([\"x\", 1])`\n\n`return None`\n",
);
let trace = dir.path().join("trace.json");
let output = cli(
@@ -84,11 +84,11 @@ fn print_is_routed_through_the_stdlib_and_not_hostop_print() {
#[test]
fn stdlib_print_modules_are_injected_only_when_print_is_used() {
let dir = TempDir::new("lazy");
write(dir.path(), "plain.mq.md", "## main\n\n*1*\n");
write(dir.path(), "plain.mq.md", "## @exec main\n\n`return 1`\n");
write(
dir.path(),
"prints.mq.md",
"## main\n\n> 打印(1)\n\n*None*\n",
"## @exec main\n\n`打印(1)`\n\n`return None`\n",
);
for (file, expected_stdlib) in [("plain.mq.md", false), ("prints.mq.md", true)] {
let target = dir.path().join(format!("{file}.json"));
+75 -53
View File
@@ -29,39 +29,45 @@ struct Base {
const BASES: &[Base] = &[
Base {
name: "factorial",
source: r##"# 数学
source: r##"# @knowledge 数学
## 阶乘
## @exec 阶乘
计算阶乘的叙述一。
1. `n <= 1`
`if n <= 1`
递归终点叙述。
递归终点叙述。
*1*
`return 1`
2. `n > 1`
`else`
递归步骤叙述。
`if n > 1`
**结果 = n * 阶乘(n=n-1)**
递归步骤叙述。
*结果*
`结果 = n * 阶乘(n=n-1)`
`return 结果`
`end`
`end`
"##,
narration: &["计算阶乘的叙述一。", "递归终点叙述。", "递归步骤叙述。"],
},
Base {
name: "explicit-signature",
source: r##"## 圆面积
source: r##"## @exec 圆面积
`(r: Number)`
`params(r: Number)`
半径叙述。
*math.eval(ref("#公式"), {"r": r}, target="A")*
`return math.eval(ref("#公式"), {"r": r}, target="A")`
### 公式
### @knowledge 公式
$$
A = \pi r^2
@@ -71,117 +77,133 @@ $$
},
Base {
name: "countdown-loop",
source: r##"## 倒计时
source: r##"## @exec 倒计时
循环叙述。
**n = 3**
`n = 3`
- `n > 0`
`while n > 0`
循环体叙述。
循环体叙述。
> 打印(n)
`打印(n)`
**n = n - 1**
`n = n - 1`
*n*
`end`
`return n`
"##,
narration: &["循环叙述。", "循环体叙述。"],
},
Base {
name: "branch-classification",
source: r##"## 分类
source: r##"## @exec 分类
背景叙述。
**结果 = 0**
`结果 = 0`
1. `n < 0`
`if n < 0`
负数分支叙述。
负数分支叙述。
**结果 = -1**
`结果 = -1`
2. `n >= 0`
`else`
非负分支叙述。
`if n >= 0`
**结果 = 1**
非负分支叙述。
*结果*
`结果 = 1`
`end`
`end`
`return 结果`
"##,
narration: &["背景叙述。", "负数分支叙述。", "非负分支叙述。"],
},
Base {
name: "call-and-print",
source: r##"## 问候
source: r##"## @exec 问候
说明为什么这样写。
> 打印("你好")
`打印("你好")`
**问候 = "你好,Marqdo"**
`问候 = "你好,Marqdo"`
再补充一句叙述。
*问候*
`return 问候`
"##,
narration: &["说明为什么这样写。", "再补充一句叙述。"],
},
Base {
name: "inline-prose-in-executable-paragraph",
source: r##"## 加一
source: r##"## @exec 加一
这是行内叙述 **结果 = 1 + 1** 后面继续叙述。
*结果*
`return 结果`
"##,
narration: &["这是行内叙述 ", " 后面继续叙述。"],
},
Base {
name: "nested-control-flow",
source: r##"## 嵌套
source: r##"## @exec 嵌套
说明嵌套结构。
**total = 0**
`total = 0`
1. `n > 0`
`if n > 0`
**m = 3**
`m = 3`
- `m > 0`
`while m > 0`
**total = total + m**
`total = total + m`
**m = m - 1**
`m = m - 1`
2. `n <= 0`
`end`
**total = 0**
`else`
*total*
`if n <= 0`
`total = 0`
`end`
`end`
`return total`
"##,
narration: &["说明嵌套结构。"],
},
Base {
name: "two-namespaces",
source: r##"# 数学
source: r##"# @knowledge 数学
命名空间叙述。
## 平方
## @exec 平方
**结果 = n * n**
`结果 = n * n`
*结果*
`return 结果`
# 文本
# @knowledge 文本
## 长度
## @exec 长度
*长度(s)*
`return 长度(s)`
"##,
narration: &["命名空间叙述。"],
},
@@ -265,7 +287,7 @@ fn insert_prose_after_headings(source: &str) -> String {
while next < lines.len() && lines[next].trim().is_empty() {
next += 1;
}
let signature = next < lines.len() && lines[next].trim_start().starts_with("`(");
let signature = next < lines.len() && lines[next].trim_start().starts_with("`params(");
if signature {
while index + 1 < lines.len() && lines[index + 1].trim().is_empty() {
index += 1;
+204 -49
View File
@@ -1,7 +1,8 @@
//! End-to-end resource stream tests through the CLI.
use std::io::Write;
use std::path::{Path, PathBuf};
use std::process::{Command, Output};
use std::process::{Command, Output, Stdio};
struct TempDir(PathBuf);
@@ -33,25 +34,43 @@ fn cli(dir: &Path, args: &[&str]) -> Output {
.expect("run marqdo")
}
fn cli_with_stdin(dir: &Path, args: &[&str], input: &[u8]) -> Output {
let mut child = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(args)
.current_dir(dir)
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.expect("spawn marqdo");
child
.stdin
.take()
.expect("stdin pipe")
.write_all(input)
.expect("write stdin");
child.wait_with_output().expect("wait for marqdo")
}
fn write_file(path: &Path, contents: &str) {
std::fs::write(path, contents).unwrap();
}
const READER: &str = r#"## main
const READER: &str = r#"## @exec main
**内存 = memory.new(1, 1)**
`内存 = memory.new(1, 1)`
**流 = resource.open("file", "data.txt", ["read"])**
`流 = resource.open("file", "data.txt", ["read"])`
**第一次 = resource.read(流, 内存, 0, 2)**
`第一次 = resource.read(流, 内存, 0, 2)`
**第二次 = resource.read(流, 内存, 2, 4)**
`第二次 = resource.read(流, 内存, 2, 4)`
**第三次 = resource.read(流, 内存, 6, 1)**
`第三次 = resource.read(流, 内存, 6, 1)`
**关闭 = resource.close(流)**
`关闭 = resource.close(流)`
*[第一次.value.count, 第一次.value.eof, 第二次.value.count, 第二次.value.eof, 第三次.value.count, 第三次.value.eof, 内存[0], 内存[1], 内存[2], 内存[3], 内存[4], 内存[5], 关闭.ok]*
`return [第一次.value.count, 第一次.value.eof, 第二次.value.count, 第二次.value.eof, 第三次.value.count, 第三次.value.eof, 内存[0], 内存[1], 内存[2], 内存[3], 内存[4], 内存[5], 关闭.ok]`
"#;
#[test]
@@ -91,29 +110,29 @@ fn project_file_stream_reads_chunks_into_memory() {
assert_eq!(values[12]["value"], true);
}
const WRITER: &str = r#"## main
const WRITER: &str = r#"## @exec main
**内存 = memory.new(1, 1)**
`内存 = memory.new(1, 1)`
**`内存[0] = 65`**
`内存[0] = 65`
**`内存[1] = 66`**
`内存[1] = 66`
**`内存[2] = 67`**
`内存[2] = 67`
**写 = resource.open("file", "out.bin", ["write"])**
`写 = resource.open("file", "out.bin", ["write"])`
**写入 = resource.write(写, 内存, 0, 3)**
`写入 = resource.write(写, 内存, 0, 3)`
**关闭写 = resource.close(写)**
`关闭写 = resource.close(写)`
**读 = resource.open("file", "out.bin", ["read"])**
`读 = resource.open("file", "out.bin", ["read"])`
**读出 = resource.read(读, 内存, 0, 3)**
`读出 = resource.read(读, 内存, 0, 3)`
**关闭读 = resource.close(读)**
`关闭读 = resource.close(读)`
*[写入.value.count, 关闭写.ok, 读出.value.count, 读出.value.eof, 内存[0], 内存[1], 内存[2], 关闭读.ok]*
`return [写入.value.count, 关闭写.ok, 读出.value.count, 读出.value.eof, 内存[0], 内存[1], 内存[2], 关闭读.ok]`
"#;
#[test]
@@ -152,39 +171,43 @@ fn project_file_stream_writes_on_close_then_reads_back() {
assert_eq!(std::fs::read(dir.path().join("out.bin")).unwrap(), b"ABC");
}
const COPY: &str = r#"## main
const COPY: &str = r#"## @exec main
**输入 = resource.open("file", "data.txt", ["read"])**
`输入 = resource.open("file", "data.txt", ["read"])`
**输出 = resource.open("file", "out.bin", ["write"])**
`输出 = resource.open("file", "out.bin", ["write"])`
**内存 = memory.new(1, 1)**
`内存 = memory.new(1, 1)`
**总数 = 0**
`总数 = 0`
**继续 = true**
`继续 = true`
- `继续`
`while 继续`
**块 = resource.read(输入, 内存, 0, 16)**
`块 = resource.read(输入, 内存, 0, 16)`
1. `块.ok`
`if 块.ok`
> resource.write(输出, 内存, 0, 块.value.count)
`resource.write(输出, 内存, 0, 块.value.count)`
**总数 = 总数 + 块.value.count**
`总数 = 总数 + 块.value.count`
**继续 = not 块.value.eof**
`继续 = not 块.value.eof`
2. `true`
`else`
**继续 = false**
`继续 = false`
**关闭读 = resource.close(输入)**
`end`
**关闭写 = resource.close(输出)**
`end`
*[总数, 关闭读.ok, 关闭写.ok]*
`关闭读 = resource.close(输入)`
`关闭写 = resource.close(输出)`
`return [总数, 关闭读.ok, 关闭写.ok]`
"#;
#[test]
@@ -238,27 +261,27 @@ fn tcp_stream_round_trips_through_mqmd_with_a_network_grant() {
let dir = TempDir::new("tcp");
let program = format!(
r#"## main
r#"## @exec main
**内存 = memory.new(1, 1)**
`内存 = memory.new(1, 1)`
**`内存[0] = 112`**
`内存[0] = 112`
**`内存[1] = 105`**
`内存[1] = 105`
**`内存[2] = 110`**
`内存[2] = 110`
**`内存[3] = 103`**
`内存[3] = 103`
**流 = resource.open("tcp", "{address}", ["read", "write"])**
`流 = resource.open("tcp", "{address}", ["read", "write"])`
**写入 = resource.write(流, 内存, 0, 4)**
`写入 = resource.write(流, 内存, 0, 4)`
**读出 = resource.read(流, 内存, 0, 4)**
`读出 = resource.read(流, 内存, 0, 4)`
**关闭 = resource.close(流)**
`关闭 = resource.close(流)`
*[写入.value.count, 读出.value.count, 内存[0], 内存[1], 内存[2], 内存[3], 关闭.ok]*
`return [写入.value.count, 读出.value.count, 内存[0], 内存[1], 内存[2], 内存[3], 关闭.ok]`
"#
);
write_file(&dir.path().join("index.mq.md"), &program);
@@ -295,3 +318,135 @@ fn file_stream_requires_a_read_grant() {
assert_eq!(report["success"], false);
assert_eq!(report["error"]["code"], "permission.denied");
}
const STDIN_READER: &str = r#"## @exec main
`输入 = resource.open("stdio", "stdin", ["read"])`
`内存 = memory.new(1, 1)`
`第一 = resource.read(输入, 内存, 0, 2)`
`第二 = resource.read(输入, 内存, 2, 8)`
`第三 = resource.read(输入, 内存, 0, 1)`
`return [第一.value.count, 第一.value.eof, 第二.value.count, 第二.value.eof, 第三.value.eof, 内存[0], 内存[1], 内存[2], 内存[3]]`
"#;
#[test]
fn piped_stdin_is_chunked_and_reaches_eof() {
let dir = TempDir::new("stdin");
write_file(&dir.path().join("index.mq.md"), STDIN_READER);
let output = cli_with_stdin(dir.path(), &["run", "index.mq.md", "--json"], b"abcd");
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
assert_eq!(report["success"], true);
let values = report["value"]["items"].as_array().expect("list result");
assert_eq!(values[0]["decimal"], "2");
assert_eq!(values[1]["value"], false);
assert_eq!(values[2]["decimal"], "2");
assert_eq!(values[3]["value"], true);
assert_eq!(values[4]["value"], true);
for (index, expected) in ["97", "98", "99", "100"].into_iter().enumerate() {
assert_eq!(values[5 + index]["decimal"], expected);
}
}
const STDERR_WRITER: &str = r#"## @exec main
`错误 = resource.open("stdio", "stderr", ["write"])`
`输出 = resource.open("stdio", "stdout", ["write"])`
`内存 = memory.new(1, 1)`
`内存[0] = 101`
`内存[1] = 114`
`内存[2] = 114`
`写错误 = resource.write(错误, 内存, 0, 3)`
`内存[0] = 111`
`内存[1] = 117`
`内存[2] = 116`
`写输出 = resource.write(输出, 内存, 0, 3)`
`关闭错误 = resource.close(错误)`
`关闭输出 = resource.close(输出)`
`return [写错误.value.count, 写输出.value.count, 关闭错误.ok, 关闭输出.ok]`
"#;
#[test]
fn program_stderr_is_separate_from_stdout_and_diagnostics() {
let dir = TempDir::new("stderr");
write_file(&dir.path().join("index.mq.md"), STDERR_WRITER);
let output = cli(dir.path(), &["run", "index.mq.md", "--json"]);
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
assert_eq!(report["success"], true);
assert_eq!(report["stdout"], "out");
assert_eq!(report["stderr"], "err");
let values = report["value"]["items"].as_array().expect("list result");
assert_eq!(values[0]["decimal"], "3");
assert_eq!(values[1]["decimal"], "3");
assert_eq!(values[2]["value"], true);
assert_eq!(values[3]["value"], true);
}
const CONSOLE_READER: &str = r#"---
import:
console: "std:io/console"
---
## @exec main
`第一 = console.读取行()`
`第二 = console.读取行()`
`第三 = console.读取行()`
`输出 = console.打印(第一.value + "|" + 第二.value)`
`return [第一.ok, 第二.ok, 第三.ok, 第三.code, 输出.ok]`
"#;
#[test]
fn stdlib_console_reads_lines_and_reports_eof() {
let dir = TempDir::new("console");
write_file(&dir.path().join("index.mq.md"), CONSOLE_READER);
let output = cli_with_stdin(dir.path(), &["run", "index.mq.md", "--json"], b"one\ntwo\n");
assert_eq!(
output.status.code(),
Some(0),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let report: serde_json::Value = serde_json::from_slice(&output.stdout).expect("json output");
assert_eq!(report["success"], true);
assert_eq!(report["stdout"], "one|two\n");
let values = report["value"]["items"].as_array().expect("list result");
assert_eq!(values[0]["value"], true);
assert_eq!(values[1]["value"], true);
assert_eq!(values[2]["value"], false);
assert_eq!(values[3]["value"], "io.eof");
assert_eq!(values[4]["value"], true);
}
+4 -4
View File
@@ -71,11 +71,11 @@ import:
io: "std:io"
---
## main
## @exec main
> io.写值(["bundle", 1, 2.5])
`io.写值(["bundle", 1, 2.5])`
*None*
`return None`
"#;
#[test]
@@ -154,7 +154,7 @@ fn tampered_bundle_is_rejected_before_module_loading() {
let dir = TempDir::new("tamper");
let bundle = dir.path().join("stdlib");
copy_stdlib(&stdlib_dir(), &bundle);
std::fs::write(bundle.join("io.mq.md"), "## tampered\n").unwrap();
std::fs::write(bundle.join("io.mq.md"), "## @knowledge tampered\n").unwrap();
write_project(dir.path(), IMPORT_IO);
let output = run(dir.path(), &bundle, &["run", "index.mq.md"]);
assert!(!output.status.success());
+3
View File
@@ -37,4 +37,7 @@ fn stdlib_http_parses_chunked_responses_and_builds_requests() {
items[4]["value"],
"POST /echo HTTP/1.1\r\nHost: example.test\r\nContent-Length: 2\r\n\r\nok"
);
assert_eq!(items[5]["decimal"], "200");
assert_eq!(items[6]["value"], "你好");
assert_eq!(items[7]["value"], "POST / HTTP/1.1\r\n\r\nAB");
}
+511
View File
@@ -0,0 +1,511 @@
//! Product-level tests for the whole bundled `marqdo-stdlib` tree.
//!
//! Every public module in `stdlib/README.md` has a `.mq.md` self-test under
//! `stdlib/tests/`, and this lane runs the whole directory through the real
//! user import path (`std:` + the verified bundle), not through a conformance
//! fixture or a Rust shortcut.
use std::io::Write;
use std::path::{Path, PathBuf};
use std::process::{Command, Stdio};
use std::sync::{Mutex, MutexGuard};
fn repo_root() -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR"))
.parent()
.unwrap()
.parent()
.unwrap()
.to_path_buf()
}
static FS_LOCK: Mutex<()> = Mutex::new(());
fn lock_fs(name: &str) -> Option<MutexGuard<'static, ()>> {
if name == "fs.mq.md" {
Some(FS_LOCK.lock().unwrap_or_else(|error| error.into_inner()))
} else {
None
}
}
fn stdlib_dir() -> PathBuf {
repo_root().join("stdlib")
}
struct TestFile {
name: String,
stdin: &'static str,
stdout: &'static str,
args: Vec<String>,
env: Vec<(String, String)>,
}
fn product_tests() -> Vec<TestFile> {
let dir = stdlib_dir().join("tests");
let mut files: Vec<TestFile> = std::fs::read_dir(&dir)
.expect("read stdlib tests")
.map(|entry| {
let path = entry.unwrap().path();
if !path.is_file() {
return TestFile {
name: String::new(),
stdin: "",
stdout: "",
args: Vec::new(),
env: Vec::new(),
};
}
let name = path.file_name().unwrap().to_string_lossy().to_string();
let (stdin, stdout) = match name.as_str() {
"console.mq.md" => ("hello\n", ""),
"io.mq.md" => ("", "stdlib self-test\n"),
"stream.mq.md" => ("", "stream\n"),
_ => ("", ""),
};
let text_args = |values: &[&str]| -> Vec<String> {
values.iter().map(|value| value.to_string()).collect()
};
let (args, env) = if name == "fs.mq.md" {
(
text_args(&[
"--allow-read",
"out",
"--allow-read",
"out/**",
"--allow-write",
"out",
"--allow-write",
"out/**",
]),
Vec::new(),
)
} else if name == "env.mq.md" {
(
text_args(&[
"--allow-env",
"MARQDO_TEST_ENV",
"--allow-env",
"MARQDO_MISSING_ENV",
]),
vec![("MARQDO_TEST_ENV".to_string(), "env-value".to_string())],
)
} else if name == "process.mq.md" {
// Use the test binary itself as the granted program: it exists
// on every native platform, unlike the old `/bin/echo` fixture.
let program = env!("CARGO_BIN_EXE_marqdo").to_string();
(
vec![
"--allow-env".to_string(),
"MARQDO_TEST_PROCESS_PROGRAM".to_string(),
"--allow-process".to_string(),
program.clone(),
],
vec![("MARQDO_TEST_PROCESS_PROGRAM".to_string(), program)],
)
} else if name == "random.mq.md" {
(text_args(&["--allow-random"]), Vec::new())
} else {
(Vec::new(), Vec::new())
};
TestFile {
name,
stdin,
stdout,
args,
env,
}
})
.filter(|file| !file.name.is_empty())
.collect();
files.sort_by(|left, right| left.name.cmp(&right.name));
files
}
fn run_product_test(file: &TestFile) -> serde_json::Value {
if file.name == "fs.mq.md" {
let _ = std::fs::remove_dir_all(stdlib_dir().join("tests/out"));
}
let mut child = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.args(["run"])
.arg(stdlib_dir().join("tests").join(&file.name))
.args(file.args.iter())
.args(["--json"])
.current_dir(stdlib_dir())
.env("MARQDO_STDLIB_DIR", stdlib_dir())
.envs(
file.env
.iter()
.map(|(key, value)| (key.as_str(), value.as_str())),
)
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.expect("spawn marqdo");
child
.stdin
.as_mut()
.unwrap()
.write_all(file.stdin.as_bytes())
.unwrap();
let output = child.wait_with_output().expect("wait marqdo");
assert_eq!(
output.status.code(),
Some(0),
"{} failed: {}",
file.name,
String::from_utf8_lossy(&output.stderr)
);
serde_json::from_slice(&output.stdout).expect("json report")
}
#[test]
fn every_public_stdlib_module_has_a_product_test() {
let names: Vec<String> = product_tests().into_iter().map(|file| file.name).collect();
assert_eq!(
names,
vec![
"clock.mq.md",
"compress.mq.md",
"console.mq.md",
"csv.mq.md",
"datetime.mq.md",
"demo.mq.md",
"env.mq.md",
"format.mq.md",
"fs.mq.md",
"http.mq.md",
"io.mq.md",
"json.mq.md",
"path.mq.md",
"process.mq.md",
"random.mq.md",
"regex.mq.md",
"stream.mq.md",
"table.mq.md",
"text.mq.md",
"utf8.mq.md",
],
"a public stdlib module is missing a product test"
);
}
#[test]
fn the_whole_bundled_library_runs_through_std_imports() {
for file in product_tests() {
let _guard = lock_fs(&file.name);
let report = run_product_test(&file);
assert_eq!(report["success"], true, "{}", file.name);
assert_eq!(
report["stdout"], file.stdout,
"{} wrote unexpected stdout",
file.name
);
}
}
#[test]
fn fs_test_exercises_the_file_operation_surface() {
let _guard = lock_fs("fs.mq.md");
let report = run_product_test(
product_tests()
.iter()
.find(|file| file.name == "fs.mq.md")
.expect("fs test"),
);
let items = report["value"]["items"].as_array().expect("list");
assert_eq!(items[0]["value"], true);
assert_eq!(items[1]["decimal"], "7");
assert_eq!(items[2]["value"], true);
assert_eq!(items[3]["value"], "file");
assert_eq!(items[4]["decimal"], "7");
assert_eq!(items[5]["items"][0]["value"], "fs-a.txt");
assert_eq!(items[6]["value"], true);
assert_eq!(items[7]["value"], true);
assert_eq!(items[8]["value"], true);
assert_eq!(items[9]["value"], "你好\n");
assert_eq!(items[10]["decimal"], "7");
assert_eq!(items[11]["value"], "你好\n世界\n");
assert_eq!(items[12]["items"][0]["value"], "fs-b.txt");
assert_eq!(items[13]["value"], true);
assert_eq!(items[14]["value"], true);
assert!(
!stdlib_dir().join("tests/out").exists(),
"the fs test did not remove its directory"
);
}
#[test]
fn json_test_parses_and_formats_nested_values() {
let report = run_product_test(
product_tests()
.iter()
.find(|file| file.name == "json.mq.md")
.expect("json test"),
);
let items = report["value"]["items"].as_array().expect("list");
assert_eq!(items[0]["value"], true);
assert_eq!(items[1]["decimal"], "1");
assert_eq!(items[2]["value"], "说\"好");
assert_eq!(
items[3]["value"],
"[1, 2.5, -300.0, 0.4, {\"a\": 1.0}, \"x\"]"
);
assert_eq!(items[4]["value"], false);
assert_eq!(items[5]["value"], "json.invalid_array");
assert_eq!(items[6]["value"], "json.trailing");
assert_eq!(items[7]["value"], false);
assert_eq!(items[8]["value"], "json.depth");
assert_eq!(items[9]["value"], true);
assert_eq!(items[10]["items"][2]["items"][0]["decimal"], "3");
}
#[test]
fn csv_test_parses_and_formats_quoted_rows() {
let report = run_product_test(
product_tests()
.iter()
.find(|file| file.name == "csv.mq.md")
.expect("csv test"),
);
let items = report["value"]["items"].as_array().expect("list");
let row = items[0]["items"].as_array().expect("row");
assert_eq!(row.len(), 3);
assert_eq!(row[0]["value"], "甲");
assert_eq!(row[1]["value"], "乙,丙");
assert_eq!(row[2]["value"], "说\"好");
assert_eq!(items[1]["value"], "甲,\"乙,丙\",\"说\"\"好\"");
let empty = items[2]["items"].as_array().expect("empty row");
assert_eq!(empty.len(), 1);
assert_eq!(empty[0]["value"], "");
assert_eq!(items[3]["value"], "csv.unterminated_quote");
let quoted = items[4]["items"].as_array().expect("quoted row");
assert_eq!(quoted[0]["value"], "a");
assert_eq!(quoted[1]["value"], "b");
assert_eq!(items[5]["value"], true);
let rows = items[6]["items"].as_array().expect("rows");
assert_eq!(rows.len(), 2);
assert_eq!(rows[1]["items"][0]["value"], "说\"好");
assert_eq!(items[7]["value"], "甲,\"乙,丙\"\n\"说\"\"好\",2");
}
#[test]
fn path_test_joins_and_normalizes_project_paths() {
let report = run_product_test(
product_tests()
.iter()
.find(|file| file.name == "path.mq.md")
.expect("path test"),
);
let items = report["value"]["items"].as_array().expect("list");
assert_eq!(items[0]["value"], "a/b/c");
assert_eq!(items[1]["value"], "c.txt");
assert_eq!(items[2]["value"], "a/b");
assert_eq!(items[3]["value"], "txt");
assert_eq!(items[4]["value"], "a/c");
assert_eq!(items[5]["value"], "/a/c");
assert_eq!(items[6]["value"], "../a/b");
assert_eq!(items[7]["value"], true);
assert_eq!(items[8]["value"], false);
}
#[test]
fn regex_test_matches_anchors_classes_and_search() {
let report = run_product_test(
product_tests()
.iter()
.find(|file| file.name == "regex.mq.md")
.expect("regex test"),
);
let items = report["value"]["items"].as_array().expect("list");
assert_eq!(items[0]["value"], true);
assert_eq!(items[1]["value"], false);
assert_eq!(items[2]["value"], true);
assert_eq!(items[3]["value"], true);
assert_eq!(items[4]["value"], true);
assert_eq!(items[5]["value"], true);
assert_eq!(items[6]["value"], true);
assert_eq!(items[7]["value"], true);
assert_eq!(items[8]["value"], true);
assert_eq!(items[9]["value"], false);
assert_eq!(items[10]["value"], "regex.invalid");
assert_eq!(items[11]["value"], "regex.unsupported");
assert_eq!(items[12]["value"], true);
assert_eq!(items[13]["value"], true);
let groups = items[14]["items"].as_array().expect("groups");
assert_eq!(groups[0]["value"], "aab");
assert_eq!(groups[1]["value"], "aa");
assert_eq!(groups[2]["value"], "b");
}
#[test]
fn datetime_test_formats_and_parses_utc_instants() {
let report = run_product_test(
product_tests()
.iter()
.find(|file| file.name == "datetime.mq.md")
.expect("datetime test"),
);
let items = report["value"]["items"].as_array().expect("list");
assert_eq!(items[0]["value"], "1970-01-01T00:00:00Z");
assert_eq!(items[1]["value"], "2023-11-14T22:13:20.123Z");
assert_eq!(items[2]["decimal"], "0");
assert_eq!(items[3]["decimal"], "1795248000123");
assert_eq!(items[4]["value"], false);
assert_eq!(items[5]["value"], false);
assert_eq!(items[6]["value"], "1969-12-31T23:59:59.999Z");
}
#[test]
fn clock_test_reports_project_mode_unix_millis() {
let report = run_product_test(
product_tests()
.iter()
.find(|file| file.name == "clock.mq.md")
.expect("clock test"),
);
let items = report["value"]["items"].as_array().expect("list");
assert_eq!(items[0]["value"], true);
assert_eq!(items[1]["value"], "Int");
assert_eq!(items[2]["value"], true);
}
#[test]
fn env_test_reads_only_granted_names() {
let report = run_product_test(
product_tests()
.iter()
.find(|file| file.name == "env.mq.md")
.expect("env test"),
);
let items = report["value"]["items"].as_array().expect("list");
assert_eq!(items[0]["value"], true);
assert_eq!(items[1]["value"], true);
assert_eq!(items[2]["value"], true);
assert_eq!(items[3]["value"], "env-value");
assert_eq!(items[4]["value"], false);
assert_eq!(items[5]["value"], "env.not_found");
}
#[test]
fn process_test_runs_an_explicitly_granted_program() {
let report = run_product_test(
product_tests()
.iter()
.find(|file| file.name == "process.mq.md")
.expect("process test"),
);
let items = report["value"]["items"].as_array().expect("list");
assert_eq!(items[0]["value"], true);
assert_eq!(items[1]["decimal"], "0");
let expected = Command::new(env!("CARGO_BIN_EXE_marqdo"))
.arg("--version")
.output()
.expect("run marqdo --version");
let expected_stdout = String::from_utf8_lossy(&expected.stdout);
assert_eq!(
items[2]["value"].as_str().expect("process stdout"),
expected_stdout.as_ref(),
"process stdout must match the granted program"
);
assert_eq!(items[3]["value"], "");
}
#[test]
fn random_test_reads_bounded_entropy() {
let report = run_product_test(
product_tests()
.iter()
.find(|file| file.name == "random.mq.md")
.expect("random test"),
);
let items = report["value"]["items"].as_array().expect("list");
assert_eq!(items[0]["value"], true);
assert_eq!(items[1]["decimal"], "16");
assert_eq!(items[2]["value"], true);
assert_eq!(items[3]["value"], true);
assert_eq!(items[4]["value"], true);
}
#[test]
fn compress_test_round_trips_gzip_text() {
let report = run_product_test(
product_tests()
.iter()
.find(|file| file.name == "compress.mq.md")
.expect("compress test"),
);
let items = report["value"]["items"].as_array().expect("list");
assert_eq!(items[0]["value"], true);
assert_eq!(items[1]["value"], true);
assert_eq!(items[2]["value"], true);
assert_eq!(items[3]["value"], "hello 世界");
}
#[test]
fn format_test_pins_value_semantics_and_handle_observation() {
let report = run_product_test(
product_tests()
.iter()
.find(|file| file.name == "format.mq.md")
.expect("format test"),
);
let items = report["value"]["items"].as_array().expect("list");
assert_eq!(items[0]["decimal"], "3");
assert_eq!(items[1]["value"], "[\"文本\", 2, 1.5]");
assert_eq!(items[2]["value"], "{a: 1}");
assert_eq!(items[3]["value"], "[\"a\", 2]");
assert_eq!(items[4]["value"], "None");
}
#[test]
fn text_and_utf8_tests_agree_on_the_byte_encoding() {
let text = run_product_test(
product_tests()
.iter()
.find(|file| file.name == "text.mq.md")
.expect("text test"),
);
let utf8 = run_product_test(
product_tests()
.iter()
.find(|file| file.name == "utf8.mq.md")
.expect("utf8 test"),
);
assert_eq!(text["value"]["items"][0]["decimal"], "2");
assert_eq!(text["value"]["items"][1]["decimal"], "195");
assert_eq!(text["value"]["items"][2]["decimal"], "169");
assert_eq!(utf8["value"]["items"][0]["decimal"], "2");
assert_eq!(utf8["value"]["items"][1]["value"], "é");
assert_eq!(utf8["value"]["items"][2]["value"], false);
}
#[test]
fn stream_and_console_use_the_resource_endpoints() {
let stream = run_product_test(
product_tests()
.iter()
.find(|file| file.name == "stream.mq.md")
.expect("stream test"),
);
assert_eq!(stream["value"]["items"][0]["value"], true);
assert_eq!(stream["value"]["items"][1]["decimal"], "7");
let console = run_product_test(
product_tests()
.iter()
.find(|file| file.name == "console.mq.md")
.expect("console test"),
);
assert_eq!(console["value"]["items"][0]["value"], true);
assert_eq!(console["value"]["items"][1]["value"], "hello");
assert!(
console["trace"]
.as_array()
.unwrap()
.iter()
.any(|event| event["op"] == "resource.read" || event["op"] == "resource.open"),
"console test did not go through a resource endpoint"
);
}
+18
View File
@@ -41,6 +41,18 @@ pub enum TokenKind {
Or,
/// `not`
Not,
/// `if` (statement keyword, not an identifier)
If,
/// `else` (statement keyword)
Else,
/// `while` (statement keyword)
While,
/// `end` (statement keyword)
End,
/// `return` (statement keyword)
Return,
/// `params` (signature keyword)
Params,
/// `(`
LParen,
/// `)`
@@ -307,6 +319,12 @@ impl<'a> Lexer<'a> {
"and" => TokenKind::And,
"or" => TokenKind::Or,
"not" => TokenKind::Not,
"if" => TokenKind::If,
"else" => TokenKind::Else,
"while" => TokenKind::While,
"end" => TokenKind::End,
"return" => TokenKind::Return,
"params" => TokenKind::Params,
_ => TokenKind::Ident(text.to_owned()),
}
}
+26 -14
View File
@@ -1,10 +1,12 @@
//! Strict, independent expression and statement parser for executable Markdown
//! payloads.
//! Strict, independent payload statement and expression parser for executable
//! Markdown payloads.
//!
//! This crate is the only grammar for Marqdo executable payloads. It knows
//! nothing about Markdown and nothing about the runtime: the Markdown AST
//! decides *where* a payload exists, this crate decides *what the payload
//! means* (see `spec/expression.ebnf` and `spec/evaluation.md`).
//! means* (see `spec/expression.ebnf` and `spec/evaluation.md`). The
//! [`statement`] module owns the `if`/`else`/`while`/`end`/`return`/`params`
//! statement keywords; the parser here owns their expressions.
//!
//! # Public API
//!
@@ -21,10 +23,15 @@
//! The contractual surface is:
//!
//! * [`parse_expression`] — parse a complete expression payload.
//! * [`parse_statement`] — parse a complete statement payload (assignment or
//! expression).
//! * [`parse_arguments`] — parse a leading `( ... )` argument list, used for
//! Markdown-link calls such as `> [计算面积](./math.mq.md#计算面积)(r=3)`.
//! * [`parse_payload_statement`]/[`PayloadStatement`] — parse one complete
//! payload under the v2 statement grammar (control keyword, `params`, or an
//! ordinary assignment/expression).
//! * [`parse_statement`] — parse a complete assignment/expression payload
//! (without control keywords).
//! * [`parse_arguments`] — parse a leading `( ... )` argument list. There is no
//! Markdown-link call in the current language; this remains the shared
//! argument-list grammar used by callers that must parse arguments without a
//! callee (for example builtin signature validation tests).
//! * the [`ast`] module (re-exported at the crate root).
//! * [`ParseError`] — structured `expr.parse` diagnostics with byte spans.
//!
@@ -33,11 +40,14 @@
//!
//! # Grammar summary
//!
//! The grammar is frozen in `spec/expression.ebnf`. In order of decreasing
//! precedence: prefix `not` / `-` / `+`, `**` (right associative), postfix
//! calls `.name` / `[index]`, `* / // %`, `+ -`, a single non-associative
//! comparison, `and`, `or`. Assignment is only available at statement start;
//! its target is a bare identifier or an index expression (`m[8] = 42`).
//! The grammar is frozen in `spec/expression.ebnf`. One payload is exactly one
//! statement: `if <expr>`, `else`, `while <expr>`, `end`, `return <expr>`,
//! `params(...)`, an assignment or a bare expression. Within an expression, in
//! order of decreasing precedence: prefix `not` / `-` / `+`, `**` (right
//! associative), postfix calls `.name` / `[index]`, `* / // %`, `+ -`, a single
//! non-associative comparison, `and`, `or`. Assignment is only available at
//! statement start; its target is a bare identifier or an index expression
//! (`m[8] = 42`).
//!
//! Notable lexical rules:
//!
@@ -79,6 +89,7 @@
pub mod ast;
pub mod lexer;
mod parser;
pub mod statement;
use std::fmt;
@@ -89,6 +100,7 @@ pub use ast::{
};
pub use marqdo_span::Span;
pub use parser::MAX_NESTING_DEPTH;
pub use statement::{parse_payload_statement, PayloadStatement};
/// Stable diagnostic code for every expression parse failure.
pub const EXPR_PARSE: &str = "expr.parse";
@@ -154,8 +166,8 @@ pub fn parse_statement(source: &str) -> Result<Statement, ParseError> {
/// positional arguments precede named arguments, named arguments are unique,
/// and a single trailing comma is allowed. Trailing tokens are an error.
///
/// This exists so the frontend can parse Markdown-link calls of the form
/// `> [计算面积](./math.mq.md#计算面积)(r=3)` without re-lexing the payload.
/// This is the shared argument-list grammar (call postfixes and callers that
/// validate arguments without a callee expression).
pub fn parse_arguments(source: &str) -> Result<Vec<Arg>, ParseError> {
parser::parse_arguments(source)
}
+33
View File
@@ -707,6 +707,12 @@ fn is_keyword(kind: &TokenKind) -> bool {
| TokenKind::And
| TokenKind::Or
| TokenKind::Not
| TokenKind::If
| TokenKind::Else
| TokenKind::While
| TokenKind::End
| TokenKind::Return
| TokenKind::Params
)
}
@@ -740,6 +746,12 @@ fn describe_token(kind: &TokenKind) -> String {
TokenKind::And => "`and`".to_owned(),
TokenKind::Or => "`or`".to_owned(),
TokenKind::Not => "`not`".to_owned(),
TokenKind::If => "`if`".to_owned(),
TokenKind::Else => "`else`".to_owned(),
TokenKind::While => "`while`".to_owned(),
TokenKind::End => "`end`".to_owned(),
TokenKind::Return => "`return`".to_owned(),
TokenKind::Params => "`params`".to_owned(),
TokenKind::LParen => "`(`".to_owned(),
TokenKind::RParen => "`)`".to_owned(),
TokenKind::LBracket => "`[`".to_owned(),
@@ -1365,8 +1377,29 @@ mod tests {
format!("{}1{}", "(".repeat(depth), ")".repeat(depth))
}
/// Run a deeply recursive parser test on an explicit stack.
///
/// The test harness gives each test thread a small stack on some native
/// platforms (macOS arm64 aborted with a stack overflow at the 128-level
/// nesting check). Production parsing happens on the main thread; pinning
/// this test to 8 MiB keeps the contract test meaningful without depending
/// on the harness default.
fn with_parser_stack<F: FnOnce() + Send + 'static>(body: F) {
std::thread::Builder::new()
.name("parser-nesting-depth".to_string())
.stack_size(8 * 1024 * 1024)
.spawn(body)
.expect("spawn parser nesting thread")
.join()
.expect("parser nesting thread panicked");
}
#[test]
fn nesting_depth_limit_is_enforced() {
with_parser_stack(nesting_depth_limit_is_enforced_body);
}
fn nesting_depth_limit_is_enforced_body() {
assert!(parse_expression(&nested_parens(MAX_NESTING_DEPTH)).is_ok());
let err = parse_expression(&nested_parens(MAX_NESTING_DEPTH + 1)).unwrap_err();
assert_eq!(err.code, "expr.parse");
+229
View File
@@ -0,0 +1,229 @@
//! The v2 payload statement grammar (RFC-LANG-0002).
//!
//! One Markdown carrier holds exactly one payload. This module is the only
//! place that decides whether that payload is a control statement
//! (`if`/`else`/`while`/`end`/`return`), a `params(...)` declaration, or an
//! ordinary assignment/expression. The frontend still owns pairing the control
//! words into blocks; this module owns their lexical recognition and the
//! expression grammar they embed.
//!
//! Keyword matching is by complete token: `ifdef(x)` and `returning()` are
//! ordinary calls, while a bare `if` used as an identifier is a reserved-word
//! error. `if`/`while`/`return` require whitespace before their expression and
//! the whole payload must be consumed.
use crate::ast::{Expr, Statement};
use crate::lexer::{self, TokenKind};
use crate::{parse_expression, parse_statement, ParseError};
use marqdo_span::Span;
/// A payload parsed by the v2 statement grammar.
#[derive(Debug, Clone, PartialEq)]
pub enum PayloadStatement {
/// `if <expr>`
If(Expr),
/// `else` (no expression)
Else,
/// `while <expr>`
While(Expr),
/// `end` (no expression)
End,
/// `return <expr>`
Return(Expr),
/// `params(...)`; the parameter list itself is validated by the frontend
/// signature parser, which owns IR parameter types and defaults.
Params,
/// An ordinary assignment or expression statement.
Ordinary(Statement),
}
/// Parse one complete executable payload under the v2 statement grammar.
///
/// The input is the concatenated carrier text, already trimmed or not; leading
/// and trailing whitespace is ignored. Expressions keep byte spans relative to
/// the original payload, including the keyword prefix, so diagnostics and IR
/// source maps point inside the file after the frontend rewrites them.
pub fn parse_payload_statement(source: &str) -> Result<PayloadStatement, ParseError> {
let trimmed = source.trim();
if trimmed.is_empty() {
return Err(ParseError::new(Span::empty(0), "empty executable payload"));
}
// `params(...)` is a declaration whose interior is validated by
// `signature::parse`; recognize the prefix before tokenizing so malformed
// parameter text (for example `params(1x: Int)`) is reported once by the
// signature parser instead of as a lexer failure.
if trimmed.starts_with("params(") {
return Ok(PayloadStatement::Params);
}
if trimmed == "params" || trimmed.starts_with("params ") || trimmed.starts_with("params\t") {
return Err(ParseError::new(
Span::new(0, "params".len()),
"malformed params declaration; write `params(name: Type, ...)`",
));
}
let tokens = lexer::tokenize(trimmed)?;
let first = tokens.first().expect("tokenize always returns EOF");
match first.kind {
TokenKind::If => {
let expr = control_expression(trimmed, first.span, "if")?;
Ok(PayloadStatement::If(expr))
}
TokenKind::While => {
let expr = control_expression(trimmed, first.span, "while")?;
Ok(PayloadStatement::While(expr))
}
TokenKind::Return => {
let expr = control_expression(trimmed, first.span, "return")?;
Ok(PayloadStatement::Return(expr))
}
TokenKind::Else => {
require_only_keyword(&tokens, "else")?;
Ok(PayloadStatement::Else)
}
TokenKind::End => {
require_only_keyword(&tokens, "end")?;
Ok(PayloadStatement::End)
}
TokenKind::Params => Err(ParseError::new(
first.span,
"malformed params declaration; write `params(name: Type, ...)`",
)),
_ => parse_statement(trimmed).map(PayloadStatement::Ordinary),
}
}
fn control_expression(source: &str, keyword: Span, name: &str) -> Result<Expr, ParseError> {
let rest = &source[keyword.end..];
if rest.is_empty() || !rest.starts_with(char::is_whitespace) {
return Err(ParseError::new(
keyword,
format!("`{name}` must be followed by whitespace and an expression"),
));
}
// Pad the expression so its AST spans stay payload-relative.
let padded = format!("{}{}", " ".repeat(keyword.end), rest);
parse_expression(&padded)
}
fn require_only_keyword(tokens: &[crate::lexer::Token], name: &str) -> Result<(), ParseError> {
let mut non_eof = tokens
.iter()
.filter(|token| !matches!(token.kind, TokenKind::Eof));
let _keyword = non_eof.next();
if let Some(extra) = non_eof.next() {
return Err(ParseError::new(
extra.span,
format!("`{name}` takes no expression"),
));
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn expr_of(source: &str) -> Expr {
match parse_payload_statement(source).expect("valid payload") {
PayloadStatement::If(expr)
| PayloadStatement::While(expr)
| PayloadStatement::Return(expr) => expr,
other => panic!("unexpected payload: {other:?}"),
}
}
#[test]
fn parses_control_statements_with_payload_relative_spans() {
for keyword in ["if", "while", "return"] {
let source = format!("{keyword} x > 0");
let expr = expr_of(&source);
assert_eq!(expr.span().start, keyword.len() + 1);
}
}
#[test]
fn else_and_end_are_exact_payloads() {
assert_eq!(
parse_payload_statement("else").unwrap(),
PayloadStatement::Else
);
assert_eq!(
parse_payload_statement("end").unwrap(),
PayloadStatement::End
);
assert!(parse_payload_statement("else x").is_err());
assert!(parse_payload_statement("end if").is_err());
}
#[test]
fn keywords_require_whitespace_and_an_expression() {
for source in ["if", "while", "return", "if(x)", "while(1)", "return(x)"] {
assert!(
parse_payload_statement(source).is_err(),
"{source} must be rejected"
);
}
}
#[test]
fn control_expressions_must_consume_the_payload() {
assert!(parse_payload_statement("if x > 0:").is_err());
assert!(parse_payload_statement("if x y").is_err());
}
#[test]
fn keyword_prefixes_are_ordinary_identifiers() {
for source in ["ifdef(x)", "returning(x)", "endpoint(x)", "while_count(x)"] {
match parse_payload_statement(source).unwrap() {
PayloadStatement::Ordinary(statement) => {
assert!(matches!(statement.kind, crate::StatementKind::Expr(_)));
}
other => panic!("{source} parsed as {other:?}"),
}
}
}
#[test]
fn bare_keywords_are_reserved_in_expressions() {
for source in ["x = if", "x = return", "[else]", "{while: 1}"] {
assert!(parse_payload_statement(source).is_err(), "{source}");
}
}
#[test]
fn params_are_recognized_before_signature_validation() {
assert_eq!(
parse_payload_statement("params(n: Int)").unwrap(),
PayloadStatement::Params
);
// Interior validation belongs to `signature::parse`; the statement
// grammar only needs to recognize the declaration payload.
assert_eq!(
parse_payload_statement("params(1x: Int)").unwrap(),
PayloadStatement::Params
);
assert!(parse_payload_statement("params").is_err());
assert!(parse_payload_statement("params (x)").is_err());
assert!(parse_payload_statement("params\t(x)").is_err());
}
#[test]
fn ordinary_statements_stay_ordinary() {
for source in ["x = 1", "print(x)", "m[0] = 1", "x = 1; return x"] {
let parsed = parse_payload_statement(source);
if source.contains(';') {
assert!(parsed.is_err(), "{source}");
} else {
assert!(matches!(parsed.unwrap(), PayloadStatement::Ordinary(_)));
}
}
}
#[test]
fn expressions_cannot_use_statement_keywords_as_identifiers() {
assert!(crate::parse_expression("if").is_err());
assert!(crate::parse_expression("return").is_err());
}
}
+54 -3
View File
@@ -864,6 +864,27 @@ fn execute(request: &RunRequest, registry: &KernelRegistry) -> Result<RawRun, Fo
})
}
/// Expand a PATH lookup name to the platform's executable file names.
///
/// Unix uses the name verbatim. Windows applies `PATHEXT` (with a conservative
/// default) because a PATH directory contains `python3.exe`, never `python3`.
fn executable_candidates(name: &str) -> Vec<std::ffi::OsString> {
// Only the Windows branch appends suffixes, so `mut` is unused on Unix.
#[allow(unused_mut)]
let mut names = vec![std::ffi::OsString::from(name)];
#[cfg(windows)]
{
if Path::new(name).extension().is_none() {
let extensions =
std::env::var("PATHEXT").unwrap_or_else(|_| ".COM;.EXE;.BAT;.CMD".to_string());
for extension in extensions.split(';').filter(|value| !value.is_empty()) {
names.push(std::ffi::OsString::from(format!("{name}{extension}")));
}
}
}
names
}
fn program_candidates(language: &str) -> &'static [&'static str] {
match language {
"python" => &["python3", "python"],
@@ -973,9 +994,14 @@ fn resolve_program(descriptor: &KernelDescriptor) -> Result<PathBuf, ForeignErro
let candidates = program_candidates(&descriptor.provider_id);
for name in candidates {
for directory in std::env::split_paths(&path_var) {
let candidate = directory.join(name);
if is_executable_file(&candidate) {
return Ok(candidate);
// Windows PATH entries contain `python.exe`, not the extension-less
// `python3` token `Command::new` accepts. Scanning PATH manually
// must apply PATHEXT or a valid interpreter is reported missing.
for executable in executable_candidates(name) {
let candidate = directory.join(executable);
if is_executable_file(&candidate) {
return Ok(candidate);
}
}
}
}
@@ -1032,6 +1058,14 @@ struct TempArtifacts {
impl TempArtifacts {
fn create(extension: &str) -> Result<Self, ForeignError> {
// `fs::canonicalize` on Windows returns a verbatim `\\?\C:\...`
// path. Node's module loader uses the working directory/script path
// with `lstat` and rejects that form (`node:fs` ENOENT/UNKNOWN), even
// though Rust can open it. Use the native temp path on Windows and
// keep canonicalization for Unix symlink safety.
#[cfg(windows)]
let base = std::env::temp_dir();
#[cfg(not(windows))]
let base = fs::canonicalize(std::env::temp_dir()).unwrap_or_else(|_| std::env::temp_dir());
let mut last_error: Option<io::Error> = None;
for _ in 0..16 {
@@ -1394,6 +1428,23 @@ mod tests {
assert_eq!(normalize_language(""), None);
}
#[test]
fn path_lookup_expands_windows_executable_suffixes() {
let names = executable_candidates("python3");
#[cfg(windows)]
{
assert!(names.iter().any(|name| {
name.to_string_lossy()
.to_ascii_lowercase()
.ends_with(".exe")
}));
}
#[cfg(not(windows))]
{
assert_eq!(names.len(), 1);
}
}
#[test]
fn request_envelope_is_versioned() {
let mut request = RunRequest::new("python", "print(1)");
+163 -45
View File
@@ -57,25 +57,45 @@ pub fn analyze_source(module_id: &str, source: &str) -> AnalysisResult {
)
.with_hint("front matter must be valid YAML between `---` delimiters"),
),
Ok(_) => match frontmatter.imports() {
Ok(mapping) => {
for (alias, target) in mapping {
imports.push(ImportDecl {
alias,
target,
span: frontmatter.span,
});
Ok(_) => {
for key in ["marqdo_semantics", "syntax"] {
if frontmatter.has_key(key) {
diagnostics.push(
Diagnostic::error(
codes::SEM_UNSUPPORTED_VERSION_DIRECTIVE,
Some(frontmatter.yaml_span),
format!(
"`{key}` is not a syntax selector; there is one current syntax"
),
)
.with_hint(
"remove the directive and migrate the source to RFC-LANG-0002 syntax",
),
);
}
}
Err(error) => diagnostics.push(
Diagnostic::error(
codes::FRONTMATTER_INVALID,
Some(frontmatter.span),
format!("invalid import declaration: {error}"),
)
.with_hint("imports are an explicit mapping: `import: { alias: \"@module\" }`"),
),
},
match frontmatter.imports() {
Ok(mapping) => {
for (alias, target) in mapping {
imports.push(ImportDecl {
alias,
target,
span: frontmatter.span,
});
}
}
Err(error) => diagnostics.push(
Diagnostic::error(
codes::FRONTMATTER_INVALID,
Some(frontmatter.span),
format!("invalid import declaration: {error}"),
)
.with_hint(
"imports are an explicit mapping: `import: { alias: \"@module\" }`",
),
),
}
}
}
}
@@ -85,15 +105,14 @@ pub fn analyze_source(module_id: &str, source: &str) -> AnalysisResult {
let mut functions: Vec<Function> = Vec::new();
for scope in &units.functions {
let blocks: Vec<&marqdo_markdown::MdNode> = document.root.children
[scope.body_start..scope.body_end]
.iter()
.collect();
let signature = signature::detect(&document, &blocks);
let explicit = signature.as_ref().map(|(_, sig)| sig);
let mut mapped = map::map_function(&document, &units, &imports, scope);
let mut mapped = map::map_function(&document, &units, scope);
diagnostics.append(&mut mapped.diagnostics);
// The mapper already proved that this payload is the function's first
// direct executable payload; a Knowledge section can never supply one.
let signature = mapped.params.as_ref().map(|declaration| {
signature::from_payload(&declaration.text, declaration.span, &declaration.segments)
});
let explicit = signature.as_ref();
let mut params: Vec<Param> = Vec::new();
let mut kind = SignatureKind::Inferred;
@@ -104,7 +123,12 @@ pub fn analyze_source(module_id: &str, source: &str) -> AnalysisResult {
}
let inference = infer::infer(&mapped.stmts, explicit, &import_names);
diagnostics.extend(inference.diagnostics);
if kind == SignatureKind::Inferred {
if kind == SignatureKind::Inferred && !inference.params.is_empty() {
diagnostics.push(Diagnostic::warning(
codes::SEM_INFERRED_SIGNATURE,
Some(scope.heading_span),
"function signature was inferred; declare it with `params(...)`",
));
params = inference
.params
.into_iter()
@@ -139,6 +163,19 @@ pub fn analyze_source(module_id: &str, source: &str) -> AnalysisResult {
};
diagnostics.extend(calls::validate_module(&module));
// One stable order for every consumer: source position first, then the
// stable code, as required by `spec/mapping.md` §8. Diagnostics without a
// precise span sort last.
diagnostics.sort_by(|left, right| {
let left_key = (left.span.map(|span| span.start), left.code.as_str());
let right_key = (right.span.map(|span| span.start), right.code.as_str());
left_key
.0
.unwrap_or(usize::MAX)
.cmp(&right_key.0.unwrap_or(usize::MAX))
.then_with(|| left_key.1.cmp(right_key.1))
});
AnalysisResult {
module,
document,
@@ -157,19 +194,25 @@ mod tests {
#[test]
fn maps_recursive_factorial() {
let source = r#"# 数学
let source = r#"# @knowledge 数学
## 阶乘
## @exec 阶乘
1. `n <= 1`
`if n <= 1`
*1*
`return 1`
2. `n > 1`
`else`
**结果 = n * 阶乘(n=n-1)**
`if n > 1`
*结果*
`结果 = n * 阶乘(n=n-1)`
`return 结果`
`end`
`end`
"#;
let result = analyze(source);
assert!(
@@ -186,17 +229,19 @@ mod tests {
#[test]
fn maps_countdown_loop() {
let source = r#"## 倒计时
let source = r#"## @exec 倒计时
**n = 3**
`n = 3`
- `n > 0`
`while n > 0`
> 打印(n)
`打印(n)`
**n = n - 1**
`n = n - 1`
*n*
`end`
`return n`
"#;
let result = analyze(source);
assert!(!result.has_errors(), "{:#?}", result.diagnostics);
@@ -205,24 +250,96 @@ mod tests {
assert!(matches!(function.body[1], marqdo_ir::Stmt::Loop { .. }));
}
#[test]
fn diagnostics_are_sorted_by_position_then_code() {
// The duplicate-heading error is found while indexing (later in the
// file); the payload syntax error is found while mapping (earlier).
// The public order must be by source position, not pipeline stage.
let source = "## @exec f\n\n`x =`\n\n## @exec f\n";
let result = analyze(source);
let codes: Vec<&str> = result
.diagnostics
.iter()
.map(|diagnostic| diagnostic.code.as_str())
.collect();
assert_eq!(
codes,
vec!["sem.syntax", "sem.duplicate_unit_name"],
"{:#?}",
result.diagnostics
);
}
#[test]
fn knowledge_section_cannot_declare_a_signature() {
// The `params(...)` payload sits in a Knowledge section, so it is inert
// documentation and must not become the function's signature.
let source = "## @exec f\n\n### @knowledge 说明\n\n`params(x: Int)`\n\n### @exec 计算\n\n`return 1`\n";
let result = analyze(source);
assert!(
!result.has_errors(),
"unexpected diagnostics: {:#?}",
result.diagnostics
);
let function = result.module.function("f").expect("f");
assert_eq!(function.signature, SignatureKind::Inferred);
assert!(function.params.is_empty());
}
#[test]
fn exec_section_params_after_prose_is_the_signature() {
// Prose and a Knowledge subsection may precede the declaration, so a
// later Exec section can still declare the first executable payload.
let source = "## @exec f\n\n说明文字。\n\n### @knowledge 原理\n\n`params(ignored: Int)`\n\n### @exec 计算\n\n`params(x: Int)`\n\n`return x`\n";
let result = analyze(source);
assert!(
!result.has_errors(),
"unexpected diagnostics: {:#?}",
result.diagnostics
);
let function = result.module.function("f").expect("f");
assert_eq!(function.signature, SignatureKind::Explicit);
assert_eq!(function.params.len(), 1);
assert_eq!(function.params[0].name, "x");
}
#[test]
fn unmatched_end_points_at_the_end_payload() {
let source = "## @exec f\n\n`x = 1`\n\n`end`\n";
let result = analyze(source);
let diagnostic = result
.diagnostics
.iter()
.find(|diagnostic| diagnostic.code == "sem.unmatched_end")
.expect("sem.unmatched_end");
let span = diagnostic.span.expect("span");
assert_eq!(
&source[span.start..span.end],
"`end`",
"the diagnostic must point at the stray end payload"
);
}
#[test]
fn invalid_strong_payload_is_a_hard_error() {
let result = analyze("## f\n\n**a *b* c**\n");
let result = analyze("## @exec f\n\n**a *b* c**\n");
assert!(result.has_errors());
assert!(result
.diagnostics
.iter()
.any(|d| d.code == codes::MARKDOWN_INVALID_CONSTRUCT));
.any(|d| d.code == "sem.payload_nested"));
}
#[test]
fn invalid_list_header_is_a_hard_error() {
let result = analyze("## f\n\n1. 没有条件\n");
// RFC-LANG-0002: a plain list is inert Markdown; the v2 negative is an
// unclosed control frame inside the owning Exec H2.
let result = analyze("## @exec f\n\n`if 1`\n");
assert!(result.has_errors());
assert!(result
.diagnostics
.iter()
.any(|d| d.code == codes::MARKDOWN_INVALID_CONSTRUCT));
.any(|d| d.code == "sem.unclosed_control"));
}
/// Strip source spans so that pure narrative edits (which shift byte
@@ -252,8 +369,9 @@ mod tests {
#[test]
fn prose_changes_do_not_change_ir() {
let base = "## f\n\n说明文字。\n\n**x = 1**\n\n更多说明。\n\n*x*\n";
let edited = "## f\n\n另一段完全不同的叙述。\n\n**x = 1**\n\n继续叙述。\n\n*x*\n";
let base = "## @exec f\n\n说明文字。\n\n`x = 1`\n\n更多说明。\n\n`return x`\n";
let edited =
"## @exec f\n\n另一段完全不同的叙述。\n\n`x = 1`\n\n继续叙述。\n\n`return x`\n";
let a = analyze(base);
let b = analyze(edited);
let a_ir = serde_json::to_value(&a.module.functions).unwrap();
-5
View File
@@ -151,11 +151,6 @@ impl<'a> CallChecker<'a> {
Stmt::Eval { expr: value, .. } | Stmt::Return { value, .. } => {
self.expr(value);
}
Stmt::Call { call, .. } => {
for Arg { value, .. } in &call.args {
self.expr(value);
}
}
Stmt::Branch { arms, .. } => {
for arm in arms {
self.expr(&arm.condition);
-6
View File
@@ -202,12 +202,6 @@ impl<'a> Analyzer<'a> {
self.read_expr(expr, &state);
state
}
Stmt::Call { call, .. } => {
for Arg { value, .. } in &call.args {
self.read_expr(value, &state);
}
state
}
Stmt::Return { value, .. } => {
self.read_expr(value, &state);
state
+332 -461
View File
@@ -6,16 +6,32 @@
//! diagnostics and never fall back to prose.
use marqdo_expression::{
parse_arguments, parse_expression, parse_statement, Arg, AssignTarget, Expr, StatementKind,
parse_payload_statement, AssignTarget, Expr, PayloadStatement as Payload, StatementKind,
};
use marqdo_ir::{CallSite, CallTarget, ImportDecl, PlaceTarget, Stmt};
use marqdo_markdown::{normalize_name, MarkdownDocument, MdKind, MdNode};
use marqdo_ir::{PlaceTarget, Stmt};
use marqdo_markdown::{MarkdownDocument, MdKind, MdNode};
use marqdo_span::Span;
use marqdo_value::{codes, Diagnostic};
use crate::payload::PayloadSegments;
use crate::units::{FunctionScope, UnitIndex};
/// The `params(...)` payload accepted for one function.
///
/// Only the first function-wide direct payload qualifies; the signature parser
/// validates the interior. Keeping the carrier text and span map here means the
/// declaration is recognized by the same traversal and mode rules that decide
/// what executes, so a Knowledge section can never declare a signature.
#[derive(Debug, Clone)]
pub struct ParamsDeclaration {
/// Payload text, e.g. `params(n: Int)`.
pub text: String,
/// Span of the carrier node.
pub span: Span,
/// Payload-offset → file-span map for default expressions.
pub segments: PayloadSegments,
}
/// A mapped function body plus diagnostics.
#[derive(Debug, Clone, Default)]
pub struct MappedBody {
@@ -23,36 +39,84 @@ pub struct MappedBody {
pub stmts: Vec<Stmt>,
/// Diagnostics produced while mapping.
pub diagnostics: Vec<Diagnostic>,
/// The accepted `params(...)` declaration, if the function has one.
pub params: Option<ParamsDeclaration>,
}
/// Map one function scope to executable statements.
pub fn map_function(
doc: &MarkdownDocument,
index: &UnitIndex,
imports: &[ImportDecl],
scope: &FunctionScope,
) -> MappedBody {
let mut mapper = Mapper {
index,
imports,
diagnostics: Vec::new(),
params: None,
saw_params: None,
executed: false,
};
let blocks = &doc.root.children[scope.body_start..scope.body_end];
let stmts = mapper.blocks(blocks);
let mut stmts = Vec::new();
let mut segment = Vec::new();
let mut mode = crate::units::SemanticMode::Exec;
for (node_index, block) in doc.root.children.iter().enumerate() {
if node_index < scope.body_start || node_index >= scope.body_end {
continue;
}
if let MdKind::Heading { .. } = block.kind {
if mode == crate::units::SemanticMode::Exec {
stmts.extend(mapper.blocks(&segment));
}
segment.clear();
if let Some(heading) = index.headings.iter().find(|h| h.node_index == node_index) {
mode = heading.mode;
}
continue;
}
if mode == crate::units::SemanticMode::Exec {
segment.push(block.clone());
}
}
if mode == crate::units::SemanticMode::Exec {
stmts.extend(mapper.blocks(&segment));
}
MappedBody {
stmts,
diagnostics: mapper.diagnostics,
params: mapper.params,
}
}
struct Mapper<'a> {
index: &'a UnitIndex,
#[allow(dead_code)]
imports: &'a [ImportDecl],
struct Mapper {
diagnostics: Vec<Diagnostic>,
/// Accepted `params(...)` payload for the function being mapped.
params: Option<ParamsDeclaration>,
/// First direct `params(...)` payload of the **function** (not just the
/// current heading segment), used to reject duplicate declarations.
saw_params: Option<Span>,
/// Whether any executable statement has already been lowered for this
/// function. `params(...)` is legal only before that point, even when the
/// earlier statements live in an earlier heading segment.
executed: bool,
}
impl<'a> Mapper<'a> {
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Stop {
End,
Else,
Eof,
}
struct RawPayload {
payload: Payload,
text: String,
span: Span,
segments: PayloadSegments,
/// True only for a carrier in the heading's direct body, not inside a
/// list or block quote (`params` is legal only there).
direct: bool,
}
impl Mapper {
fn error(&mut self, code: &str, span: Option<Span>, message: impl Into<String>) {
self.diagnostics
.push(Diagnostic::error(code, span, message));
@@ -70,471 +134,278 @@ impl<'a> Mapper<'a> {
}
fn blocks(&mut self, blocks: &[MdNode]) -> Vec<Stmt> {
let mut stmts = Vec::new();
let mut raw = Vec::new();
for block in blocks {
stmts.extend(self.block(block));
self.extract(block, &mut raw, true);
}
let mut position = 0;
let (stmts, stop) = self.sequence(&raw, &mut position, false, false);
if stop != Stop::Eof {
// `sequence` leaves `position` on the stray `end` payload, so the
// diagnostic points at the keyword instead of the previous
// statement. A control frame may never close across a heading.
self.error(
codes::SEM_UNMATCHED_END,
raw.get(position).map(|node| node.span),
"`end` has no open `if`/`while` in this heading segment",
);
}
stmts
}
fn block(&mut self, block: &MdNode) -> Vec<Stmt> {
match &block.kind {
MdKind::Paragraph => self.paragraph(block),
MdKind::List { ordered, .. } => self.list(block, *ordered),
MdKind::BlockQuote => self.quote(block),
// Level>=3 headings and inert document values stay out of the IR.
_ => Vec::new(),
}
}
fn paragraph(&mut self, paragraph: &MdNode) -> Vec<Stmt> {
let mut stmts = Vec::new();
for child in &paragraph.children {
match &child.kind {
MdKind::Strong => {
if let Some(stmt) = self.exec_statement(child) {
stmts.push(stmt);
fn extract(&mut self, node: &MdNode, out: &mut Vec<RawPayload>, direct: bool) {
match &node.kind {
MdKind::Paragraph => {
for child in &node.children {
self.extract(child, out, direct);
}
}
MdKind::BlockQuote | MdKind::Item { .. } | MdKind::List { .. } => {
for child in &node.children {
self.extract(child, out, false);
}
}
MdKind::Strong | MdKind::Emph => {
let (text, segments) = match PayloadSegments::of(node) {
Ok(result) => result,
Err(error) => {
self.error(
codes::SEM_PAYLOAD_NESTED,
Some(error.span),
format!("invalid payload carrier: {}", error.reason),
);
return;
}
}
MdKind::Emph => {
if let Some(stmt) = self.return_statement(child) {
stmts.push(stmt);
}
}
// Lazy/inert inline structures: never recurse into them.
MdKind::Link { .. }
| MdKind::Image { .. }
| MdKind::Strikethrough
| MdKind::Code { .. }
| MdKind::Math { .. }
| MdKind::Text
| MdKind::SoftBreak
| MdKind::LineBreak
| MdKind::HtmlInline
| MdKind::FootnoteReference { .. } => {}
_ => {}
};
self.push(text, node.span, segments, out, direct);
}
MdKind::Code { ticks } => {
let literal = node.literal.as_deref().unwrap_or_default();
let inner = Span::new(
node.span.start.saturating_add(*ticks),
node.span.end.saturating_sub(*ticks),
);
self.push(
literal.to_string(),
node.span,
PayloadSegments::single(literal, inner),
out,
direct,
);
}
MdKind::Heading { .. } => {}
_ => {}
}
stmts
}
fn exec_statement(&mut self, node: &MdNode) -> Option<Stmt> {
let (payload, segments) = match PayloadSegments::of(node) {
Ok(result) => result,
Err(error) => {
self.error_with_hint(
codes::MARKDOWN_INVALID_CONSTRUCT,
Some(error.span),
error.reason,
"keep the executable payload as plain text or a single code span",
);
return None;
}
};
if payload.trim().is_empty() {
fn push(
&mut self,
text: String,
span: Span,
segments: PayloadSegments,
out: &mut Vec<RawPayload>,
direct: bool,
) {
if text.trim().is_empty() {
self.error(
codes::MARKDOWN_INVALID_CONSTRUCT,
Some(node.span),
"execution statement has an empty payload",
codes::SEM_EMPTY_PAYLOAD,
Some(span),
"executable payload is empty",
);
return None;
return;
}
match parse_statement(&payload) {
Ok(mut statement) => {
segments.rewrite_statement(&mut statement);
Some(match statement.kind {
StatementKind::Assign { target, value } => Stmt::Assign {
target: match target {
AssignTarget::Local { name, name_span } => {
PlaceTarget::Local { name, name_span }
}
AssignTarget::Index {
base,
index,
span: target_span,
} => PlaceTarget::Index {
base: *base,
index: *index,
span: target_span,
},
},
value,
span: node.span,
},
StatementKind::Expr(expr) => Stmt::Eval {
expr,
span: node.span,
},
})
if let Some(mut payload) = self.parse_payload(&text, &segments) {
if let Payload::If(expr) | Payload::While(expr) | Payload::Return(expr) = &mut payload {
segments.rewrite_expr(expr);
}
Err(error) => {
self.error_with_hint(
codes::EXPR_PARSE,
Some(segments.map_span(error.span)),
format!("invalid execution statement: {}", error.message),
"statements are `name = expression` or an expression; put the payload in a code span when Markdown emphasis conflicts",
);
None
}
}
}
fn return_statement(&mut self, node: &MdNode) -> Option<Stmt> {
let (payload, segments) = match PayloadSegments::of(node) {
Ok(result) => result,
Err(error) => {
self.error_with_hint(
codes::MARKDOWN_INVALID_CONSTRUCT,
Some(error.span),
error.reason,
"a return payload must be plain text or a single code span",
);
return None;
}
};
if payload.trim().is_empty() {
self.error(
codes::MARKDOWN_INVALID_CONSTRUCT,
Some(node.span),
"return statement has an empty payload",
);
return None;
}
match parse_expression(&payload) {
Ok(mut expr) => {
segments.rewrite_expr(&mut expr);
Some(Stmt::Return {
value: expr,
span: node.span,
})
}
Err(error) => {
self.error_with_hint(
codes::EXPR_PARSE,
Some(segments.map_span(error.span)),
format!("invalid return expression: {}", error.message),
"`*表达式*` returns the value of the expression",
);
None
}
}
}
fn list(&mut self, list: &MdNode, ordered: bool) -> Vec<Stmt> {
if list.children.is_empty() {
self.error(
codes::MARKDOWN_INVALID_CONSTRUCT,
Some(list.span),
"list in an executable position has no items",
);
return Vec::new();
}
let mut arms = Vec::new();
let mut loops = Vec::new();
for item in &list.children {
let Some((condition, condition_span, body_blocks)) = self.condition_of(item) else {
continue;
};
let body = self.blocks(body_blocks);
if ordered {
arms.push(marqdo_ir::BranchArm {
condition,
condition_span,
body,
span: item.span,
});
} else {
loops.push(Stmt::Loop {
condition,
condition_span,
body,
span: item.span,
});
}
}
if ordered {
if arms.is_empty() {
return Vec::new();
}
vec![Stmt::Branch {
arms,
span: list.span,
}]
} else {
loops
}
}
/// Extract the condition header of a list item and its remaining blocks.
fn condition_of<'b>(&mut self, item: &'b MdNode) -> Option<(Expr, Span, &'b [MdNode])> {
let Some(first) = item.children.first() else {
self.error_with_hint(
codes::MARKDOWN_INVALID_CONSTRUCT,
Some(item.span),
"list item in an executable position must start with a condition code span",
"write the condition first: `` `n > 0` ``",
);
return None;
};
if !matches!(first.kind, MdKind::Paragraph) {
self.error_with_hint(
codes::MARKDOWN_INVALID_CONSTRUCT,
Some(first.span),
"list item must start with a paragraph containing the condition code span",
"put the condition on the first line of the item",
);
return None;
}
let Some(condition_node) = first.children.first() else {
self.error(
codes::MARKDOWN_INVALID_CONSTRUCT,
Some(first.span),
"list item paragraph is empty; a condition code span is required",
);
return None;
};
if !matches!(condition_node.kind, MdKind::Code { .. }) {
self.error_with_hint(
codes::MARKDOWN_INVALID_CONSTRUCT,
Some(condition_node.span),
"list item must start with a condition inside an inline code span",
"the condition is a `Code` span: `` `n > 0` ``; other text is description",
);
return None;
}
// The remainder of the condition paragraph is description only. The
// semantic mapper does not descend into it, so an executable `Strong`
// or `Emph` there would otherwise be silently swallowed. Reject it as
// a hard shape error instead of guessing the author's intent.
for tail in first.children.iter().skip(1) {
if matches!(tail.kind, MdKind::Strong | MdKind::Emph) {
self.error_with_hint(
codes::MARKDOWN_INVALID_CONSTRUCT,
Some(tail.span),
format!(
"`{}` inside a list-item condition paragraph is not executed and cannot be silently ignored",
tail.kind.tag()
),
"put executable statements in their own paragraph after the condition line",
);
return None;
}
}
let literal = condition_node.literal.clone().unwrap_or_default();
// `condition_node.span` includes the backtick delimiters. The payload
// map must start at the first literal byte so parse diagnostics and
// expression spans point at the expression, not at a delimiter.
let ticks = match condition_node.kind {
MdKind::Code { ticks } => ticks,
_ => 0,
};
let literal_span = Span::new(
condition_node.span.start.saturating_add(ticks),
condition_node.span.end.saturating_sub(ticks),
);
let segments = PayloadSegments::single(&literal, literal_span);
match parse_expression(&literal) {
Ok(mut expr) => {
segments.rewrite_expr(&mut expr);
Some((expr, condition_node.span, &item.children[1..]))
}
Err(error) => {
self.error_with_hint(
codes::EXPR_PARSE,
Some(segments.map_span(error.span)),
format!("invalid condition expression: {}", error.message),
"conditions are Boolean expressions, e.g. `n > 0`",
);
None
}
}
}
fn quote(&mut self, quote: &MdNode) -> Vec<Stmt> {
if quote.children.len() != 1 || quote.children[0].kind != MdKind::Paragraph {
self.error_with_hint(
codes::MARKDOWN_INVALID_CONSTRUCT,
Some(quote.span),
"a block quote invocation must contain exactly one paragraph",
"use one block quote per call; prose belongs in normal paragraphs",
);
return Vec::new();
}
let paragraph = &quote.children[0];
self.call_paragraph(quote, paragraph)
}
fn call_paragraph(&mut self, quote: &MdNode, paragraph: &MdNode) -> Vec<Stmt> {
let first = paragraph.children.first();
let is_link_form = first
.map(|node| matches!(node.kind, MdKind::Link { .. }))
.unwrap_or(false);
if is_link_form {
return self.link_call(quote, paragraph);
}
let (payload, segments) = match PayloadSegments::of(paragraph) {
Ok(result) => result,
Err(error) => {
self.error_with_hint(
codes::MARKDOWN_INVALID_CONSTRUCT,
Some(error.span),
error.reason,
"a block quote call payload must be plain text or one code span",
);
return Vec::new();
}
};
if payload.trim().is_empty() {
self.error(
codes::MARKDOWN_INVALID_CONSTRUCT,
Some(quote.span),
"block quote invocation has an empty payload",
);
return Vec::new();
}
match parse_expression(&payload) {
Ok(mut expr) => {
segments.rewrite_expr(&mut expr);
if !matches!(expr, Expr::Call { .. }) {
self.error_with_hint(
codes::MARKDOWN_INVALID_CONSTRUCT,
Some(segments.map_span(expr_span(&expr))),
"a block quote must be a complete function call",
"write `> 函数(参数)`; use a `**strong**` statement to bind a result",
);
return Vec::new();
}
vec![Stmt::Eval {
expr,
span: quote.span,
}]
}
Err(error) => {
self.error_with_hint(
codes::EXPR_PARSE,
Some(segments.map_span(error.span)),
format!("invalid call expression: {}", error.message),
"block quotes call functions explicitly: `> 函数(参数)`",
);
Vec::new()
}
}
}
fn link_call(&mut self, quote: &MdNode, paragraph: &MdNode) -> Vec<Stmt> {
let link = &paragraph.children[0];
let (url, link_span) = match &link.kind {
MdKind::Link { url, .. } => (url.clone(), link.span),
_ => unreachable!("link form checked by caller"),
};
let target = self.resolve_link_target(&url, link_span);
let rest = &paragraph.children[1..];
let (suffix, segments) = match PayloadSegments::from_children(rest) {
Ok(result) => result,
Err(error) => {
self.error_with_hint(
codes::MARKDOWN_INVALID_CONSTRUCT,
Some(error.span),
error.reason,
"link call arguments must follow the link as plain text or code spans",
);
return Vec::new();
}
};
if suffix.trim().is_empty() {
self.error_with_hint(
codes::MARKDOWN_INVALID_CONSTRUCT,
Some(quote.span),
"link-form call requires an explicit argument list",
"write `> [函数](#函数)()` even for zero arguments",
);
return Vec::new();
}
let mut args: Vec<Arg> = match parse_arguments(&suffix) {
Ok(args) => args,
Err(error) => {
self.error_with_hint(
codes::EXPR_PARSE,
Some(segments.map_span(error.span)),
format!("invalid call arguments: {}", error.message),
"arguments are `name = value` or positional values",
);
return Vec::new();
}
};
segments.rewrite_args(&mut args);
let Some(target) = target else {
return Vec::new();
};
vec![Stmt::Call {
call: CallSite {
target,
args,
span: quote.span,
},
span: quote.span,
}]
}
fn resolve_link_target(&mut self, url: &str, span: Span) -> Option<CallTarget> {
if let Some(fragment) = url.strip_prefix('#') {
let unit_id = self.index.anchors.get(fragment).cloned().or_else(|| {
self.index
.units
.iter()
.find(|unit| unit.name == fragment)
.map(|unit| unit.id.clone())
out.push(RawPayload {
payload,
text,
span,
segments,
direct,
});
let Some(unit_id) = unit_id else {
self.error(
codes::REF_UNRESOLVED,
Some(span),
format!("no heading matches call target `{url}`"),
);
return None;
};
let Some(scope) = self
.index
.functions
.iter()
.find(|scope| scope.unit == unit_id)
else {
self.error(
codes::REF_UNRESOLVED,
Some(span),
format!("call target `{url}` is not a function"),
);
return None;
};
Some(CallTarget::Resolved {
module: unit_id.module.clone(),
function: scope.id.name.clone(),
})
} else if url.starts_with("http://") || url.starts_with("https://") {
self.error_with_hint(
codes::REF_UNRESOLVED,
Some(span),
"external links cannot be called",
"link targets used as calls must resolve to a Marqdo function",
);
None
} else {
self.error_with_hint(
codes::MODULE_UNRESOLVED,
Some(span),
format!("cannot resolve cross-module call target `{url}`"),
"import the module explicitly in front matter and call it through the alias",
);
None
}
}
}
fn expr_span(expr: &Expr) -> Span {
marqdo_expression::Expr::span(expr)
}
fn parse_payload(&mut self, text: &str, segments: &PayloadSegments) -> Option<Payload> {
match parse_payload_statement(text) {
Ok(payload) => Some(payload),
Err(error) => {
self.error_with_hint(
codes::SEM_SYNTAX,
Some(segments.map_span(error.span)),
format!("invalid execution payload: {}", error.message),
"one carrier is exactly one statement; control keywords need their own carrier",
);
None
}
}
}
/// Normalise a heading name (re-exported for callers that resolve link targets).
pub fn normalized_heading_name(node: &MdNode) -> Option<String> {
normalize_name(&marqdo_markdown::plain_text(node))
fn sequence(
&mut self,
nodes: &[RawPayload],
position: &mut usize,
stop_on_else: bool,
in_else_body: bool,
) -> (Vec<Stmt>, Stop) {
let mut out = Vec::new();
while *position < nodes.len() {
let raw = &nodes[*position];
match &raw.payload {
Payload::Else => {
if stop_on_else {
return (out, Stop::Else);
}
self.error(
if in_else_body {
codes::SEM_DUPLICATE_ELSE
} else {
codes::SEM_UNMATCHED_ELSE
},
Some(raw.span),
if in_else_body {
"`if` has more than one `else`"
} else {
"`else` has no open `if`"
},
);
*position += 1;
}
Payload::End => {
return (out, Stop::End);
}
Payload::If(condition) => {
self.executed = true;
let condition = condition.clone();
let span = raw.span;
*position += 1;
let (then_body, stop) = self.sequence(nodes, position, true, false);
let mut else_body = Vec::new();
let finish = match stop {
Stop::Else => {
*position += 1;
let (body, stop) = self.sequence(nodes, position, false, true);
else_body = body;
stop
}
other => other,
};
if finish != Stop::End {
self.error(
codes::SEM_UNCLOSED_CONTROL,
Some(span),
"`if` is not closed by `end`",
);
} else {
*position += 1;
}
let mut arms = vec![marqdo_ir::BranchArm {
condition,
condition_span: span,
body: then_body,
span,
}];
if !else_body.is_empty() {
arms.push(marqdo_ir::BranchArm {
condition: Expr::Bool(true),
condition_span: span,
body: else_body,
span,
});
}
out.push(Stmt::Branch { arms, span });
}
Payload::While(condition) => {
self.executed = true;
let condition = condition.clone();
let span = raw.span;
*position += 1;
let (body, stop) = self.sequence(nodes, position, false, false);
if stop != Stop::End {
self.error(
codes::SEM_UNCLOSED_CONTROL,
Some(span),
"`while` is not closed by `end`",
);
} else {
*position += 1;
}
out.push(Stmt::Loop {
condition,
condition_span: span,
body,
span,
});
}
Payload::Return(value) => {
self.executed = true;
out.push(Stmt::Return {
value: value.clone(),
span: raw.span,
});
*position += 1;
}
Payload::Params => {
if !raw.direct {
self.error(
codes::SEM_SIGNATURE_POSITION,
Some(raw.span),
"`params(...)` must be a direct payload, not inside a list or block quote",
);
} else if self.saw_params.is_some() {
self.error(
codes::SEM_SIGNATURE_POSITION,
Some(raw.span),
"`params(...)` may be declared only once per function",
);
} else if self.executed || *position != 0 {
self.error(
codes::SEM_SIGNATURE_POSITION,
Some(raw.span),
"`params(...)` must be the first executable payload of the Exec H2, before any statement in any heading segment",
);
} else {
self.saw_params = Some(raw.span);
self.params = Some(ParamsDeclaration {
text: raw.text.clone(),
span: raw.span,
segments: raw.segments.clone(),
});
}
*position += 1;
}
Payload::Ordinary(statement) => {
self.executed = true;
let mut statement = statement.clone();
raw.segments.rewrite_statement(&mut statement);
let span = raw.span;
match statement.kind {
StatementKind::Assign { target, value } => out.push(Stmt::Assign {
target: match target {
AssignTarget::Local { name, name_span } => {
PlaceTarget::Local { name, name_span }
}
AssignTarget::Index {
base,
index,
span: target_span,
} => PlaceTarget::Index {
base: *base,
index: *index,
span: target_span,
},
},
value,
span,
}),
StatementKind::Expr(expr) => out.push(Stmt::Eval { expr, span }),
}
*position += 1;
}
}
}
(out, Stop::Eof)
}
}
+26 -19
View File
@@ -1,7 +1,9 @@
//! Executable payload extraction with a payload → file span map.
//! Payload extraction with a payload → file span map.
//!
//! A payload is the concatenation of the direct `text` and `code` children of
//! an executable node. The expression parser sees only that concatenated
//! Inside an `@exec` scope, `Strong`, `Emph` and inline `Code` are equivalent
//! one-statement carriers. For `Strong`/`Emph` the payload is the
//! concatenation of their direct `text` and `code` children; a nested carrier
//! is an error. The expression/statement parser sees only the concatenated
//! string, so its spans are payload-relative; this module maps them back to the
//! original file so diagnostics point at real source.
@@ -311,40 +313,42 @@ mod extra_span_tests {
#[test]
fn condition_literal_span_excludes_backticks_for_one_and_two_ticks() {
for source in ["## f\n\n1. `n ** 2 ==`\n", "## f\n\n1. ``n ** 2 ==``\n"] {
for source in [
"## @exec f\n\n`if n ** 2 ==`\n",
"## @exec f\n\n``if n ** 2 ==``\n",
] {
let result = crate::analyze_source("index.mq.md", source);
let diagnostic = result
.diagnostics
.iter()
.find(|diagnostic| diagnostic.code == marqdo_value::codes::EXPR_PARSE)
.unwrap_or_else(|| panic!("no expr.parse diagnostic for {source:?}"));
.find(|diagnostic| diagnostic.code == marqdo_value::codes::SEM_SYNTAX)
.unwrap_or_else(|| panic!("no sem.syntax diagnostic for {source:?}"));
let span = diagnostic
.span
.expect("condition parse error has a source span");
assert_eq!(
source.as_bytes()[span.start],
b'`',
"span must stop before the closing backtick: {span:?} in {source:?}"
);
assert_eq!(
&source[span.start.saturating_sub(2)..span.start],
"==",
"span must point at the missing operand"
let condition = source.find("n ** 2 ==").expect("condition text");
let closing = source.rfind('`').expect("closing backtick");
assert!(
span.start >= condition && span.end <= closing,
"span must point inside the carrier payload, before the closing backtick: {span:?} in {source:?}"
);
}
}
#[test]
fn condition_description_rejects_executable_roles() {
// RFC-LANG-0002 no longer rejects Strong/Emph next to a code span:
// every carrier is an independent statement. The preserved intent is
// that a malformed control sequence after `if` is a hard error; it is
// repurposed as an unclosed-control negative.
for tail in ["**print(1)**", "*1*"] {
let source = format!("## f\n\n1. `n > 0` {tail}\n\n > print(1)\n");
let source = format!("## @exec f\n\n`if n > 0` {tail}\n");
let result = crate::analyze_source("index.mq.md", &source);
assert!(
result
.diagnostics
.iter()
.any(|diagnostic| diagnostic.code
== marqdo_value::codes::MARKDOWN_INVALID_CONSTRUCT),
.any(|diagnostic| diagnostic.code == "sem.unclosed_control"),
"executable tail `{tail}` was not rejected: {:#?}",
result.diagnostics
);
@@ -353,7 +357,10 @@ mod extra_span_tests {
#[test]
fn condition_description_allows_inert_inline_content() {
let source = "## f\n\n**n = 1**\n\n1. `n > 0` 普通说明、`惰性代码` 与 [链接](#f)。\n\n > print(1)\n";
// Inline code spans are payload carriers in Exec scope, so the old inert
// `惰性代码` example is now a statement; prose, a link and a carrier-free
// block quote remain inert and must not be rejected.
let source = "## @exec f\n\n`n = 1`\n\n1. `n > 0` 普通说明与 [链接](#f)。\n\n > print(1)\n\n`return n`\n";
let result = crate::analyze_source("index.mq.md", source);
assert!(
!result.has_errors(),
+1 -1
View File
@@ -121,7 +121,7 @@ mod tests {
#[test]
fn formulas_and_code_are_attributed_to_functions() {
let resources = resources_of(
"# ns\n\n## f\n\n$$A = pi r^2$$\n\n```python\nprint(1)\n```\n\n## g\n\n$$B = 1$$\n",
"# @knowledge ns\n\n## @exec f\n\n$$A = pi r^2$$\n\n```python\nprint(1)\n```\n\n## @exec g\n\n$$B = 1$$\n",
);
assert_eq!(resources.len(), 3);
let Resource::Formula { unit, latex, .. } = &resources[0] else {
+48 -59
View File
@@ -1,12 +1,13 @@
//! Optional explicit function signatures.
//!
//! `` `(r: Number, precision: Int = 2)` `` as the first paragraph of a
//! function body declares the public parameter list. It is a *declaration*:
//! it does not execute. Without it, parameters are inferred from free reads.
//! `` `params(r: Number, precision: Int = 2)` `` as the first function-wide
//! direct payload of an Exec H2 declares the public parameter list. It is a
//! *declaration*: it does not execute. Without it, parameters are inferred from
//! free reads (and an inferred signature keeps the named-arguments-only rule).
//! See RFC-LANG-0002 and `spec/mapping.md` §6.
use marqdo_expression::{parse_expression, Expr, Statement, StatementKind};
use marqdo_ir::{Param, SignatureKind};
use marqdo_markdown::{MarkdownDocument, MdKind, MdNode};
use marqdo_span::Span;
use marqdo_value::{codes, Diagnostic};
use unicode_ident::{is_xid_continue, is_xid_start};
@@ -18,7 +19,7 @@ use crate::payload::PayloadSegments;
pub struct Signature {
/// Parameters in declaration order.
pub params: Vec<Param>,
/// Span of the declaration code span.
/// Span of the declaration payload carrier.
pub span: Span,
/// Diagnostics raised while parsing the declaration.
pub diagnostics: Vec<Diagnostic>,
@@ -31,65 +32,53 @@ impl Signature {
}
}
/// Detect and parse a signature paragraph.
/// Parse the `params(...)` payload the mapper accepted for a function.
///
/// A paragraph qualifies only when it is the first block in the function body
/// (ignoring inert level>=3 section headings) and consists of exactly one
/// inline code span whose payload starts with `(`. Any other first paragraph is
/// prose and inference is used instead.
pub fn detect(_doc: &MarkdownDocument, blocks: &[&MdNode]) -> Option<(usize, Signature)> {
for (index, block) in blocks.iter().enumerate() {
match &block.kind {
MdKind::Heading { level, .. } if *level >= 3 => continue,
MdKind::Paragraph => {
let is_candidate = block.children.len() == 1
&& matches!(block.children[0].kind, MdKind::Code { .. })
&& block.children[0]
.literal
.as_deref()
.map(|text| text.trim_start().starts_with('('))
.unwrap_or(false);
if !is_candidate {
return None;
}
let code = &block.children[0];
let source = code.literal.clone().unwrap_or_default();
let mut signature = parse(&source, code.span);
// Rewrite parameter default expression spans onto the file.
let segments = PayloadSegments::of(&MdNode {
kind: MdKind::Strong,
span: code.span,
literal: None,
children: vec![code.clone()],
})
.ok()
.map(|(_, segments)| segments);
if let Some(segments) = segments {
for param in &mut signature.params {
if let Some(default) = &mut param.default {
segments.rewrite_expr(default);
}
}
}
return Some((index, signature));
}
_ => return None,
/// The mapper owns recognition: only the first function-wide direct payload of
/// an Exec H2 can be a declaration, and Knowledge sections never produce
/// payloads. This function only validates the declaration text and rewrites
/// default-value spans onto the file.
pub fn from_payload(text: &str, span: Span, segments: &PayloadSegments) -> Signature {
let mut signature = parse(text, span);
for param in &mut signature.params {
if let Some(default) = &mut param.default {
segments.rewrite_expr(default);
}
}
None
signature
}
/// Parse signature payload text (without the code span delimiters).
pub fn parse(source: &str, span: Span) -> Signature {
let mut diagnostics = Vec::new();
let trimmed = source.trim();
let inner = if let Some(rest) = trimmed.strip_prefix('(') {
let is_params = trimmed.starts_with("params(");
let syntax_code = if is_params {
codes::SEM_SYNTAX
} else {
codes::EXPR_PARSE
};
let inner = if let Some(rest) = trimmed.strip_prefix("params(") {
match rest.strip_suffix(')') {
Some(inner) => inner,
None => {
diagnostics.push(Diagnostic::error(
codes::SEM_SYNTAX,
Some(span),
"function signature must be closed by `)`",
));
rest
}
}
} else if let Some(rest) = trimmed.strip_prefix('(') {
// Plugin/ABI descriptors keep their existing parenthesised signature
// spelling; language payloads are detected only as `params(...)`.
rest.strip_suffix(')').unwrap_or(rest)
} else {
diagnostics.push(Diagnostic::error(
codes::EXPR_PARSE,
syntax_code,
Some(span),
"function signature must be parenthesised, e.g. `(r: Number)`",
"function signature must be `params(...)`, e.g. `params(r: Number)`",
));
trimmed
};
@@ -102,7 +91,7 @@ pub fn parse(source: &str, span: Span) -> Signature {
if entry.trim().is_empty() {
if !inner.trim().is_empty() {
diagnostics.push(Diagnostic::error(
codes::EXPR_PARSE,
syntax_code,
Some(span),
"empty parameter in function signature",
));
@@ -129,7 +118,7 @@ pub fn parse(source: &str, span: Span) -> Signature {
}
}
Err(message) => diagnostics.push(Diagnostic::error(
codes::EXPR_PARSE,
syntax_code,
Some(span),
format!("invalid function signature: {message}"),
)),
@@ -293,7 +282,7 @@ mod tests {
#[test]
fn parses_typed_signature_with_default() {
let signature = parse("(r: Number, precision: Int = 2)", Span::new(0, 38));
let signature = parse("params(r: Number, precision: Int = 2)", Span::new(0, 38));
assert!(!signature.has_errors(), "{:?}", signature.diagnostics);
assert_eq!(signature.params.len(), 2);
assert_eq!(signature.params[0].name, "r");
@@ -307,20 +296,20 @@ mod tests {
#[test]
fn parses_untyped_and_empty() {
let signature = parse("()", Span::new(0, 2));
let signature = parse("params()", Span::new(0, 8));
assert!(signature.params.is_empty());
assert!(!signature.has_errors());
let signature = parse("(n)", Span::new(0, 3));
let signature = parse("params(n)", Span::new(0, 9));
assert_eq!(signature.params[0].name, "n");
}
#[test]
fn rejects_duplicates_and_bad_defaults() {
let duplicate = parse("(a, a)", Span::new(0, 6));
let duplicate = parse("params(a, a)", Span::new(0, 12));
assert!(duplicate.has_errors());
let bad_default = parse("(a = f())", Span::new(0, 10));
let bad_default = parse("params(a = f())", Span::new(0, 16));
assert!(bad_default.has_errors());
let bad_name = parse("(1x)", Span::new(0, 4));
let bad_name = parse("params(1x)", Span::new(0, 10));
assert!(bad_name.has_errors());
}
+234 -113
View File
@@ -1,22 +1,24 @@
//! Heading scope index.
//! Heading scope and semantic-mode index (RFC-LANG-0002).
//!
//! Markdown headings are sibling blocks, so function scopes are derived from
//! heading levels, not from container nesting:
//! Markdown headings are sibling blocks, so scopes are derived from heading
//! levels, not from container nesting:
//!
//! * `heading(level=1)` -> namespace / object domain (blueprint 1.2, 2.1)
//! * `heading(level=2)` -> function
//! * `heading(level>=3)` -> named section inside the enclosing unit, never a
//! new callable function
//! * `heading(level=1)` -> namespace / top-level unit
//! * `heading(level=2, mode=Exec)` -> callable function
//! * `heading(level=2, mode=Knowledge)` -> inert knowledge unit
//! * `heading(level>=3)` -> section of the enclosing unit, never a callable
//! function
//!
//! The range of a heading extends to the next heading of the same or a lower
//! level (same or smaller number). Headings inside block quotes, list items or
//! code fences cannot appear as direct children of the document, hence they can
//! never create functions.
//! Each heading declares `@exec`/`@knowledge` or inherits the nearest ancestor
//! heading's mode; the document root is Knowledge. The range of a heading
//! extends to the next heading of the same or a lower level (same or smaller
//! number). Headings inside block quotes, list items or code fences cannot
//! appear as direct children of the document, hence they never create units.
use std::collections::BTreeMap;
use marqdo_ir::{FunctionId, ModuleId, UnitId, UnitInfo, UnitKind};
use marqdo_markdown::{normalize_name, plain_text, MarkdownDocument, MdKind};
use marqdo_markdown::{normalize_name, plain_text, MarkdownDocument, MdKind, MdNode};
use marqdo_span::Span;
use marqdo_value::{codes, Diagnostic};
@@ -35,6 +37,8 @@ pub struct HeadingInfo {
pub path: Vec<String>,
/// Role derived from level and position.
pub kind: UnitKind,
/// Semantic mode declared or inherited by this heading.
pub mode: SemanticMode,
}
/// A function range mapped from a level-2 heading.
@@ -93,6 +97,90 @@ impl UnitIndex {
}
}
fn has_payload(node: &MdNode) -> bool {
match &node.kind {
MdKind::Strong | MdKind::Emph | MdKind::Code { .. } => true,
MdKind::Paragraph | MdKind::List { .. } | MdKind::Item { .. } | MdKind::BlockQuote => {
node.children.iter().any(has_payload)
}
_ => false,
}
}
/// Semantic mode declared or inherited by a heading (RFC-LANG-0002).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum SemanticMode {
/// Document/knowledge region; no executable statements.
Knowledge,
/// Executable region; only an Exec H2 is callable.
Exec,
}
fn heading_declaration(
node: &MdNode,
) -> (Option<SemanticMode>, Option<String>, Option<Diagnostic>) {
let Some(first) = node.children.first() else {
return (None, None, None);
};
let MdKind::Text = first.kind else {
return (None, None, None);
};
let literal = first.literal.as_deref().unwrap_or_default();
let (mode, rest) = if let Some(rest) = literal.strip_prefix("@exec") {
if !rest.is_empty() && !rest.starts_with(char::is_whitespace) {
return (None, None, None);
}
(SemanticMode::Exec, rest)
} else if let Some(rest) = literal.strip_prefix("@knowledge") {
if !rest.is_empty() && !rest.starts_with(char::is_whitespace) {
return (None, None, None);
}
(SemanticMode::Knowledge, rest)
} else if literal.starts_with('@')
&& literal
.trim_start_matches('@')
.chars()
.next()
.is_some_and(char::is_alphabetic)
{
return (
None,
None,
Some(Diagnostic::error(
codes::SEM_INVALID_MODE_MARKER,
Some(first.span),
"unknown heading mode marker; use exactly `@exec` or `@knowledge`",
)),
);
} else {
return (None, None, None);
};
let title = rest.trim();
if title.is_empty() {
return (
Some(mode),
None,
Some(Diagnostic::error(
codes::SEM_EMPTY_UNIT_NAME,
Some(node.span),
"mode declaration has an empty heading name",
)),
);
}
if title.starts_with("@exec") || title.starts_with("@knowledge") {
return (
Some(mode),
None,
Some(Diagnostic::error(
codes::SEM_INVALID_MODE_MARKER,
Some(node.span),
"a heading may declare only one semantic mode",
)),
);
}
(Some(mode), Some(title.to_string()), None)
}
/// Build the heading/function index for a parsed document.
pub fn build_units(doc: &MarkdownDocument, module: &ModuleId) -> UnitIndex {
let mut index = UnitIndex::default();
@@ -108,23 +196,44 @@ pub fn build_units(doc: &MarkdownDocument, module: &ModuleId) -> UnitIndex {
function: None,
});
// Stack of (level, path) for ancestor headings with smaller level.
let mut stack: Vec<(u8, Vec<String>)> = Vec::new();
// Stack of (level, path, mode) for ancestors.
let mut stack: Vec<(u8, Vec<String>, SemanticMode)> = Vec::new();
let mut seen: BTreeMap<(Vec<String>, String), Span> = BTreeMap::new();
let mut current_namespace: Option<String> = None;
let mut current_function: Option<FunctionId> = None;
let mut heading_infos: Vec<HeadingInfo> = Vec::new();
for (node_index, child) in doc.root.children.iter().enumerate() {
let MdKind::Heading { level, .. } = &child.kind else {
let MdKind::Heading { level, setext } = &child.kind else {
continue;
};
let level = *level;
let raw_name = normalize_name(&plain_text(child));
let Some(name) = raw_name else {
// RFC-LANG-0002: only ATX headings may carry a mode declaration; a
// Setext heading simply inherits the nearest ancestor mode.
let (declared, declared_name, declaration_error) = if *setext {
(None, None, None)
} else {
heading_declaration(child)
};
if let Some(error) = declaration_error {
index.diagnostics.push(error);
if declared_name.is_none() {
continue;
}
}
let inherited = stack
.last()
.map(|(_, _, mode)| *mode)
.unwrap_or(SemanticMode::Knowledge);
let mode = declared.unwrap_or(inherited);
let raw_name = match declared_name {
Some(name) => name,
None => plain_text(child),
};
let Some(name) = normalize_name(&raw_name) else {
index.diagnostics.push(
Diagnostic::error(
codes::MARKDOWN_INVALID_CONSTRUCT,
codes::SEM_EMPTY_UNIT_NAME,
Some(child.span),
"heading name must not be empty",
)
@@ -133,7 +242,7 @@ pub fn build_units(doc: &MarkdownDocument, module: &ModuleId) -> UnitIndex {
continue;
};
while let Some((parent_level, _)) = stack.last() {
while let Some((parent_level, _, _)) = stack.last() {
if *parent_level >= level {
stack.pop();
} else {
@@ -142,7 +251,7 @@ pub fn build_units(doc: &MarkdownDocument, module: &ModuleId) -> UnitIndex {
}
let parent_path = stack
.last()
.map(|(_, path)| path.clone())
.map(|(_, path, _)| path.clone())
.unwrap_or_default();
let path = {
let mut path = parent_path.clone();
@@ -150,14 +259,12 @@ pub fn build_units(doc: &MarkdownDocument, module: &ModuleId) -> UnitIndex {
path
};
// Duplicate canonical names in the same scope are a hard error; never
// silently rename with -1/-2 suffixes.
let key = (parent_path.clone(), name.clone());
if let Some(previous) = seen.get(&key) {
let previous = *previous;
index.diagnostics.push(
Diagnostic::error(
codes::MARKDOWN_INVALID_CONSTRUCT,
codes::SEM_DUPLICATE_UNIT_NAME,
Some(child.span),
format!("duplicate heading name `{name}` in the same scope"),
)
@@ -170,14 +277,25 @@ pub fn build_units(doc: &MarkdownDocument, module: &ModuleId) -> UnitIndex {
let kind = match level {
1 => UnitKind::Namespace,
2 => UnitKind::Function,
2 if mode == SemanticMode::Exec => UnitKind::Function,
2 => UnitKind::Knowledge,
_ => UnitKind::Section,
};
if level == 1 {
current_namespace = Some(name.clone());
current_function = None;
}
let function = if level == 2 {
if level >= 3 && mode == SemanticMode::Exec && current_function.is_none() {
index.diagnostics.push(
Diagnostic::error(
codes::SEM_EXEC_WITHOUT_FUNCTION,
Some(child.span),
"executable section has no owning Exec H2 function",
)
.with_hint("put the section under an Exec H2, or mark it @knowledge"),
);
}
let function = if level == 2 && mode == SemanticMode::Exec {
let local = name.clone();
let qualified = match &current_namespace {
Some(namespace) => format!("{namespace}.{local}"),
@@ -189,9 +307,12 @@ pub fn build_units(doc: &MarkdownDocument, module: &ModuleId) -> UnitIndex {
};
current_function = Some(id.clone());
Some(id)
} else if level >= 3 {
} else if level >= 3 && mode == SemanticMode::Exec {
current_function.clone()
} else {
if level <= 2 {
current_function = None;
}
None
};
@@ -202,6 +323,7 @@ pub fn build_units(doc: &MarkdownDocument, module: &ModuleId) -> UnitIndex {
span: child.span,
path: path.clone(),
kind,
mode,
};
let unit_id = UnitId {
module: module.clone(),
@@ -215,12 +337,40 @@ pub fn build_units(doc: &MarkdownDocument, module: &ModuleId) -> UnitIndex {
kind,
function,
});
if level == 1 && mode == SemanticMode::Exec {
// H1 Exec: direct-body carriers before the next heading have no
// owning H2 function. A Knowledge H2 body is inert (inline code
// there is documentation), and Exec H3 sections are checked above.
let mut scan = node_index + 1;
while scan < doc.root.children.len() {
if let MdKind::Heading { .. } = doc.root.children[scan].kind {
break;
}
if has_payload(&doc.root.children[scan]) {
index.diagnostics.push(Diagnostic::error(
codes::SEM_EXEC_WITHOUT_FUNCTION,
Some(doc.root.children[scan].span),
"executable payload has no owning Exec H2 function",
));
break;
}
scan += 1;
}
}
heading_infos.push(info.clone());
stack.push((level, info.path.clone()));
stack.push((level, info.path.clone(), mode));
}
// Function body ranges and spans.
for heading in heading_infos.iter().filter(|h| h.level == 2) {
// A Knowledge H2 has no function range.
if !index
.units
.iter()
.any(|unit| unit.id.path == heading.path && unit.kind == UnitKind::Function)
{
continue;
}
let body_start = heading.node_index + 1;
let body_end = doc.root.children[body_start..]
.iter()
@@ -239,33 +389,23 @@ pub fn build_units(doc: &MarkdownDocument, module: &ModuleId) -> UnitIndex {
} else {
heading.span
};
let local_name = heading.name.clone();
let namespace = match heading.path.len() {
0 => None,
_ if heading.path.len() == 1 => None,
_ => Some(heading.path[0].clone()),
};
let namespace = namespace.filter(|_| {
index
.units
.iter()
.any(|unit| unit.kind == UnitKind::Namespace && unit.name == heading.path[0])
});
let qualified = match &namespace {
Some(namespace) => format!("{namespace}.{local_name}"),
None => local_name.clone(),
let unit = index
.units
.iter()
.find(|unit| unit.id.path == heading.path && unit.kind == UnitKind::Function)
.expect("function unit exists");
let FunctionId { name, .. } = unit.function.clone().expect("function id");
let local_name = name.rsplit('.').next().unwrap_or(&name).to_string();
let namespace = if name.contains('.') {
Some(name.rsplit_once('.').unwrap().0.to_string())
} else {
None
};
index.functions.push(FunctionScope {
id: FunctionId {
module: module.clone(),
name: qualified,
},
id: unit.function.clone().unwrap(),
local_name,
namespace,
unit: UnitId {
module: module.clone(),
path: heading.path.clone(),
},
unit: unit.id.clone(),
heading_span: heading.span,
body_start,
body_end,
@@ -273,67 +413,34 @@ pub fn build_units(doc: &MarkdownDocument, module: &ModuleId) -> UnitIndex {
});
}
// Warnings for would-be execution nodes outside every function body.
for (node_index, child) in doc.root.children.iter().enumerate() {
if index.function_containing_block(node_index).is_some() {
for heading in &heading_infos {
let Some(unit) = index.units.iter().find(|unit| unit.id.path == heading.path) else {
continue;
};
let slug = github_slug(&heading.name);
if slug.is_empty() {
continue;
}
let mut warn = |span: Span, what: &str| {
index.diagnostics.push(
Diagnostic::warning(
codes::MARKDOWN_INERT_EXECUTABLE,
Some(span),
format!("{what} outside a function body is inert document content"),
)
.with_hint("move it under a level-2 heading (`## 函数名`) to execute it"),
);
};
match &child.kind {
MdKind::BlockQuote => warn(child.span, "a block quote invocation"),
MdKind::List { .. } => warn(child.span, "a list control structure"),
MdKind::Paragraph => {
let mut found: Option<(Span, &'static str)> = None;
child.walk(&mut |node| {
if found.is_none() {
match node.kind {
MdKind::Strong => found = Some((node.span, "an execution statement")),
MdKind::Emph => found = Some((node.span, "a return statement")),
_ => {}
}
}
});
if let Some((span, what)) = found {
warn(span, what);
let mut count = index.anchors.keys().filter(|key| *key == &slug).count();
let key = if count == 0 {
slug.clone()
} else {
loop {
count += 1;
let candidate = format!("{slug}-{count}");
if !index.anchors.contains_key(&candidate) {
break candidate;
}
}
_ => {}
}
}
// Derive GitHub-style anchors for navigation and `ref("#...")`. Anchors are
// not identity: they may collide and receive -1/-2 suffixes exactly like
// GitHub's slugger, while internal ids stay canonical.
let mut slug_counts: BTreeMap<String, usize> = BTreeMap::new();
for unit in &index.units {
if unit.level == 0 {
continue;
}
let base = github_slug(&unit.name);
if base.is_empty() {
continue;
}
let count = slug_counts.entry(base.clone()).or_insert(0);
let slug = if *count == 0 {
base
} else {
format!("{base}-{count}")
};
*count += 1;
index.anchors.insert(slug, unit.id.clone());
index.anchors.insert(key, unit.id.clone());
}
index.headings = heading_infos;
index
.diagnostics
.dedup_by(|a, b| a.code == b.code && a.span == b.span && a.message == b.message);
index
}
/// GitHub-style heading slug, kept in a separate namespace from canonical
@@ -366,7 +473,7 @@ mod tests {
#[test]
fn builds_function_ranges() {
let index = index_of(
"# 命名空间\n\n## 阶乘\n\n**x = 1**\n\n### 小节\n\n正文\n\n## 下一个\n\n*y*\n",
"# @knowledge 命名空间\n\n## @exec 阶乘\n\n`x = 1`\n\n### 小节\n\n正文\n\n## @exec 下一个\n\n`return y`\n",
);
assert_eq!(index.functions.len(), 2);
assert_eq!(index.functions[0].local_name, "阶乘");
@@ -380,25 +487,40 @@ mod tests {
#[test]
fn root_level_function_without_namespace() {
let index = index_of("## main\n\n> 打印(1)\n");
let index = index_of("## @exec main\n\n`打印(1)`\n");
assert_eq!(index.functions.len(), 1);
assert_eq!(index.functions[0].id.name, "main");
assert!(index.functions[0].namespace.is_none());
assert!(index.diagnostics.is_empty());
}
#[test]
fn setext_headings_inherit_and_never_declare_a_mode() {
// `@exec f` is the level-2 heading text under a Setext underline; it is
// not a declaration, so the H2 stays Knowledge and creates no function.
let index = index_of("# @knowledge 项目\n\n@exec f\n---\n\n`return 1`\n");
assert!(index.functions.is_empty(), "{:?}", index.functions);
let heading = index
.headings
.iter()
.find(|heading| heading.level == 2)
.expect("level-2 heading");
assert_eq!(heading.mode, SemanticMode::Knowledge);
assert_eq!(heading.name, "@exec f");
}
#[test]
fn empty_heading_is_an_error() {
let index = index_of("## \n\n正文\n");
let index = index_of("## @exec\n\n正文\n");
assert!(index
.diagnostics
.iter()
.any(|d| d.code == codes::MARKDOWN_INVALID_CONSTRUCT && d.message.contains("empty")));
.any(|d| d.code == "sem.empty_unit_name" && d.message.contains("empty")));
}
#[test]
fn duplicate_function_names_are_errors() {
let index = index_of("## f\n\n## f\n");
let index = index_of("## @exec f\n\n## @exec f\n");
let errors = index
.diagnostics
.iter()
@@ -409,7 +531,7 @@ mod tests {
#[test]
fn same_name_in_different_namespaces_is_allowed() {
let index = index_of("# a\n\n## f\n\n# b\n\n## f\n");
let index = index_of("# @knowledge a\n\n## @exec f\n\n# @knowledge b\n\n## @exec f\n");
let duplicates = index
.diagnostics
.iter()
@@ -423,11 +545,10 @@ mod tests {
#[test]
fn executable_outside_function_warns_but_does_not_execute() {
let index = index_of("**x = 1**\n\n## f\n\n*y*\n");
assert!(index
.diagnostics
.iter()
.any(|d| d.code == codes::MARKDOWN_INERT_EXECUTABLE));
// RFC-LANG-0002: a Strong at the Knowledge root is inert prose with no
// diagnostic; only an Exec H2 is a callable function.
let index = index_of("**x = 1**\n\n## @exec f\n\n`return y`\n");
assert!(index.diagnostics.is_empty());
assert_eq!(index.functions.len(), 1);
}
}
+7
View File
@@ -10,7 +10,12 @@ authors.workspace = true
[dependencies]
libloading = { workspace = true }
rustls = { workspace = true }
rustls-native-certs = { workspace = true }
marqdo-plugin-abi = { workspace = true }
flate2 = { workspace = true }
getrandom = { workspace = true }
regex = { workspace = true }
glob = { workspace = true }
marqdo-span = { workspace = true }
marqdo-value = { workspace = true }
@@ -24,3 +29,5 @@ thiserror = { workspace = true }
[dev-dependencies]
tempfile = { workspace = true }
rcgen = { workspace = true }
rustls-pemfile = { workspace = true }
+70
View File
@@ -317,6 +317,12 @@ pub struct Capabilities {
pub fs_write: Vec<String>,
/// Allowed network hosts (empty = no network).
pub net_hosts: Vec<String>,
/// Allowed environment variable names (empty = no environment).
pub env: Vec<String>,
/// Allowed process program paths (empty = no process execution).
pub process_programs: Vec<String>,
/// Whether OS entropy may be read (ADR-0046).
pub random: bool,
/// May spawn processes (foreign kernels).
pub process: bool,
/// May run foreign kernels (Python/Node).
@@ -336,6 +342,9 @@ impl Capabilities {
fs_read: Vec::new(),
fs_write: Vec::new(),
net_hosts: Vec::new(),
env: Vec::new(),
process_programs: Vec::new(),
random: false,
process: false,
foreign: false,
plugins: Vec::new(),
@@ -361,6 +370,67 @@ impl Capabilities {
self
}
/// Grant read access to one environment variable name (ADR-0043).
pub fn allow_env(mut self, name: impl Into<String>) -> Self {
self.env.push(name.into());
self
}
/// Check an environment variable grant. Names are exact, not globs.
pub fn check_env(&self, name: &str, span: Option<Span>) -> Result<(), RuntimeError> {
if self.env.iter().any(|allowed| allowed == name) {
Ok(())
} else {
Err(denied(
span,
format!("environment variable `{name}` was not granted"),
format!("grant it with --allow-env {name}"),
))
}
}
/// Grant execution of one exact program path (ADR-0044).
pub fn allow_process(mut self, program: impl Into<String>) -> Self {
self.process_programs.push(program.into());
self
}
/// Check an exact process-program grant.
pub fn check_process(&self, program: &str, span: Option<Span>) -> Result<(), RuntimeError> {
if self
.process_programs
.iter()
.any(|allowed| allowed == program)
{
Ok(())
} else {
Err(denied(
span,
format!("process `{program}` was not granted"),
format!("grant it with --allow-process {program}"),
))
}
}
/// Grant OS entropy reads (ADR-0046).
pub fn allow_random(mut self) -> Self {
self.random = true;
self
}
/// Check the entropy grant.
pub fn check_random(&self, span: Option<Span>) -> Result<(), RuntimeError> {
if self.random {
Ok(())
} else {
Err(denied(
span,
"OS entropy was not granted",
"grant it with --allow-random",
))
}
}
/// Grant foreign kernel execution.
pub fn allow_foreign(mut self) -> Self {
self.foreign = true;
+466 -1
View File
@@ -4,10 +4,12 @@
//! symlink escapes outside the project root are rejected before the capability
//! check, and the capability check happens before any I/O.
use std::io::{Read, Seek, SeekFrom, Write};
use std::path::{Component, Path, PathBuf};
use marqdo_span::Span;
use marqdo_value::{ErrorCategory, RuntimeError};
use marqdo_value::{ErrorCategory, RuntimeError, Value};
use std::collections::BTreeMap;
/// Maximum size for a single file read.
pub const MAX_READ_BYTES: u64 = 8 * 1024 * 1024;
@@ -143,6 +145,391 @@ impl ProjectFs {
})
}
/// Execute a path operation on the project filesystem.
///
/// This is the host side of `resource.request`; it validates every path
/// against the project root, then returns the recoverable `Result` map.
/// Capability checks are performed by the host before this method is
/// called. Unknown operations are a programming error and stay fatal.
pub fn request(
&self,
operation: &str,
args: &[String],
span: Option<Span>,
) -> Result<Value, RuntimeError> {
match operation {
"exists" => {
let (absolute, relative) = self.resolve(one_arg(operation, args, span)?, span)?;
if !absolute.exists() {
return Ok(fs_ok(Value::Bool(false)));
}
self.ensure_inside(&absolute, &relative, span)?;
Ok(fs_ok(Value::Bool(true)))
}
"metadata" => {
let (absolute, relative) = self.resolve(one_arg(operation, args, span)?, span)?;
if !absolute.exists() {
return Ok(fs_error(
"fs.not_found",
&format!("`{relative}` does not exist"),
));
}
let absolute = self.ensure_inside(&absolute, &relative, span)?;
let metadata = std::fs::metadata(&absolute).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot stat `{relative}`: {error}"),
)
})?;
let kind = if metadata.is_file() {
"file"
} else if metadata.is_dir() {
"directory"
} else {
"other"
};
let size = if metadata.is_file() {
metadata.len()
} else {
0
};
let mut value = BTreeMap::new();
value.insert("kind".to_string(), Value::text(kind));
value.insert(
"size".to_string(),
Value::int(size.min(i64::MAX as u64) as i64),
);
Ok(fs_ok(Value::Map(value)))
}
"list" => {
let (absolute, relative) = self.resolve(one_arg(operation, args, span)?, span)?;
if !absolute.exists() {
return Ok(fs_error(
"fs.not_found",
&format!("`{relative}` does not exist"),
));
}
let absolute = self.ensure_inside(&absolute, &relative, span)?;
let metadata = std::fs::metadata(&absolute).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot stat `{relative}`: {error}"),
)
})?;
if !metadata.is_dir() {
return Ok(fs_error(
"fs.not_directory",
&format!("`{relative}` is not a directory"),
));
}
let mut names = Vec::new();
let entries = std::fs::read_dir(&absolute).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot list `{relative}`: {error}"),
)
})?;
for entry in entries {
let entry = entry.map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot list `{relative}`: {error}"),
)
})?;
names.push(entry.file_name().to_string_lossy().to_string());
}
names.sort();
Ok(fs_ok(Value::List(
names.into_iter().map(Value::text).collect(),
)))
}
"mkdir" => {
let (absolute, relative) = self.resolve(one_arg(operation, args, span)?, span)?;
self.ensure_parent_inside(&absolute, &relative, span)?;
if absolute.exists() {
return Ok(fs_error(
"fs.exists",
&format!("`{relative}` already exists"),
));
}
match std::fs::create_dir(&absolute) {
Ok(()) => Ok(fs_ok(Value::None)),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(fs_error(
"fs.not_found",
&format!("parent directory of `{relative}` does not exist"),
)),
Err(error) => Err(RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot create directory `{relative}`: {error}"),
)),
}
}
"remove" => {
let (absolute, relative) = self.resolve(one_arg(operation, args, span)?, span)?;
if !absolute.exists() {
return Ok(fs_error(
"fs.not_found",
&format!("`{relative}` does not exist"),
));
}
let absolute = self.ensure_inside(&absolute, &relative, span)?;
let metadata = std::fs::metadata(&absolute).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot stat `{relative}`: {error}"),
)
})?;
if metadata.is_dir() {
let mut entries = std::fs::read_dir(&absolute).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot list `{relative}`: {error}"),
)
})?;
if entries.next().is_some() {
return Ok(fs_error(
"fs.not_empty",
&format!("`{relative}` is not empty"),
));
}
std::fs::remove_dir(&absolute).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot remove `{relative}`: {error}"),
)
})?;
} else {
std::fs::remove_file(&absolute).map_err(|error| {
let category = if error.kind() == std::io::ErrorKind::PermissionDenied {
ErrorCategory::PermissionDenied
} else {
ErrorCategory::HostError
};
RuntimeError::new(
category,
span,
format!("cannot remove `{relative}`: {error}"),
)
})?;
}
Ok(fs_ok(Value::None))
}
"append" => {
if args.len() != 2 {
return Err(fs_arity(operation, 2, args.len(), span));
}
let (absolute, relative) = self.resolve(&args[0], span)?;
self.ensure_parent_inside(&absolute, &relative, span)?;
let bytes = args[1].as_bytes();
if bytes.len() > MAX_READ_BYTES as usize {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!(
"append to `{relative}` exceeds the {} byte limit",
MAX_READ_BYTES
),
));
}
let mut file = std::fs::OpenOptions::new()
.append(true)
.create(true)
.open(&absolute)
.map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot append to `{relative}`: {error}"),
)
})?;
file.write_all(bytes).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot append to `{relative}`: {error}"),
)
})?;
Ok(fs_ok(Value::int(bytes.len() as i64)))
}
"read_at" => {
if args.len() != 3 {
return Err(fs_arity(operation, 3, args.len(), span));
}
let (absolute, relative) = self.resolve(&args[0], span)?;
let offset = parse_offset(&args[1], span)?;
let length = parse_offset(&args[2], span)?;
if length > MAX_READ_BYTES {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
format!("read_at length exceeds the {} byte limit", MAX_READ_BYTES),
));
}
let mut file = std::fs::File::open(&absolute).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot read `{relative}`: {error}"),
)
})?;
file.seek(SeekFrom::Start(offset)).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot seek `{relative}`: {error}"),
)
})?;
let mut buffer = Vec::new();
file.take(length)
.read_to_end(&mut buffer)
.map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot read `{relative}`: {error}"),
)
})?;
let text = String::from_utf8(buffer).map_err(|_| {
RuntimeError::new(
ErrorCategory::Type,
span,
format!("`{relative}` slice is not valid UTF-8"),
)
})?;
Ok(fs_ok(Value::text(text)))
}
"walk" => {
let (absolute, relative) = self.resolve(one_arg(operation, args, span)?, span)?;
if !absolute.exists() {
return Ok(fs_error(
"fs.not_found",
&format!("`{relative}` does not exist"),
));
}
let absolute = self.ensure_inside(&absolute, &relative, span)?;
let metadata = std::fs::metadata(&absolute).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot stat `{relative}`: {error}"),
)
})?;
if !metadata.is_dir() {
return Ok(fs_error(
"fs.not_directory",
&format!("`{relative}` is not a directory"),
));
}
let mut entries = Vec::new();
walk_directory(&absolute, "", &mut entries, 0).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot walk `{relative}`: {error}"),
)
})?;
if entries.len() > 4096 {
return Err(RuntimeError::new(
ErrorCategory::ResourceExhausted,
span,
"walk exceeds the 4096 entry limit",
));
}
Ok(fs_ok(Value::List(
entries.into_iter().map(Value::text).collect(),
)))
}
"rename" => {
if args.len() != 2 {
return Err(fs_arity(operation, 2, args.len(), span));
}
let (from_absolute, from) = self.resolve(&args[0], span)?;
let (to_absolute, to) = self.resolve(&args[1], span)?;
if !from_absolute.exists() {
return Ok(fs_error(
"fs.not_found",
&format!("`{from}` does not exist"),
));
}
let from_absolute = self.ensure_inside(&from_absolute, &from, span)?;
self.ensure_parent_inside(&to_absolute, &to, span)?;
if to_absolute.exists() {
return Ok(fs_error("fs.exists", &format!("`{to}` already exists")));
}
match std::fs::rename(&from_absolute, &to_absolute) {
Ok(()) => Ok(fs_ok(Value::None)),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(fs_error(
"fs.not_found",
&format!("parent directory of `{to}` does not exist"),
)),
Err(error) => Err(RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot rename `{from}` to `{to}`: {error}"),
)),
}
}
other => Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
span,
format!("unsupported filesystem operation `{other}`"),
)),
}
}
fn ensure_inside(
&self,
absolute: &Path,
relative: &str,
span: Option<Span>,
) -> Result<PathBuf, RuntimeError> {
let canonical = absolute.canonicalize().map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot resolve `{relative}`: {error}"),
)
})?;
if !canonical.starts_with(&self.root) {
return Err(invalid_path(span, "path escapes the project root"));
}
Ok(canonical)
}
fn ensure_parent_inside(
&self,
absolute: &Path,
relative: &str,
span: Option<Span>,
) -> Result<(), RuntimeError> {
let mut ancestor = absolute.parent();
while let Some(current) = ancestor {
if current.exists() {
let canonical = current.canonicalize().map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("cannot resolve `{relative}`: {error}"),
)
})?;
if !canonical.starts_with(&self.root) {
return Err(invalid_path(span, "path escapes the project root"));
}
return Ok(());
}
ancestor = current.parent();
}
Ok(())
}
/// Write a file inside the project root, creating parent directories.
pub fn write(
&self,
@@ -200,10 +587,88 @@ impl ProjectFs {
}
}
fn parse_offset(raw: &str, span: Option<Span>) -> Result<u64, RuntimeError> {
raw.parse::<u64>().map_err(|_| {
RuntimeError::new(
ErrorCategory::Type,
span,
format!("`{raw}` is not a non-negative integer offset"),
)
})
}
fn walk_directory(
dir: &Path,
prefix: &str,
out: &mut Vec<String>,
depth: usize,
) -> std::io::Result<()> {
if depth > 64 {
return Err(std::io::Error::new(
std::io::ErrorKind::InvalidInput,
"directory depth exceeds 64",
));
}
let mut entries = std::fs::read_dir(dir)?.collect::<Result<Vec<_>, _>>()?;
entries.sort_by_key(|entry| entry.file_name());
for entry in entries {
let file_type = entry.file_type()?;
if file_type.is_symlink() {
continue;
}
let name = entry.file_name().to_string_lossy().to_string();
let relative = if prefix.is_empty() {
name.clone()
} else {
format!("{prefix}/{name}")
};
if file_type.is_dir() {
walk_directory(&entry.path(), &relative, out, depth + 1)?;
} else {
out.push(relative);
}
}
Ok(())
}
fn invalid_path(span: Option<Span>, message: impl Into<String>) -> RuntimeError {
RuntimeError::new(ErrorCategory::PermissionDenied, span, message)
}
fn one_arg<'a>(
operation: &str,
args: &'a [String],
span: Option<Span>,
) -> Result<&'a str, RuntimeError> {
if args.len() != 1 {
return Err(fs_arity(operation, 1, args.len(), span));
}
Ok(&args[0])
}
fn fs_arity(operation: &str, expected: usize, actual: usize, span: Option<Span>) -> RuntimeError {
RuntimeError::new(
ErrorCategory::Arity,
span,
format!("filesystem operation `{operation}` expects {expected} argument(s), got {actual}"),
)
}
fn fs_ok(value: Value) -> Value {
let mut map = BTreeMap::new();
map.insert("ok".to_string(), Value::Bool(true));
map.insert("value".to_string(), value);
Value::Map(map)
}
fn fs_error(code: &str, message: &str) -> Value {
let mut map = BTreeMap::new();
map.insert("ok".to_string(), Value::Bool(false));
map.insert("code".to_string(), Value::text(code));
map.insert("message".to_string(), Value::text(message));
Value::Map(map)
}
#[cfg(test)]
mod tests {
use super::*;
File diff suppressed because it is too large. Load diff
+83
View File
@@ -0,0 +1,83 @@
//! `Host::invoke` dispatch table.
//!
//! Child module of `host`: it may call the parent's private capability,
//! resource and trace helpers. `StdHost` remains the only host implementation.
use super::*;
pub(crate) fn invoke(
host: &mut StdHost,
op: HostOp,
args: Vec<Value>,
ctx: &ExecutionContext,
) -> Result<Value, RuntimeError> {
// Addressable memory is language-local state. Handles must not reach a
// plugin, foreign kernel or file adapter, where they could not be
// interpreted safely (RFC-CM-0001 §8, ADR-0022 §4).
if args.iter().any(Value::contains_memory) {
return Err(RuntimeError::new(
ErrorCategory::Type,
ctx.span,
format!(
"Memory values cannot cross the host boundary (`{}`); copy bytes into \
Text explicitly first",
op.name()
),
));
}
if args.iter().any(Value::contains_resource) {
return Err(RuntimeError::new(
ErrorCategory::Type,
ctx.span,
format!(
"Resource handles cannot cross the generic host boundary (`{}`); use \
resource.* operations",
op.name()
),
));
}
let started = Instant::now();
let result = host.dispatch(op, args, ctx);
let summary = match &result {
Ok(_) => "ok".to_string(),
Err(error) => error.code().to_string(),
};
let estimate = TraceState::estimate(
"effect",
None,
Some(&ctx.module_id),
Some(&summary),
Some(op.name()),
);
if host.trace_state.accept(estimate) {
let seq = host.next_seq();
host.trace.record(
seq,
op.name(),
ctx.module_id.clone(),
ctx.span,
started.elapsed(),
summary,
);
}
result
}
pub(crate) fn dispatch(
host: &mut StdHost,
op: HostOp,
args: Vec<Value>,
ctx: &ExecutionContext,
) -> Result<Value, RuntimeError> {
match op {
HostOp::FsRead => host.op_fs_read(args, ctx),
HostOp::FsWrite => host.op_fs_write(args, ctx),
HostOp::MathParse
| HostOp::MathEval
| HostOp::MathSubstitute
| HostOp::MathSimplify
| HostOp::MathDifferentiate => crate::kernels::math(host, op, args, ctx),
HostOp::ForeignRun | HostOp::ForeignCall => crate::kernels::foreign(host, op, args, ctx),
HostOp::PluginInvoke => host.op_plugin_invoke(args, ctx),
}
}
+41
View File
@@ -0,0 +1,41 @@
//! Trace/effect bookkeeping helpers for `StdHost`.
//!
//! Child module of `host`; it may use the parent's private trace state.
use super::*;
impl StdHost {
pub(crate) fn next_seq(&mut self) -> u64 {
let seq = self.next_event_seq;
self.next_event_seq += 1;
seq
}
pub(crate) fn record_resource_effect(
&mut self,
op: &str,
ctx: &ExecutionContext,
started: Instant,
summary: &str,
) {
let estimate = TraceState::estimate(
"effect",
None,
Some(&ctx.module_id),
Some(summary),
Some(op),
);
if !self.trace_state.accept(estimate) {
return;
}
let seq = self.next_seq();
self.trace.record(
seq,
op,
ctx.module_id.clone(),
ctx.span,
started.elapsed(),
summary.to_string(),
);
}
}
+127
View File
@@ -0,0 +1,127 @@
//! Trusted resource-provider registration (ADR-0033/0035/0036/0037/0038).
//!
//! Registration is host-side installation metadata; language imports cannot
//! install or authorize a provider. Native libraries and helper processes are
//! trusted, not sandboxed.
use super::*;
impl StdHost {
/// Register an explicitly trusted in-process provider (ADR-0033).
///
/// In-process providers share the host address space and are not a
/// sandbox; only trusted adapters may be registered. Registration is not a
/// language operation and does not grant access by itself.
pub fn register_resource_provider(
&mut self,
provider: Box<dyn ResourceProvider>,
) -> Result<(), RuntimeError> {
self.register_provider_checked(provider)
}
/// Register a trusted C ABI resource provider (ADR-0036).
///
/// The shared library runs in the host address space and is **trusted**, not
/// sandboxed. Registration still requires the provider-name grant at
/// `resource.open` and is project-mode only.
pub fn register_c_resource_provider(
&mut self,
name: &str,
kinds: &[&str],
library: &std::path::Path,
) -> Result<(), RuntimeError> {
let kinds = kinds
.iter()
.map(|kind| (*kind).to_string())
.collect::<Vec<_>>();
let provider = CResourceProvider::open(name, &kinds, library)?;
self.register_provider_checked(Box::new(provider))
}
/// Register a trusted C ABI provider from a `marqdo.provider` manifest
/// (ADR-0037). The manifest is host-side installation metadata; language
/// imports cannot install or authorize a provider.
pub fn register_provider_package(
&mut self,
manifest_path: &std::path::Path,
) -> Result<(), RuntimeError> {
let provider = load_package(manifest_path)?;
self.register_provider_checked(Box::new(provider))
}
/// Register a trusted worker-process resource provider (ADR-0035).
///
/// The helper speaks framed JSON over stdin/stdout and is **not** an OS
/// sandbox; only explicitly trusted helpers may be spawned.
pub fn register_worker_resource_provider(
&mut self,
name: &str,
kinds: &[&str],
program: &std::path::Path,
args: &[String],
) -> Result<(), RuntimeError> {
let kinds = kinds
.iter()
.map(|kind| (*kind).to_string())
.collect::<Vec<_>>();
let provider = WorkerResourceProvider::spawn(name, &kinds, program, args)?;
self.register_provider_checked(Box::new(provider))
}
fn register_provider_checked(
&mut self,
provider: Box<dyn ResourceProvider>,
) -> Result<(), RuntimeError> {
if provider.protocol_version() != 1 {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!(
"resource provider `{}` uses unsupported protocol version {}",
provider.name(),
provider.protocol_version()
),
));
}
let name = provider.name().to_string();
if name.is_empty() || name == "stdio" || name == "file" {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!("resource provider name `{name}` is reserved or empty"),
));
}
let kinds = provider.kind_strings();
if kinds.is_empty() {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!("resource provider `{name}` declares no kinds"),
));
}
for kind in &kinds {
if kind.is_empty() || kind == "stdio" || kind == "file" {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!("resource provider `{name}` declares reserved kind `{kind}`"),
));
}
if let Some((existing, _)) = self.resource_providers.get(kind) {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
None,
format!(
"resource kind `{kind}` is already registered by provider `{existing}`"
),
));
}
}
let provider: SharedResourceProvider = std::sync::Arc::new(std::sync::Mutex::new(provider));
for kind in kinds {
self.resource_providers
.insert(kind, (name.clone(), std::sync::Arc::clone(&provider)));
}
Ok(())
}
}
+462
View File
@@ -0,0 +1,462 @@
//! Resource/stream open dispatch plus provider and TCP endpoints.
//!
//! Child module of `host`: it may use the parent's private fields, and keeps
//! the open-path policy out of the generic `Host::invoke` dispatch.
use super::*;
fn parse_host_port(descriptor: &str, span: Option<Span>) -> Result<(String, u16), RuntimeError> {
let invalid = || {
RuntimeError::new(
ErrorCategory::ResourceUnsupported,
span,
format!("tcp descriptor `{descriptor}` must be `host:port` or `[ipv6]:port`"),
)
};
if let Some(rest) = descriptor.strip_prefix('[') {
let (host, tail) = rest.split_once(']').ok_or_else(invalid)?;
let port = tail.strip_prefix(':').ok_or_else(invalid)?;
let port = port.parse::<u16>().map_err(|_| invalid())?;
if host.is_empty() {
return Err(invalid());
}
return Ok((host.to_string(), port));
}
let (host, port) = descriptor.rsplit_once(':').ok_or_else(invalid)?;
if host.is_empty() {
return Err(invalid());
}
let port = port.parse::<u16>().map_err(|_| invalid())?;
Ok((host.to_string(), port))
}
impl StdHost {
pub(crate) fn open_resource_stream(
&mut self,
kind: &str,
descriptor: &str,
rights: &[String],
ctx: &ExecutionContext,
) -> Result<Value, RuntimeError> {
let started = Instant::now();
let requested: BTreeSet<&str> = rights.iter().map(String::as_str).collect();
if self.mode == SandboxMode::Strict {
// stdio/stdout is the existing console channel already permitted by
// strict L1 (CM-A11); stdin, stderr, file/network/process providers
// are not, because strict isolation has not been proven for them.
let stdio_console =
kind == "stdio" && descriptor == "stdout" && requested == BTreeSet::from(["write"]);
if !stdio_console {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"strict mode permits only the existing stdio/stdout console channel; \
other resource streams are refused",
));
}
}
if self.mode == SandboxMode::Pure && kind == "file" {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"pure mode does not provide project file resources",
));
}
match kind {
"stdio" => {
let allowed = match descriptor {
"stdout" | "stderr" => requested == BTreeSet::from(["write"]),
"stdin" => requested == BTreeSet::from(["read"]),
_ => false,
};
if !allowed {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"stdio supports stdout/stderr with rights [write] and stdin with rights [read]",
));
}
if !self.capabilities.console {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"the console capability is required for stdio resources",
));
}
let handle = match descriptor {
"stdin" => self.resources.open_stdin(),
"stderr" => self.resources.open_stderr(),
"stdout" => self.resources.open_stdout(),
_ => unreachable!(),
};
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some(&format!("kind=stdio descriptor={descriptor}")),
);
self.record_resource_effect("resource.open", ctx, started, "ok");
Ok(Value::Resource(handle))
}
"file" => {
if requested == BTreeSet::from(["read"]) {
self.capabilities.check_read(descriptor, ctx.span)?;
let (_, relative) = self.fs.resolve(descriptor, ctx.span)?;
let bytes = self.fs.read_bytes(descriptor, ctx.span)?;
let handle = self.resources.open_file(relative, bytes);
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some("kind=file rights=read"),
);
self.record_resource_effect("resource.open", ctx, started, "ok");
return Ok(Value::Resource(handle));
}
if requested == BTreeSet::from(["write"]) {
self.capabilities.check_write(descriptor, ctx.span)?;
let (_, relative) = self.fs.resolve(descriptor, ctx.span)?;
let handle = self.resources.open_file_writer(relative);
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some("kind=file rights=write"),
);
self.record_resource_effect("resource.open", ctx, started, "ok");
return Ok(Value::Resource(handle));
}
Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"project file streams support exactly [read] or [write] in this build",
))
}
"tcp" => {
if self.mode != SandboxMode::Project {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"tcp resources are available only in project mode",
));
}
if requested != BTreeSet::from(["read", "write"]) {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"tcp resources require exactly [read, write] in this build",
));
}
let (host, port) = parse_host_port(descriptor, ctx.span)?;
self.capabilities.check_net(&host, ctx.span)?;
let addresses = (host.as_str(), port).to_socket_addrs().map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
ctx.span,
format!("cannot resolve `{host}:{port}`: {error}"),
)
})?;
let mut stream = None;
let mut last_error = None;
for address in addresses {
match TcpStream::connect_timeout(&address, Duration::from_secs(5)) {
Ok(connected) => {
stream = Some(connected);
break;
}
Err(error) => last_error = Some((address, error)),
}
}
let Some(stream) = stream else {
let detail = match last_error {
Some((address, error)) => format!("{address}: {error}"),
None => format!("`{host}:{port}` resolved to no addresses"),
};
return Err(RuntimeError::new(
ErrorCategory::HostError,
ctx.span,
format!("cannot connect to `{host}:{port}`: {detail}"),
));
};
let _ = stream.set_read_timeout(Some(Duration::from_secs(5)));
let _ = stream.set_write_timeout(Some(Duration::from_secs(5)));
let handle = self.resources.open_tcp(stream);
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some(&format!("kind=tcp host={host} port={port}")),
);
self.record_resource_effect("resource.open", ctx, started, "ok");
Ok(Value::Resource(handle))
}
"tls" => {
if self.mode != SandboxMode::Project {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"tls resources are available only in project mode",
));
}
if requested != BTreeSet::from(["read", "write"]) {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"tls resources require exactly [read, write] in this build",
));
}
let (host, port) = parse_host_port(descriptor, ctx.span)?;
self.capabilities.check_net(&host, ctx.span)?;
let addresses = (host.as_str(), port).to_socket_addrs().map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
ctx.span,
format!("cannot resolve `{host}:{port}`: {error}"),
)
})?;
let mut tcp = None;
let mut last_error = None;
for address in addresses {
match TcpStream::connect_timeout(&address, Duration::from_secs(5)) {
Ok(connected) => {
tcp = Some(connected);
break;
}
Err(error) => last_error = Some((address, error)),
}
}
let Some(tcp) = tcp else {
let detail = match last_error {
Some((address, error)) => format!("{address}: {error}"),
None => format!("`{host}:{port}` resolved to no addresses"),
};
return Err(RuntimeError::new(
ErrorCategory::HostError,
ctx.span,
format!("cannot connect tls to `{host}:{port}`: {detail}"),
));
};
let _ = tcp.set_read_timeout(Some(Duration::from_secs(5)));
let _ = tcp.set_write_timeout(Some(Duration::from_secs(5)));
let mut roots = rustls::RootCertStore::empty();
for certificate in rustls_native_certs::load_native_certs().certs {
let _ = roots.add(certificate);
}
for certificate in &self.tls_extra_roots {
roots.add(certificate.clone()).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
ctx.span,
format!("cannot add TLS root: {error}"),
)
})?;
}
if roots.is_empty() {
return Err(RuntimeError::new(
ErrorCategory::HostError,
ctx.span,
"no trusted TLS roots are available",
));
}
let config = rustls::ClientConfig::builder()
.with_root_certificates(roots)
.with_no_client_auth();
let server_name =
rustls::pki_types::ServerName::try_from(host.clone()).map_err(|_| {
RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
format!("`{host}` is not a valid TLS server name"),
)
})?;
let connection =
rustls::ClientConnection::new(std::sync::Arc::new(config), server_name)
.map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
ctx.span,
format!("cannot start TLS: {error}"),
)
})?;
let stream = rustls::StreamOwned::new(connection, tcp);
let handle = self.resources.open_tls(stream);
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some(&format!("kind=tls host={host} port={port}")),
);
self.record_resource_effect("resource.open", ctx, started, "ok");
Ok(Value::Resource(handle))
}
"tcp-listener" => {
if self.mode != SandboxMode::Project {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"tcp-listener resources are available only in project mode",
));
}
if requested != BTreeSet::from(["accept"]) {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"tcp-listener requires exactly [accept] in this build",
));
}
let (host, port) = parse_host_port(descriptor, ctx.span)?;
self.capabilities.check_net(&host, ctx.span)?;
let addresses = (host.as_str(), port).to_socket_addrs().map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
ctx.span,
format!("cannot resolve `{host}:{port}`: {error}"),
)
})?;
let mut listener = None;
let mut last_error = None;
for address in addresses {
match TcpListener::bind(address) {
Ok(bound) => {
listener = Some(bound);
break;
}
Err(error) => last_error = Some((address, error)),
}
}
let Some(listener) = listener else {
let detail = match last_error {
Some((address, error)) => format!("{address}: {error}"),
None => format!("`{host}:{port}` resolved to no addresses"),
};
return Err(RuntimeError::new(
ErrorCategory::HostError,
ctx.span,
format!("cannot listen on `{host}:{port}`: {detail}"),
));
};
listener.set_nonblocking(true).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
ctx.span,
format!("cannot configure tcp-listener: {error}"),
)
})?;
let handle = self.resources.open_tcp_listener(listener);
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some(&format!("kind=tcp-listener host={host} port={port}")),
);
self.record_resource_effect("resource.open", ctx, started, "ok");
Ok(Value::Resource(handle))
}
"udp" => {
if self.mode != SandboxMode::Project {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"udp resources are available only in project mode",
));
}
if requested != BTreeSet::from(["read", "write"]) {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
"udp resources require exactly [read, write] in this build",
));
}
let (host, port) = parse_host_port(descriptor, ctx.span)?;
self.capabilities.check_net(&host, ctx.span)?;
let addresses = (host.as_str(), port).to_socket_addrs().map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
ctx.span,
format!("cannot resolve `{host}:{port}`: {error}"),
)
})?;
let mut socket = None;
let mut last_error = None;
for address in addresses {
let bind = if address.is_ipv4() {
"0.0.0.0:0"
} else {
"[::]:0"
};
let candidate = UdpSocket::bind(bind).and_then(|socket| {
socket.connect(address)?;
Ok(socket)
});
match candidate {
Ok(connected) => {
socket = Some(connected);
break;
}
Err(error) => last_error = Some((address, error)),
}
}
let Some(socket) = socket else {
let detail = match last_error {
Some((address, error)) => format!("{address}: {error}"),
None => format!("`{host}:{port}` resolved to no addresses"),
};
return Err(RuntimeError::new(
ErrorCategory::HostError,
ctx.span,
format!("cannot connect udp to `{host}:{port}`: {detail}"),
));
};
let _ = socket.set_read_timeout(Some(Duration::from_secs(5)));
let _ = socket.set_write_timeout(Some(Duration::from_secs(5)));
let handle = self.resources.open_udp(socket);
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some(&format!("kind=udp host={host} port={port}")),
);
self.record_resource_effect("resource.open", ctx, started, "ok");
Ok(Value::Resource(handle))
}
other => {
if self.mode != SandboxMode::Project {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
"resource providers are available only in project mode",
));
}
let Some((provider_name, provider)) = self.resource_providers.get(other).cloned()
else {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
ctx.span,
format!("resource kind `{other}` is not registered"),
));
};
self.capabilities
.check_resource_provider(&provider_name, ctx.span)?;
let handle = self
.resources
.open_provider(provider, other, descriptor, rights, ctx.span)?;
self.record_semantic(
"resource.open",
None,
Some(&ctx.module_id),
ctx.span,
Some(&format!("kind={other} provider={provider_name}")),
);
self.record_resource_effect("resource.open", ctx, started, "ok");
Ok(Value::Resource(handle))
}
}
}
}
+1
View File
@@ -12,6 +12,7 @@ pub mod handles;
pub mod host;
pub mod kernels;
mod plugin_worker;
mod process;
mod provider_c;
mod provider_worker;
pub mod resource;
+139
View File
@@ -0,0 +1,139 @@
//! Explicitly granted process execution (ADR-0044).
//!
//! The child is **not** an OS sandbox: it inherits the CLI user's privileges.
//! The host checks an exact program grant before calling this module.
use std::collections::BTreeMap;
use std::path::Path;
use std::process::{Command, Stdio};
use std::time::{Duration, Instant};
use marqdo_value::{ErrorCategory, RuntimeError, Value};
use crate::context::ExecutionContext;
fn process_error(code: &str, message: &str) -> Value {
let mut map = BTreeMap::new();
map.insert("ok".to_string(), Value::Bool(false));
map.insert("code".to_string(), Value::text(code));
map.insert("message".to_string(), Value::text(message));
Value::Map(map)
}
pub(crate) fn run_process(
root: &Path,
program: &str,
args: &[String],
ctx: &ExecutionContext,
) -> Result<Value, RuntimeError> {
const LIMIT: usize = 1024 * 1024;
let mut command = Command::new(program);
command
.args(args)
.current_dir(root)
.env_clear()
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::piped());
#[cfg(windows)]
{
// Windows process creation and DLL loading require the system root;
// this is not a package-resolution path or a capability grant.
for name in ["SYSTEMROOT", "WINDIR"] {
if let Some(value) = std::env::var_os(name) {
command.env(name, value);
}
}
}
let child = command.spawn();
let child = match child {
Ok(child) => child,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
return Ok(process_error(
"process.not_found",
&format!("program `{program}` was not found"),
))
}
Err(error) if error.kind() == std::io::ErrorKind::PermissionDenied => {
return Err(RuntimeError::new(
ErrorCategory::PermissionDenied,
ctx.span,
format!("cannot execute `{program}`: {error}"),
))
}
Err(error) => {
return Err(RuntimeError::new(
ErrorCategory::HostError,
ctx.span,
format!("cannot start `{program}`: {error}"),
))
}
};
let mut child = child;
let deadline = Instant::now() + Duration::from_secs(5);
let mut timed_out = false;
loop {
match child.try_wait() {
Ok(Some(_)) => break,
Ok(None) if Instant::now() < deadline => {
std::thread::sleep(Duration::from_millis(10));
}
Ok(None) => {
timed_out = true;
let _ = child.kill();
break;
}
Err(error) => {
return Err(RuntimeError::new(
ErrorCategory::HostError,
ctx.span,
format!("cannot wait for `{program}`: {error}"),
))
}
}
}
let output = child.wait_with_output().map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
ctx.span,
format!("cannot collect `{program}` output: {error}"),
)
})?;
if timed_out {
return Ok(process_error(
"process.timeout",
&format!("program `{program}` exceeded the 5 second limit"),
));
}
if output.stdout.len() > LIMIT || output.stderr.len() > LIMIT {
return Ok(process_error(
"process.output_too_large",
&format!("program `{program}` exceeded the {LIMIT} byte output limit"),
));
}
let stdout = String::from_utf8(output.stdout).map_err(|_| {
RuntimeError::new(
ErrorCategory::Type,
ctx.span,
format!("program `{program}` stdout is not valid UTF-8"),
)
})?;
let stderr = String::from_utf8(output.stderr).map_err(|_| {
RuntimeError::new(
ErrorCategory::Type,
ctx.span,
format!("program `{program}` stderr is not valid UTF-8"),
)
})?;
let mut value = BTreeMap::new();
value.insert(
"status".to_string(),
Value::int(output.status.code().unwrap_or(-1) as i64),
);
value.insert("stdout".to_string(), Value::text(stdout));
value.insert("stderr".to_string(), Value::text(stderr));
let mut map = BTreeMap::new();
map.insert("ok".to_string(), Value::Bool(true));
map.insert("value".to_string(), Value::Map(value));
Ok(Value::Map(map))
}
+397 -8
View File
@@ -1,12 +1,15 @@
//! Host-owned resource/stream table (ADR-0025).
//!
//! The first slice deliberately supports only stdout writes and bounded
//! project-file reads loaded at open. Handles are run-scoped and opaque; the
//! table validates run identity, slot identity and generation on every call.
//! The stream table owns run-scoped stdio endpoints, bounded project-file
//! read/write streams, explicitly registered provider tokens and IP-literal TCP
//! streams. Handles are opaque and the table validates run identity, slot
//! identity and generation on every call. Bytes move only through `Memory`.
use std::collections::BTreeMap;
use std::io::{Read, Write};
use std::net::TcpStream;
use std::net::{TcpListener, TcpStream, UdpSocket};
use rustls::{ClientConnection, StreamOwned};
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex};
@@ -75,10 +78,23 @@ pub struct ResourceRead {
pub eof: bool,
}
struct StdinCursor {
bytes: Vec<u8>,
position: usize,
}
#[allow(clippy::large_enum_variant)]
enum Stream {
Stdin {
cursor: Arc<Mutex<StdinCursor>>,
closed: bool,
},
Stdout {
closed: bool,
},
Stderr {
closed: bool,
},
File {
relative: String,
bytes: Vec<u8>,
@@ -100,6 +116,18 @@ enum Stream {
stream: TcpStream,
closed: bool,
},
Udp {
socket: UdpSocket,
closed: bool,
},
TcpListener {
listener: TcpListener,
closed: bool,
},
Tls {
stream: StreamOwned<ClientConnection, TcpStream>,
closed: bool,
},
}
struct Slot {
@@ -110,11 +138,16 @@ struct Slot {
impl Slot {
fn closed(&self) -> bool {
match &self.stream {
Stream::Stdin { closed, .. } => *closed,
Stream::Stdout { closed } => *closed,
Stream::Stderr { closed } => *closed,
Stream::File { closed, .. } => *closed,
Stream::FileWriter { closed, .. } => *closed,
Stream::Provider { closed, .. } => *closed,
Stream::Tcp { closed, .. } => *closed,
Stream::Udp { closed, .. } => *closed,
Stream::TcpListener { closed, .. } => *closed,
Stream::Tls { closed, .. } => *closed,
}
}
}
@@ -124,6 +157,7 @@ pub struct ResourceTable {
run_id: u64,
next_resource: u64,
slots: BTreeMap<u64, Slot>,
stdin: Arc<Mutex<StdinCursor>>,
}
impl Default for ResourceTable {
@@ -139,6 +173,10 @@ impl ResourceTable {
run_id: NEXT_RUN_ID.fetch_add(1, Ordering::Relaxed),
next_resource: 1,
slots: BTreeMap::new(),
stdin: Arc::new(Mutex::new(StdinCursor {
bytes: Vec::new(),
position: 0,
})),
}
}
@@ -152,6 +190,47 @@ impl ResourceTable {
self.insert(Stream::Stdout { closed: false })
}
/// Register the pre-opened stderr stream.
pub fn open_stderr(&mut self) -> ResourceHandle {
self.insert(Stream::Stderr { closed: false })
}
/// Set the run-scoped stdin bytes at assembly time.
///
/// All `stdio/stdin` handles share this cursor so repeated console reads
/// continue through the input instead of restarting at byte zero.
pub fn set_stdin(&mut self, bytes: Vec<u8>) {
self.stdin = Arc::new(Mutex::new(StdinCursor { bytes, position: 0 }));
}
/// Register a read-only handle over the run-scoped stdin cursor.
pub fn open_stdin(&mut self) -> ResourceHandle {
self.insert(Stream::Stdin {
cursor: Arc::clone(&self.stdin),
closed: false,
})
}
/// Return the stdio descriptor name for a handle, if it is a stdio stream.
pub fn stdio_name(
&self,
handle: ResourceHandle,
span: Option<Span>,
) -> Result<Option<&'static str>, RuntimeError> {
let slot = self.slot(handle, span)?;
Ok(match &slot.stream {
Stream::Stdin { .. } => Some("stdin"),
Stream::Stdout { .. } => Some("stdout"),
Stream::Stderr { .. } => Some("stderr"),
Stream::File { .. } | Stream::FileWriter { .. } => None,
Stream::Provider { .. }
| Stream::Tcp { .. }
| Stream::Udp { .. }
| Stream::TcpListener { .. }
| Stream::Tls { .. } => None,
})
}
/// Register a project-file read stream opened from already-bounded bytes.
pub fn open_file(&mut self, relative: impl Into<String>, bytes: Vec<u8>) -> ResourceHandle {
self.insert(Stream::File {
@@ -179,6 +258,83 @@ impl ResourceTable {
})
}
/// Register a connected UDP socket.
pub fn open_udp(&mut self, socket: UdpSocket) -> ResourceHandle {
self.insert(Stream::Udp {
socket,
closed: false,
})
}
/// Register a bound TCP listener.
pub fn open_tcp_listener(&mut self, listener: TcpListener) -> ResourceHandle {
self.insert(Stream::TcpListener {
listener,
closed: false,
})
}
/// Register a TLS client stream over a TCP socket.
pub fn open_tls(&mut self, stream: StreamOwned<ClientConnection, TcpStream>) -> ResourceHandle {
self.insert(Stream::Tls {
stream,
closed: false,
})
}
/// Accept one inbound TCP connection from a listener, with a 5 s deadline.
pub fn accept(
&mut self,
handle: ResourceHandle,
span: Option<Span>,
) -> Result<ResourceHandle, RuntimeError> {
let slot = self.slot_mut(handle, span)?;
if slot.closed() {
return Err(RuntimeError::new(
ErrorCategory::ResourceClosed,
span,
"Resource handle is closed",
));
}
let Stream::TcpListener { listener, .. } = &mut slot.stream else {
return Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
span,
"resource.accept requires a tcp-listener handle",
));
};
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(5);
loop {
match listener.accept() {
Ok((stream, _)) => {
let _ = stream.set_read_timeout(Some(std::time::Duration::from_secs(5)));
let _ = stream.set_write_timeout(Some(std::time::Duration::from_secs(5)));
return Ok(self.insert(Stream::Tcp {
stream,
closed: false,
}));
}
Err(error) if error.kind() == std::io::ErrorKind::WouldBlock => {
if std::time::Instant::now() >= deadline {
return Err(RuntimeError::new(
ErrorCategory::HostError,
span,
"tcp-listener accept timed out after 5 seconds",
));
}
std::thread::sleep(std::time::Duration::from_millis(10));
}
Err(error) => {
return Err(RuntimeError::new(
ErrorCategory::HostError,
span,
format!("tcp-listener accept failed: {error}"),
))
}
}
}
}
/// Open a provider token and wrap it in a host-issued handle.
pub fn open_provider(
&mut self,
@@ -268,6 +424,17 @@ impl ResourceTable {
));
}
match &mut slot.stream {
Stream::Stdin { cursor, .. } => {
let mut cursor = cursor.lock().map_err(|_| stdin_lock_error(span))?;
let start = cursor.position;
let end = start.saturating_add(max_bytes).min(cursor.bytes.len());
let chunk = cursor.bytes[start..end].to_vec();
cursor.position = end;
Ok(ResourceRead {
bytes: chunk,
eof: end >= cursor.bytes.len(),
})
}
Stream::File {
bytes, position, ..
} => {
@@ -285,6 +452,11 @@ impl ResourceTable {
span,
"stdout does not support resource.read",
)),
Stream::Stderr { .. } => Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
span,
"stderr does not support resource.read",
)),
Stream::FileWriter { .. } => Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
span,
@@ -294,7 +466,18 @@ impl ResourceTable {
provider, token, ..
} => {
let mut provider = provider.lock().map_err(|_| provider_lock_error(span))?;
provider.read(*token, max_bytes, span)
let read = provider.read(*token, max_bytes, span)?;
if read.bytes.len() > max_bytes {
return Err(RuntimeError::new(
ErrorCategory::ResourceFault,
span,
format!(
"resource provider returned {} bytes for a {max_bytes}-byte read",
read.bytes.len()
),
));
}
Ok(read)
}
Stream::Tcp { stream, .. } => {
let mut buffer = vec![0u8; max_bytes];
@@ -311,6 +494,41 @@ impl ResourceTable {
eof: count == 0,
})
}
Stream::TcpListener { .. } => Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
span,
"a tcp-listener does not support resource.read; use resource.accept",
)),
Stream::Tls { stream, .. } => {
let mut buffer = vec![0u8; max_bytes];
let count = stream.read(&mut buffer).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("tls read failed: {error}"),
)
})?;
buffer.truncate(count);
Ok(ResourceRead {
bytes: buffer,
eof: count == 0,
})
}
Stream::Udp { socket, .. } => {
let mut buffer = vec![0u8; max_bytes];
let count = socket.recv(&mut buffer).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("udp read failed: {error}"),
)
})?;
buffer.truncate(count);
Ok(ResourceRead {
bytes: buffer,
eof: false,
})
}
}
}
@@ -333,7 +551,12 @@ impl ResourceTable {
));
}
match &mut slot.stream {
Stream::Stdout { .. } => Ok(bytes.len()),
Stream::Stdout { .. } | Stream::Stderr { .. } => Ok(bytes.len()),
Stream::Stdin { .. } => Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
span,
"stdin is a read-only resource stream",
)),
Stream::File { relative, .. } => Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
span,
@@ -369,7 +592,18 @@ impl ResourceTable {
provider, token, ..
} => {
let mut provider = provider.lock().map_err(|_| provider_lock_error(span))?;
provider.write(*token, bytes, span)
let count = provider.write(*token, bytes, span)?;
if count > bytes.len() {
return Err(RuntimeError::new(
ErrorCategory::ResourceFault,
span,
format!(
"resource provider reported {count} bytes written for a {}-byte request",
bytes.len()
),
));
}
Ok(count)
}
Stream::Tcp { stream, .. } => stream.write(bytes).map_err(|error| {
RuntimeError::new(
@@ -378,6 +612,25 @@ impl ResourceTable {
format!("tcp write failed: {error}"),
)
}),
Stream::TcpListener { .. } => Err(RuntimeError::new(
ErrorCategory::ResourceUnsupported,
span,
"a tcp-listener does not support resource.write; use resource.accept",
)),
Stream::Tls { stream, .. } => stream.write(bytes).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("tls write failed: {error}"),
)
}),
Stream::Udp { socket, .. } => socket.send(bytes).map_err(|error| {
RuntimeError::new(
ErrorCategory::HostError,
span,
format!("udp write failed: {error}"),
)
}),
}
}
@@ -396,7 +649,9 @@ impl ResourceTable {
));
}
match &mut slot.stream {
Stream::Stdin { closed, .. } => *closed = true,
Stream::Stdout { closed } => *closed = true,
Stream::Stderr { closed } => *closed = true,
Stream::File { closed, .. } => *closed = true,
Stream::FileWriter { closed, .. } => *closed = true,
Stream::Provider {
@@ -413,6 +668,9 @@ impl ResourceTable {
let _ = stream.shutdown(std::net::Shutdown::Both);
*closed = true;
}
Stream::Udp { closed, .. } => *closed = true,
Stream::TcpListener { closed, .. } => *closed = true,
Stream::Tls { closed, .. } => *closed = true,
}
Ok(())
}
@@ -447,10 +705,15 @@ impl ResourceTable {
pub fn kind(&self, handle: ResourceHandle, span: Option<Span>) -> Result<String, RuntimeError> {
let slot = self.slot(handle, span)?;
Ok(match &slot.stream {
Stream::Stdout { .. } => "stdio".to_string(),
Stream::Stdin { .. } | Stream::Stdout { .. } | Stream::Stderr { .. } => {
"stdio".to_string()
}
Stream::File { .. } | Stream::FileWriter { .. } => "file".to_string(),
Stream::Provider { kind, .. } => kind.clone(),
Stream::Tcp { .. } => "tcp".to_string(),
Stream::Udp { .. } => "udp".to_string(),
Stream::TcpListener { .. } => "tcp-listener".to_string(),
Stream::Tls { .. } => "tls".to_string(),
})
}
}
@@ -463,6 +726,14 @@ fn provider_lock_error(span: Option<Span>) -> RuntimeError {
)
}
fn stdin_lock_error(span: Option<Span>) -> RuntimeError {
RuntimeError::new(
ErrorCategory::HostError,
span,
"stdin cursor lock is poisoned",
)
}
fn unknown_handle(span: Option<Span>) -> RuntimeError {
RuntimeError::new(
ErrorCategory::ResourceFault,
@@ -536,4 +807,122 @@ mod tests {
"resource.unsupported"
);
}
#[test]
fn stdin_reads_are_chunked_and_end_at_eof() {
let mut table = ResourceTable::new();
table.set_stdin(b"ab".to_vec());
let stdin = table.open_stdin();
assert_eq!(
table.read(stdin, 1, None).unwrap(),
ResourceRead {
bytes: b"a".to_vec(),
eof: false
}
);
assert_eq!(
table.read(stdin, 8, None).unwrap(),
ResourceRead {
bytes: b"b".to_vec(),
eof: true
}
);
assert_eq!(
table.read(stdin, 1, None).unwrap(),
ResourceRead {
bytes: Vec::new(),
eof: true
}
);
assert_eq!(
table.write(stdin, b"x", None).unwrap_err().code(),
"resource.unsupported"
);
table.close(stdin, None).unwrap();
assert_eq!(
table.read(stdin, 1, None).unwrap_err().code(),
"resource.closed"
);
}
#[test]
fn stderr_is_write_only() {
let mut table = ResourceTable::new();
let stderr = table.open_stderr();
assert_eq!(table.write(stderr, b"diagnostic", None).unwrap(), 10);
assert_eq!(
table.read(stderr, 1, None).unwrap_err().code(),
"resource.unsupported"
);
assert_eq!(table.stdio_name(stderr, None).unwrap(), Some("stderr"));
table.close(stderr, None).unwrap();
assert_eq!(
table.write(stderr, b"x", None).unwrap_err().code(),
"resource.closed"
);
}
}
#[cfg(test)]
mod provider_tests {
use super::*;
struct OverReporting;
impl ResourceProvider for OverReporting {
fn name(&self) -> &str {
"over-reporting"
}
fn kinds(&self) -> &[&str] {
&["bad"]
}
fn open(
&mut self,
_kind: &str,
_descriptor: &str,
_rights: &[String],
_span: Option<Span>,
) -> Result<u64, RuntimeError> {
Ok(1)
}
fn read(
&mut self,
_token: u64,
max_bytes: usize,
_span: Option<Span>,
) -> Result<ResourceRead, RuntimeError> {
Ok(ResourceRead {
bytes: vec![0; max_bytes + 1],
eof: false,
})
}
fn write(
&mut self,
_token: u64,
bytes: &[u8],
_span: Option<Span>,
) -> Result<usize, RuntimeError> {
Ok(bytes.len() + 1)
}
fn close(&mut self, _token: u64, _span: Option<Span>) -> Result<(), RuntimeError> {
Ok(())
}
}
#[test]
fn provider_counts_are_bounded_by_the_request() {
let mut table = ResourceTable::new();
let provider: SharedResourceProvider = Arc::new(Mutex::new(Box::new(OverReporting)));
let handle = table
.open_provider(provider, "bad", "descriptor", &["read".to_string()], None)
.unwrap();
assert_eq!(
table.read(handle, 2, None).unwrap_err().code(),
"runtime.resource"
);
assert_eq!(
table.write(handle, b"x", None).unwrap_err().code(),
"runtime.resource"
);
}
}
+723 -19
View File
@@ -53,6 +53,63 @@ fn stdout_write_obeys_the_output_budget_atomically() {
assert!(host.take_output().is_empty());
}
#[test]
fn pure_mode_reads_stdin_through_a_resource() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path())
.with_stdin(b"abcd".to_vec());
let ctx = context();
let handle = handle(
host.resource_open("stdio", "stdin", &["read".to_string()], &ctx)
.unwrap(),
);
let first = host.resource_read(handle, 2, &ctx).unwrap();
assert_eq!(first.bytes, b"ab");
assert!(!first.eof);
let rest = host.resource_read(handle, 8, &ctx).unwrap();
assert_eq!(rest.bytes, b"cd");
assert!(rest.eof);
assert_eq!(host.resource_read(handle, 1, &ctx).unwrap().bytes, b"");
let write = host.resource_write(handle, b"x", &ctx).unwrap_err();
assert_eq!(write.code(), "resource.unsupported");
}
#[test]
fn stdout_and_stderr_are_captured_separately() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path());
let ctx = context();
let stdout = handle(
host.resource_open("stdio", "stdout", &["write".to_string()], &ctx)
.unwrap(),
);
let stderr = handle(
host.resource_open("stdio", "stderr", &["write".to_string()], &ctx)
.unwrap(),
);
assert_eq!(host.resource_write(stdout, b"out", &ctx).unwrap(), 3);
assert_eq!(host.resource_write(stderr, b"err", &ctx).unwrap(), 3);
assert_eq!(host.output(), b"out");
assert_eq!(host.error(), b"err");
assert_eq!(host.take_output(), b"out");
assert_eq!(host.take_error(), b"err");
}
#[test]
fn strict_mode_refuses_stdin_and_stderr() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(SandboxMode::Strict, Capabilities::pure(), dir.path())
.with_stdin(b"x".to_vec());
let ctx = context();
for (descriptor, rights) in [("stdin", "read"), ("stderr", "write")] {
let error = host
.resource_open("stdio", descriptor, &[rights.to_string()], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
}
#[test]
fn strict_mode_allows_only_the_existing_stdio_console_channel() {
let dir = tempfile::tempdir().unwrap();
@@ -414,28 +471,21 @@ fn tcp_resource_round_trips_with_a_network_grant() {
}
#[test]
fn tcp_requires_network_grant_and_rejects_dns_names() {
fn tcp_requires_a_host_grant_in_project_mode() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
let ctx = context();
let denied = host
.resource_open(
"tcp",
"127.0.0.1:9",
&["read".to_string(), "write".to_string()],
&ctx,
)
.unwrap_err();
assert_eq!(denied.category, ErrorCategory::PermissionDenied);
let dns = host
.resource_open(
"tcp",
"example.test:80",
&["read".to_string(), "write".to_string()],
&ctx,
)
.unwrap_err();
assert_eq!(dns.category, ErrorCategory::ResourceUnsupported);
for descriptor in ["127.0.0.1:9", "example.test:80"] {
let denied = host
.resource_open(
"tcp",
descriptor,
&["read".to_string(), "write".to_string()],
&ctx,
)
.unwrap_err();
assert_eq!(denied.category, ErrorCategory::PermissionDenied);
}
for mode in [SandboxMode::Pure, SandboxMode::Strict] {
let mut host = StdHost::new(
@@ -454,3 +504,657 @@ fn tcp_requires_network_grant_and_rejects_dns_names() {
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
}
#[test]
fn tcp_dns_name_connects_with_a_host_grant() {
use std::io::{Read, Write};
use std::net::TcpListener;
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let port = listener.local_addr().unwrap().port();
let server = std::thread::spawn(move || {
let (mut stream, _) = listener.accept().unwrap();
let mut buffer = [0u8; 4];
stream.read_exact(&mut buffer).unwrap();
stream.write_all(&buffer).unwrap();
});
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_net("localhost"),
dir.path(),
);
let ctx = context();
let handle = handle(
host.resource_open(
"tcp",
&format!("localhost:{port}"),
&["read".to_string(), "write".to_string()],
&ctx,
)
.unwrap(),
);
assert_eq!(host.resource_write(handle, b"ping", &ctx).unwrap(), 4);
assert_eq!(host.resource_read(handle, 4, &ctx).unwrap().bytes, b"ping");
host.resource_close(handle, &ctx).unwrap();
server.join().unwrap();
}
fn field<'a>(value: &'a Value, name: &str) -> &'a Value {
match value {
Value::Map(map) => map.get(name).unwrap_or_else(|| panic!("missing `{name}`")),
other => panic!("expected a Map, got {other:?}"),
}
}
#[test]
fn resource_requests_are_project_only_and_grant_checked() {
let dir = tempfile::tempdir().unwrap();
let ctx = context();
let mut pure = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path());
let error = pure
.resource_request("fs.exists", vec![Value::text("out")], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
let mut project = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
let error = project
.resource_request("fs.mkdir", vec![Value::text("out")], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
#[test]
fn fs_requests_create_list_rename_and_remove() {
let dir = tempfile::tempdir().unwrap();
let capabilities = Capabilities::project()
.allow_read("out")
.allow_read("out/**")
.allow_write("out")
.allow_write("out/**");
let mut host = StdHost::new(SandboxMode::Project, capabilities, dir.path());
let ctx = context();
let missing = host
.resource_request("fs.exists", vec![Value::text("out")], &ctx)
.unwrap();
assert_eq!(field(&missing, "value"), &Value::Bool(false));
let created = host
.resource_request("fs.mkdir", vec![Value::text("out")], &ctx)
.unwrap();
assert_eq!(field(&created, "ok"), &Value::Bool(true));
std::fs::write(dir.path().join("out/a.txt"), b"hello").unwrap();
let metadata = host
.resource_request("fs.metadata", vec![Value::text("out/a.txt")], &ctx)
.unwrap();
assert_eq!(
field(&metadata, "value"),
&Value::Map({
let mut map = std::collections::BTreeMap::new();
map.insert("kind".to_string(), Value::text("file"));
map.insert("size".to_string(), Value::int(5));
map
})
);
let listed = host
.resource_request("fs.list", vec![Value::text("out")], &ctx)
.unwrap();
assert_eq!(
field(&listed, "value"),
&Value::List(vec![Value::text("a.txt")])
);
let renamed = host
.resource_request(
"fs.rename",
vec![Value::text("out/a.txt"), Value::text("out/b.txt")],
&ctx,
)
.unwrap();
assert_eq!(field(&renamed, "ok"), &Value::Bool(true));
assert!(!dir.path().join("out/a.txt").exists());
assert!(dir.path().join("out/b.txt").exists());
let removed = host
.resource_request("fs.remove", vec![Value::text("out/b.txt")], &ctx)
.unwrap();
assert_eq!(field(&removed, "ok"), &Value::Bool(true));
let removed_dir = host
.resource_request("fs.remove", vec![Value::text("out")], &ctx)
.unwrap();
assert_eq!(field(&removed_dir, "ok"), &Value::Bool(true));
assert!(!dir.path().join("out").exists());
}
#[test]
fn fs_requests_reject_traversal_before_touching_the_filesystem() {
let dir = tempfile::tempdir().unwrap();
let capabilities = Capabilities::project()
.allow_write("../**")
.allow_read("../**");
let mut host = StdHost::new(SandboxMode::Project, capabilities, dir.path());
let ctx = context();
let error = host
.resource_request("fs.mkdir", vec![Value::text("../escape")], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
#[test]
fn clock_now_is_project_only_and_returns_unix_millis() {
let dir = tempfile::tempdir().unwrap();
let ctx = context();
let mut pure = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path());
let error = pure
.resource_request("clock.now", Vec::new(), &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
let mut project = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
let value = project
.resource_request("clock.now", Vec::new(), &ctx)
.unwrap();
assert_eq!(field(&value, "ok"), &Value::Bool(true));
let Value::Int(millis) = field(&value, "value") else {
panic!("clock.now did not return an Int");
};
let millis = millis.to_string().parse::<i64>().expect("i64 millis");
assert!(millis > 1_600_000_000_000);
let error = project
.resource_request("clock.now", vec![Value::text("x")], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::Arity);
}
#[test]
fn env_get_requires_an_exact_grant_and_reports_missing_names() {
let dir = tempfile::tempdir().unwrap();
let ctx = context();
let mut denied = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
let error = denied
.resource_request("env.get", vec![Value::text("PATH")], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
let mut pure = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path());
let error = pure
.resource_request("env.get", vec![Value::text("PATH")], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
let capabilities = Capabilities::project()
.allow_env("PATH")
.allow_env("MARQDO_TEST_MISSING_ENV");
let mut host = StdHost::new(SandboxMode::Project, capabilities, dir.path());
let present = host
.resource_request("env.get", vec![Value::text("PATH")], &ctx)
.unwrap();
assert_eq!(field(&present, "ok"), &Value::Bool(true));
let missing = host
.resource_request(
"env.get",
vec![Value::text("MARQDO_TEST_MISSING_ENV")],
&ctx,
)
.unwrap();
assert_eq!(field(&missing, "ok"), &Value::Bool(false));
assert_eq!(field(&missing, "code"), &Value::text("env.not_found"));
let error = host
.resource_request("env.get", Vec::new(), &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::Arity);
}
#[cfg(unix)]
#[test]
fn process_run_requires_an_exact_program_grant() {
let dir = tempfile::tempdir().unwrap();
let ctx = context();
let mut denied = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
let error = denied
.resource_request(
"process.run",
vec![
Value::text("/bin/echo"),
Value::List(vec![Value::text("x")]),
],
&ctx,
)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
let mut pure = StdHost::new(SandboxMode::Pure, Capabilities::pure(), dir.path());
let error = pure
.resource_request(
"process.run",
vec![
Value::text("/bin/echo"),
Value::List(vec![Value::text("x")]),
],
&ctx,
)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
let capabilities = Capabilities::project().allow_process("/bin/echo");
let mut host = StdHost::new(SandboxMode::Project, capabilities, dir.path());
let result = host
.resource_request(
"process.run",
vec![
Value::text("/bin/echo"),
Value::List(vec![Value::text("hello")]),
],
&ctx,
)
.unwrap();
assert_eq!(field(&result, "ok"), &Value::Bool(true));
let value = field(&result, "value");
assert_eq!(field(value, "status"), &Value::int(0));
assert_eq!(field(value, "stdout"), &Value::text("hello\n"));
assert_eq!(field(value, "stderr"), &Value::text(""));
let missing = host
.resource_request(
"process.run",
vec![
Value::text("/bin/definitely-not-a-program"),
Value::List(Vec::new()),
],
&ctx,
)
.unwrap_err();
assert_eq!(missing.category, ErrorCategory::PermissionDenied);
}
#[test]
fn fs_append_read_at_and_walk_are_bounded() {
let dir = tempfile::tempdir().unwrap();
let capabilities = Capabilities::project()
.allow_read("out")
.allow_read("out/**")
.allow_write("out")
.allow_write("out/**");
let mut host = StdHost::new(SandboxMode::Project, capabilities, dir.path());
let ctx = context();
host.resource_request("fs.mkdir", vec![Value::text("out")], &ctx)
.unwrap();
let appended = host
.resource_request(
"fs.append",
vec![Value::text("out/data.txt"), Value::text("hello")],
&ctx,
)
.unwrap();
assert_eq!(field(&appended, "value"), &Value::int(5));
host.resource_request(
"fs.append",
vec![Value::text("out/data.txt"), Value::text(" world")],
&ctx,
)
.unwrap();
let slice = host
.resource_request(
"fs.read_at",
vec![Value::text("out/data.txt"), Value::int(6), Value::int(5)],
&ctx,
)
.unwrap();
assert_eq!(field(&slice, "value"), &Value::text("world"));
let walked = host
.resource_request("fs.walk", vec![Value::text("out")], &ctx)
.unwrap();
assert_eq!(
field(&walked, "value"),
&Value::List(vec![Value::text("data.txt")])
);
let error = host
.resource_request(
"fs.read_at",
vec![Value::text("out/data.txt"), Value::int(-1), Value::int(1)],
&ctx,
)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::Type);
}
#[test]
fn random_bytes_require_the_entropy_grant() {
let dir = tempfile::tempdir().unwrap();
let ctx = context();
let mut denied = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
let error = denied
.resource_request("random.bytes", vec![Value::int(8)], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_random(),
dir.path(),
);
let value = host
.resource_request("random.bytes", vec![Value::int(16)], &ctx)
.unwrap();
assert_eq!(field(&value, "ok"), &Value::Bool(true));
let bytes = field(&value, "value").clone();
let Value::List(bytes) = bytes else {
panic!("random.bytes did not return a List");
};
assert_eq!(bytes.len(), 16);
for byte in bytes {
let Value::Int(byte) = byte else {
panic!("random byte is not an Int");
};
let byte = byte.to_string().parse::<i64>().unwrap();
assert!((0..=255).contains(&byte));
}
let error = host
.resource_request("random.bytes", vec![Value::int(0)], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::Type);
let mut pure = StdHost::new(
SandboxMode::Pure,
Capabilities::pure().allow_random(),
dir.path(),
);
let error = pure
.resource_request("random.bytes", vec![Value::int(8)], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
#[test]
fn udp_resource_round_trips_a_datagram_with_a_network_grant() {
use std::net::UdpSocket;
let server = UdpSocket::bind("127.0.0.1:0").unwrap();
let port = server.local_addr().unwrap().port();
let server = std::thread::spawn(move || {
let mut buffer = [0u8; 16];
let (count, peer) = server.recv_from(&mut buffer).unwrap();
server.send_to(&buffer[..count], peer).unwrap();
});
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_net("127.0.0.1"),
dir.path(),
);
let ctx = context();
let handle = handle(
host.resource_open(
"udp",
&format!("127.0.0.1:{port}"),
&["read".to_string(), "write".to_string()],
&ctx,
)
.unwrap(),
);
assert_eq!(host.resource_write(handle, b"ping", &ctx).unwrap(), 4);
let read = host.resource_read(handle, 16, &ctx).unwrap();
assert_eq!(read.bytes, b"ping");
assert!(!read.eof);
host.resource_close(handle, &ctx).unwrap();
server.join().unwrap();
}
#[test]
fn udp_is_project_only_and_grant_checked() {
let dir = tempfile::tempdir().unwrap();
let ctx = context();
for mode in [SandboxMode::Pure, SandboxMode::Strict] {
let mut host = StdHost::new(
mode,
Capabilities::project().allow_net("127.0.0.1"),
dir.path(),
);
let error = host
.resource_open(
"udp",
"127.0.0.1:9",
&["read".to_string(), "write".to_string()],
&ctx,
)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
let mut project = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
let error = project
.resource_open(
"udp",
"127.0.0.1:9",
&["read".to_string(), "write".to_string()],
&ctx,
)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
#[test]
fn tcp_listener_accepts_an_inbound_connection() {
use std::io::{Read, Write};
use std::net::{TcpListener, TcpStream};
let probe = TcpListener::bind("127.0.0.1:0").unwrap();
let port = probe.local_addr().unwrap().port();
drop(probe);
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_net("127.0.0.1"),
dir.path(),
);
let ctx = context();
let listener = handle(
host.resource_open(
"tcp-listener",
&format!("127.0.0.1:{port}"),
&["accept".to_string()],
&ctx,
)
.unwrap(),
);
let client = std::thread::spawn(move || {
let mut stream = TcpStream::connect(format!("127.0.0.1:{port}")).unwrap();
stream.write_all(b"ping").unwrap();
let mut buffer = [0u8; 4];
stream.read_exact(&mut buffer).unwrap();
assert_eq!(&buffer, b"ping");
});
let accepted = host.resource_accept(listener, &ctx).unwrap();
let value = field(&accepted, "value");
let Value::Resource(connection) = value else {
panic!("resource.accept did not return a Resource");
};
assert_eq!(
host.resource_read(*connection, 4, &ctx).unwrap().bytes,
b"ping"
);
assert_eq!(host.resource_write(*connection, b"ping", &ctx).unwrap(), 4);
host.resource_close(*connection, &ctx).unwrap();
host.resource_close(listener, &ctx).unwrap();
client.join().unwrap();
}
#[test]
fn tcp_listener_requires_accept_rights_and_project_mode() {
let dir = tempfile::tempdir().unwrap();
let ctx = context();
let mut project = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_net("127.0.0.1"),
dir.path(),
);
let error = project
.resource_open("tcp-listener", "127.0.0.1:9", &["read".to_string()], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::ResourceUnsupported);
let mut strict = StdHost::new(
SandboxMode::Strict,
Capabilities::project().allow_net("127.0.0.1"),
dir.path(),
);
let error = strict
.resource_open("tcp-listener", "127.0.0.1:9", &["accept".to_string()], &ctx)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
#[test]
fn tls_resource_round_trips_with_an_explicit_root() {
use std::io::{Read, Write};
use std::net::TcpListener;
use std::sync::Arc;
let certified = rcgen::generate_simple_self_signed(vec!["localhost".to_string()]).unwrap();
let certificate = certified.cert.der().clone();
let key = rustls::pki_types::PrivateKeyDer::Pkcs8(rustls::pki_types::PrivatePkcs8KeyDer::from(
certified.key_pair.serialize_der(),
));
let config = rustls::ServerConfig::builder()
.with_no_client_auth()
.with_single_cert(vec![certificate.clone()], key)
.unwrap();
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let port = listener.local_addr().unwrap().port();
let server = std::thread::spawn(move || {
let (stream, _) = listener.accept().unwrap();
let connection = rustls::ServerConnection::new(Arc::new(config)).unwrap();
let mut stream = rustls::StreamOwned::new(connection, stream);
let mut buffer = [0u8; 4];
stream.read_exact(&mut buffer).unwrap();
stream.write_all(&buffer).unwrap();
});
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::project().allow_net("localhost"),
dir.path(),
)
.with_tls_ca(certificate);
let ctx = context();
let handle = handle(
host.resource_open(
"tls",
&format!("localhost:{port}"),
&["read".to_string(), "write".to_string()],
&ctx,
)
.unwrap(),
);
assert_eq!(host.resource_write(handle, b"ping", &ctx).unwrap(), 4);
assert_eq!(host.resource_read(handle, 4, &ctx).unwrap().bytes, b"ping");
host.resource_close(handle, &ctx).unwrap();
server.join().unwrap();
}
#[test]
fn tls_requires_project_mode_and_a_host_grant() {
let dir = tempfile::tempdir().unwrap();
let ctx = context();
let mut project = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
let error = project
.resource_open(
"tls",
"localhost:9",
&["read".to_string(), "write".to_string()],
&ctx,
)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
let mut strict = StdHost::new(
SandboxMode::Strict,
Capabilities::project().allow_net("localhost"),
dir.path(),
);
let error = strict
.resource_open(
"tls",
"localhost:9",
&["read".to_string(), "write".to_string()],
&ctx,
)
.unwrap_err();
assert_eq!(error.category, ErrorCategory::PermissionDenied);
}
#[test]
fn compression_round_trips_and_rejects_invalid_gzip() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
let ctx = context();
let compressed = host
.resource_request("compress.gzip", vec![Value::text("hello 世界")], &ctx)
.unwrap();
assert_eq!(field(&compressed, "ok"), &Value::Bool(true));
let bytes = field(&compressed, "value").clone();
let decompressed = host
.resource_request("compress.gunzip", vec![bytes], &ctx)
.unwrap();
assert_eq!(field(&decompressed, "value"), &Value::text("hello 世界"));
let invalid = host
.resource_request(
"compress.gunzip",
vec![Value::List(vec![Value::int(1), Value::int(2)])],
&ctx,
)
.unwrap();
assert_eq!(field(&invalid, "ok"), &Value::Bool(false));
assert_eq!(field(&invalid, "code"), &Value::text("compress.invalid"));
}
#[test]
fn regex_operations_support_groups_alternation_and_errors() {
let dir = tempfile::tempdir().unwrap();
let mut host = StdHost::new(SandboxMode::Project, Capabilities::project(), dir.path());
let ctx = context();
let matched = host
.resource_request(
"regex.match",
vec![Value::text("^(a+)(b|c)$"), Value::text("aab")],
&ctx,
)
.unwrap();
assert_eq!(field(&matched, "value"), &Value::Bool(true));
let captures = host
.resource_request(
"regex.captures",
vec![Value::text("^(a+)(b|c)$"), Value::text("aab")],
&ctx,
)
.unwrap();
let value = field(&captures, "value");
assert_eq!(field(value, "matched"), &Value::Bool(true));
assert_eq!(
field(value, "groups"),
&Value::List(vec![
Value::text("aab"),
Value::text("aa"),
Value::text("b")
])
);
let invalid = host
.resource_request(
"regex.match",
vec![Value::text("["), Value::text("a")],
&ctx,
)
.unwrap();
assert_eq!(field(&invalid, "ok"), &Value::Bool(false));
assert_eq!(field(&invalid, "code"), &Value::text("regex.invalid"));
}
+16
View File
@@ -298,6 +298,20 @@ const RESOURCE_CLOSE: Builtin = Builtin {
named_only: false,
effectful: true,
};
const RESOURCE_REQUEST: Builtin = Builtin {
name: "resource.request",
aliases: &[],
params: &[req("operation"), req("arguments")],
named_only: false,
effectful: true,
};
const RESOURCE_ACCEPT: Builtin = Builtin {
name: "resource.accept",
aliases: &[],
params: &[req("handle")],
named_only: false,
effectful: true,
};
const TABLE_CELL: Builtin = Builtin {
name: "table.cell",
aliases: &[],
@@ -345,6 +359,8 @@ pub const BUILTINS: &[Builtin] = &[
RESOURCE_READ,
RESOURCE_WRITE,
RESOURCE_CLOSE,
RESOURCE_REQUEST,
RESOURCE_ACCEPT,
];
/// Find a builtin by canonical name or alias.
+12 -53
View File
@@ -12,7 +12,7 @@
use std::collections::BTreeMap;
use std::fmt;
use marqdo_expression::{Arg, Expr};
use marqdo_expression::Expr;
use marqdo_span::Span;
use serde::{Deserialize, Serialize};
@@ -152,6 +152,8 @@ pub enum UnitKind {
Namespace,
/// Level-2 heading: function.
Function,
/// Level-2 heading in Knowledge mode: inert knowledge unit.
Knowledge,
/// Level-3+ heading inside a function: named section, not a callable unit.
Section,
}
@@ -226,8 +228,9 @@ pub struct Function {
/// Writable place of an assignment (ADR-0022).
///
/// The Markdown surface is unchanged: both variants are `strong` nodes whose
/// payload contains `target = expression`. `Local` rebinds a frame slot;
/// Both variants come from an executable payload whose text is
/// `target = expression` (any carrier: `Strong`, `Emph` or inline `Code`).
/// `Local` rebinds a frame slot;
/// `Index` writes one byte into an addressable runtime `Memory` region and is
/// rejected for every other base value.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
@@ -281,21 +284,14 @@ pub enum Stmt {
/// Span of the whole statement.
span: Span,
},
/// Markdown-link call form from a block quote.
Call {
/// Resolved call site.
call: CallSite,
/// Span of the whole statement.
span: Span,
},
/// Ordered list: first matching condition wins.
/// `if`/`else` group: the first true arm wins.
Branch {
/// Arms in AST order.
arms: Vec<BranchArm>,
/// Span of the list node.
span: Span,
},
/// Bullet list item: a `while` loop.
/// `while`/`end` group.
Loop {
/// Loop condition.
condition: Expr,
@@ -303,10 +299,10 @@ pub enum Stmt {
condition_span: Span,
/// Loop body.
body: Vec<Stmt>,
/// Span of the list item.
/// Span of the whole `while` statement.
span: Span,
},
/// Return from the enclosing function (`emph` node).
/// Return from the enclosing function (`return` statement).
Return {
/// Returned expression.
value: Expr,
@@ -321,7 +317,6 @@ impl Stmt {
match self {
Stmt::Assign { span, .. }
| Stmt::Eval { span, .. }
| Stmt::Call { span, .. }
| Stmt::Branch { span, .. }
| Stmt::Loop { span, .. }
| Stmt::Return { span, .. } => *span,
@@ -329,12 +324,12 @@ impl Stmt {
}
}
/// One arm of a branch (ordered list item).
/// One arm of an `if`/`else` group.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct BranchArm {
/// Boolean condition expression.
pub condition: Expr,
/// Span of the condition code span.
/// Span of the condition payload.
pub condition_span: Span,
/// Statements executed when the condition is true.
pub body: Vec<Stmt>,
@@ -342,42 +337,6 @@ pub struct BranchArm {
pub span: Span,
}
/// Target of a call statement.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(tag = "target", rename_all = "snake_case")]
pub enum CallTarget {
/// Unqualified function in the current module.
Function {
/// Canonical function name.
name: String,
},
/// Function reached through an imported module alias.
Qualified {
/// Local import alias.
module_alias: String,
/// Canonical function name.
function: String,
},
/// Function resolved from a Markdown link target.
Resolved {
/// Canonical module identity.
module: ModuleId,
/// Canonical function name.
function: String,
},
}
/// A call statement with evaluated arguments.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct CallSite {
/// Callee target.
pub target: CallTarget,
/// Arguments in source order.
pub args: Vec<Arg>,
/// Span of the call.
pub span: Span,
}
/// One explicit import declared in front matter.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct ImportDecl {
+2 -2
View File
@@ -72,9 +72,9 @@ pub enum MdKind {
Paragraph,
/// Literal text.
Text,
/// Strong emphasis: an execution statement in executable position.
/// Strong emphasis: one payload carrier in an `@exec` scope.
Strong,
/// Emphasis: a return statement in executable position.
/// Emphasis: one payload carrier in an `@exec` scope.
Emph,
/// GFM strikethrough.
Strikethrough,
+25
View File
@@ -66,6 +66,19 @@ impl FrontMatter {
Ok(result)
}
/// Whether the top-level YAML mapping contains `key`.
///
/// Returns `false` for invalid YAML. Used to reject removed syntax
/// selectors such as `marqdo_semantics` or `syntax` without making the
/// frontend depend on a YAML crate.
pub fn has_key(&self, key: &str) -> bool {
let Ok(yaml) = &self.yaml else {
return false;
};
yaml.as_mapping()
.is_some_and(|mapping| mapping.contains_key(key))
}
/// The raw YAML parses successfully.
pub fn is_valid(&self) -> bool {
self.yaml.is_ok()
@@ -153,6 +166,18 @@ mod tests {
assert!(fm.imports().is_err());
}
#[test]
fn detects_removed_syntax_selector_keys() {
let literal = "---\nmarqdo_semantics: 2\nimport:\n x: \"@x\"\n---\n";
let fm = parse_literal(literal, Span::new(0, literal.len()));
assert!(fm.has_key("marqdo_semantics"));
assert!(!fm.has_key("syntax"));
assert!(fm.has_key("import"));
let invalid = parse_literal("---\nimport: [unclosed\n---", Span::new(0, 24));
assert!(!invalid.has_key("import"), "invalid YAML reports no key");
}
#[test]
fn import_shapes_are_validated() {
let literal = "---\nimport: geometry\n---\n";
+2 -2
View File
@@ -273,7 +273,7 @@ mod tests {
fn excludes_git_and_marqdo_directories() {
let dir = tempfile::tempdir().unwrap();
let root = dir.path();
write(&root.join("index.mq.md"), b"## main\n");
write(&root.join("index.mq.md"), b"## @exec main\n");
write(&root.join(".git/HEAD"), b"ref: main\n");
write(&root.join(".marqdo/env/env.json"), b"{}");
write(&root.join("nested/.marqdo/state"), b"x");
@@ -343,7 +343,7 @@ mod tests {
let dir = tempfile::tempdir().unwrap();
let source = dir.path().join("source");
let target = dir.path().join("target");
write(&source.join("index.mq.md"), b"## main\n");
write(&source.join("index.mq.md"), b"## @exec main\n");
write(&source.join("data/value.txt"), b"42");
write(&source.join(".git/HEAD"), b"ignored");
let digest = hash_tree(&source).unwrap();
+23 -12
View File
@@ -65,7 +65,10 @@ impl Dependency {
(None, Some(_)) if !is_full_commit(self.rev.as_deref().unwrap_or("")) => Err(format!(
"dependency `{name}` git rev must be a full lowercase 40- or 64-character commit hash"
)),
(Some(path), None) if path.is_absolute() => Err(format!(
// `has_root`, not `is_absolute`: on Windows a drive-less rooted
// path such as `/tmp/x` or `\tmp\x` is not `is_absolute()` but
// still escapes the project when joined. Reject every rooted path.
(Some(path), None) if path.has_root() => Err(format!(
"dependency `{name}` path must be project-relative"
)),
_ => Ok(()),
@@ -1599,7 +1602,7 @@ capabilities:
"project:\n name: geometry\n version: 0.1.0\n",
)
.unwrap();
std::fs::write(first.join("index.mq.md"), "## f\n\n*1*\n").unwrap();
std::fs::write(first.join("index.mq.md"), "## @exec f\n\n`return 1`\n").unwrap();
let manifest = Manifest::load(&root.join(MANIFEST_FILE)).unwrap();
let resolver =
PackageResolver::new(root.clone(), Some(manifest.clone()), None).with_unlocked(true);
@@ -1629,6 +1632,14 @@ capabilities:
fn rejects_absolute_dependency_paths() {
let yaml = "project:\n name: p\n version: 0.1.0\ndependencies:\n x:\n path: /tmp/x\n";
assert!(Manifest::from_yaml(yaml).is_err());
#[cfg(windows)]
{
// Rooted but drive-less: `is_absolute()` is false on Windows, yet
// joining it escapes the project root.
let rooted =
"project:\n name: p\n version: 0.1.0\ndependencies:\n x:\n path: \\tmp\\x\n";
assert!(Manifest::from_yaml(rooted).is_err());
}
}
fn have_git() -> bool {
@@ -1668,7 +1679,7 @@ capabilities:
"project:\n name: remote\n version: 0.2.0\n",
)
.unwrap();
std::fs::write(dep.join("index.mq.md"), "## hello\n\n*1*\n").unwrap();
std::fs::write(dep.join("index.mq.md"), "## @exec hello\n\n`return 1`\n").unwrap();
git(&dep, &["init", "-q"]);
git(&dep, &["add", "."]);
git(&dep, &["commit", "-q", "-m", "initial"]);
@@ -1721,7 +1732,7 @@ capabilities:
"project:\n name: geometry\n version: 0.1.0\n",
)
.unwrap();
std::fs::write(dep.join("index.mq.md"), "## area\n\n*1*\n").unwrap();
std::fs::write(dep.join("index.mq.md"), "## @exec area\n\n`return 1`\n").unwrap();
std::fs::write(
root.join(MANIFEST_FILE),
"project:\n name: p\n version: 0.1.0\ndependencies:\n geometry:\n path: ./deps/geometry\n",
@@ -1765,19 +1776,19 @@ capabilities:
"project:\n name: root\n version: 0.1.0\ndependencies:\n a:\n path: ./deps/a\n",
)
.unwrap();
std::fs::write(root.join("index.mq.md"), "## main\n\n*1*\n").unwrap();
std::fs::write(root.join("index.mq.md"), "## @exec main\n\n`return 1`\n").unwrap();
std::fs::write(
a.join(MANIFEST_FILE),
"project:\n name: a\n version: 0.1.0\ndependencies:\n b:\n path: ./b\n",
)
.unwrap();
std::fs::write(a.join("index.mq.md"), "## a\n\n*1*\n").unwrap();
std::fs::write(a.join("index.mq.md"), "## @exec a\n\n`return 1`\n").unwrap();
std::fs::write(
b.join(MANIFEST_FILE),
"project:\n name: b\n version: 0.1.0\n",
)
.unwrap();
std::fs::write(b.join("index.mq.md"), "## b\n\n*2*\n").unwrap();
std::fs::write(b.join("index.mq.md"), "## @exec b\n\n`return 2`\n").unwrap();
let manifest = Manifest::load(&root.join(MANIFEST_FILE)).unwrap();
let resolver = PackageResolver::new(root.clone(), Some(manifest), None).with_unlocked(true);
let graph = resolver.resolve_lock_graph().unwrap();
@@ -1813,7 +1824,7 @@ capabilities:
"project:\n name: geometry\n version: 0.1.0\n",
)
.unwrap();
std::fs::write(dep.join("index.mq.md"), "## area\n\n*1*\n").unwrap();
std::fs::write(dep.join("index.mq.md"), "## @exec area\n\n`return 1`\n").unwrap();
std::fs::write(
root.join(MANIFEST_FILE),
"project:\n name: p\n version: 0.1.0\ndependencies:\n geometry:\n path: ./deps/geometry\n",
@@ -1858,11 +1869,11 @@ capabilities:
"project:\n name: p\n version: 0.1.0\n",
)
.unwrap();
// macOS temp_dir is `/var/folders/...` while canonicalize resolves the
// `/var -> /private/var` symlink; compare canonical forms on both sides.
assert_eq!(
discover_root(&nested).unwrap(),
dir.path()
.canonicalize()
.unwrap_or_else(|_| dir.path().to_path_buf())
discover_root(&nested).unwrap().canonicalize().unwrap(),
dir.path().canonicalize().unwrap()
);
assert!(discover_root(Path::new("/")).is_none());
}
@@ -478,6 +478,21 @@ pub(crate) fn dispatch(
Err(error) => resource_recoverable(error),
}
}
"resource.accept" => {
let handle = resource_handle(take(&bound, "handle", builtin, span)?, span)?;
interpreter.resource_accept(handle, &module, span)
}
"resource.request" => {
let operation = expect_text_value(take(&bound, "operation", builtin, span)?, span)?;
let value_type = bound["arguments"].type_name();
let Value::List(arguments) = take(&bound, "arguments", builtin, span)? else {
return Err(RuntimeError::type_error(
span,
format!("resource.request arguments must be a List, got {value_type}"),
));
};
interpreter.resource_request(&operation, arguments, &module, span)
}
other => Err(RuntimeError::new(
ErrorCategory::KernelUnavailable,
span,
+34 -34
View File
@@ -4,7 +4,7 @@ use std::collections::BTreeMap;
use marqdo_expression::{BinaryOp, Expr, RecordEntry};
use marqdo_host::{ExecutionContext, Host, HostOp, ResourceRead};
use marqdo_ir::{builtins, CallTarget, Function, FunctionId, ModuleId, PlaceTarget, Program, Stmt};
use marqdo_ir::{builtins, Function, FunctionId, ModuleId, PlaceTarget, Program, Stmt};
use marqdo_span::Span;
use marqdo_value::{ErrorCategory, ResourceHandle, RuntimeError, Value};
@@ -344,9 +344,6 @@ impl<'a> Interpreter<'a> {
Stmt::Eval { expr, .. } => {
self.eval_expr(expr, frame)?;
}
Stmt::Call { call, span } => {
self.exec_call_site(program, call, *span, frame)?;
}
Stmt::Return { value, span } => {
let value = self.eval_expr(value, frame)?;
let module = frame.function.module.canonical();
@@ -424,36 +421,6 @@ impl<'a> Interpreter<'a> {
Ok(Flow::Next)
}
fn exec_call_site(
&mut self,
program: &'a Program,
call: &'a marqdo_ir::CallSite,
span: Span,
frame: &mut Frame,
) -> Result<Value, RuntimeError> {
let args = self.eval_args(&call.args, frame)?;
let (module, name) = match &call.target {
CallTarget::Function { name } => (frame.function.module.clone(), name.clone()),
CallTarget::Qualified {
module_alias,
function,
} => {
let Some(target) = program.resolve_import(&frame.function.module, module_alias)
else {
return Err(RuntimeError::new(
ErrorCategory::UnknownSymbol,
Some(span),
format!("unknown imported module alias `{module_alias}`"),
));
};
(target.clone(), function.clone())
}
CallTarget::Resolved { module, function } => (module.clone(), function.clone()),
};
let caller = frame.function.module.clone();
self.call_by_name_from(&module, &name, args, Some(span), Some(&caller))
}
/// Commit one byte into an addressable region after all operand checks.
///
/// The three operands are already evaluated, so a rejected value never
@@ -903,6 +870,39 @@ impl<'a> Interpreter<'a> {
})
}
/// Accept one inbound TCP connection from a listener handle.
pub(crate) fn resource_accept(
&mut self,
handle: ResourceHandle,
module: &ModuleId,
span: Option<Span>,
) -> Result<Value, RuntimeError> {
let context = ExecutionContext::new(module.to_string(), span);
self.host
.resource_accept(handle, &context)
.map_err(|mut error| {
error.mark_callsite();
error
})
}
/// Execute one non-streaming resource operation.
pub(crate) fn resource_request(
&mut self,
operation: &str,
args: Vec<Value>,
module: &ModuleId,
span: Option<Span>,
) -> Result<Value, RuntimeError> {
let context = ExecutionContext::new(module.to_string(), span);
self.host
.resource_request(operation, args, &context)
.map_err(|mut error| {
error.mark_callsite();
error
})
}
/// Close a resource stream.
pub(crate) fn resource_close(
&mut self,
+17 -15
View File
@@ -64,28 +64,28 @@ fn run_and_output(source: &str, function: &str) -> (Result<Value, RuntimeError>,
#[test]
fn recursion_branch_and_return() {
let source = "## 阶乘\n\n1. `n <= 1`\n\n *1*\n\n2. `n > 1`\n\n **结果 = n * 阶乘(n=n-1)**\n\n *结果*\n";
let source = "## @exec 阶乘\n\n`if n <= 1`\n\n`return 1`\n\n`else`\n\n`if n > 1`\n\n`结果 = n * 阶乘(n=n-1)`\n\n`return 结果`\n\n`end`\n\n`end`\n";
let value = run_named(source, "阶乘", &[("n", Value::int(5))]).unwrap();
assert_eq!(value, Value::int(120));
}
#[test]
fn loop_mutates_and_returns_unit_for_fallthrough() {
let source = "## 倒计时\n\n**n = 3**\n\n- `n > 0`\n\n **n = n - 1**\n";
let source = "## @exec 倒计时\n\n`n = 3`\n\n`while n > 0`\n\n`n = n - 1`\n\n`end`\n";
let (result, _) = run_and_output(source, "倒计时");
assert_eq!(result.unwrap(), Value::Unit);
}
#[test]
fn return_inside_loop_propagates() {
let source = "## 找\n\n**i = 0**\n\n- `true`\n\n **i = i + 1**\n\n - `i >= 3`\n\n *i*\n";
let source = "## @exec 找\n\n`i = 0`\n\n`while true`\n\n`i = i + 1`\n\n`if i >= 3`\n\n`return i`\n\n`end`\n\n`end`\n";
let (result, _) = run_and_output(source, "找");
assert_eq!(result.unwrap(), Value::int(3));
}
#[test]
fn explicit_signature_supports_positional_arguments() {
let source = "## 加\n\n`(a: Int, b: Int = 10)`\n\n*a + b*\n";
let source = "## @exec 加\n\n`params(a: Int, b: Int = 10)`\n\n`return a + b`\n";
let program = build(source);
let mut host = host();
let result = run(
@@ -101,7 +101,7 @@ fn explicit_signature_supports_positional_arguments() {
#[test]
fn builtins_for_conversions_and_length() {
let source = "## f\n\n*长度(\"你好\") + 整数(\"40\") + 整数(2.0)*\n";
let source = "## @exec f\n\n`return 长度(\"你好\") + 整数(\"40\") + 整数(2.0)`\n";
let value = run_named(source, "f", &[]).unwrap();
assert_eq!(value, Value::int(44));
}
@@ -109,21 +109,21 @@ fn builtins_for_conversions_and_length() {
#[test]
fn function_reference_can_be_called() {
let source =
"## 平方\n\n`(n: Int)`\n\n*n * n*\n\n## 主\n\n**f = ref(\"#平方\")**\n\n*f(n=4)*\n";
"## @exec 平方\n\n`params(n: Int)`\n\n`return n * n`\n\n## @exec 主\n\n`f = ref(\"#平方\")`\n\n`return f(n=4)`\n";
let value = run_named(source, "主", &[]).unwrap();
assert_eq!(value, Value::int(16));
}
#[test]
fn non_bool_condition_is_a_runtime_error() {
let source = "## f\n\n**x = 1**\n\n- `x`\n\n **x = x + 1**\n\n*x*\n";
let source = "## @exec f\n\n`x = 1`\n\n`while x`\n\n`x = x + 1`\n\n`end`\n\n`return x`\n";
let error = run_named(source, "f", &[]).unwrap_err();
assert_eq!(error.code(), "runtime.type");
}
#[test]
fn fuel_budget_is_enforced() {
let program = build("## f\n\n- `true`\n\n **x = 1**\n\n*0*\n");
let program = build("## @exec f\n\n`while true`\n\n`x = 1`\n\n`end`\n\n`return 0`\n");
let mut host = StdHost::new(
SandboxMode::Project,
Capabilities::pure(),
@@ -143,7 +143,7 @@ fn fuel_budget_is_enforced() {
#[test]
fn recursion_depth_is_bounded() {
let source = "## f\n\n*f(n=n + 1)*\n";
let source = "## @exec f\n\n`return f(n=n + 1)`\n";
let program = build(source);
let mut host = StdHost::new(
SandboxMode::Project,
@@ -165,14 +165,15 @@ fn recursion_depth_is_bounded() {
#[test]
fn ambiguous_ref_resource_is_an_error() {
let source = "## f\n\n### 公式\n\n$$a = 1$$\n\n$$b = 2$$\n\n**x = ref(\"#公式\")**\n\n*x*\n";
let source =
"## @exec f\n\n### 公式\n\n$$a = 1$$\n\n$$b = 2$$\n\n`x = ref(\"#公式\")`\n\n`return x`\n";
let error = run_named(source, "f", &[]).unwrap_err();
assert_eq!(error.code(), "ref.ambiguous");
}
#[test]
fn print_without_the_bundled_stdlib_is_a_structured_error() {
let source = "## f\n\n> 打印(1)\n\n*None*\n";
let source = "## @exec f\n\n`打印(1)`\n\n`return None`\n";
let error = run_named(source, "f", &[]).unwrap_err();
assert_eq!(error.code(), "kernel.unavailable");
assert!(
@@ -184,7 +185,7 @@ fn print_without_the_bundled_stdlib_is_a_structured_error() {
#[test]
fn function_ref_and_resource_members_are_readable() {
let source = "## 目标\n\n*1*\n\n## main\n\n### 公式\n\n$$a = 1$$\n\n### 代码\n\n```text\nx\n```\n\n**f = ref(\"#目标\")**\n\n**g = ref(\"#公式\")**\n\n**c = ref(\"#代码\")**\n\n*[f.module, f.name, g.latex, g.display, c.language, c.source]*\n";
let source = "## @exec 目标\n\n`return 1`\n\n## @exec main\n\n### 公式\n\n$$a = 1$$\n\n### 代码\n\n```text\nx\n```\n\n`f = ref(\"#目标\")`\n\n`g = ref(\"#公式\")`\n\n`c = ref(\"#代码\")`\n\n`return [f.module, f.name, g.latex, g.display, c.language, c.source]`\n";
let value = run_named(source, "main", &[]).unwrap();
let Value::List(items) = value else {
panic!("expected list");
@@ -199,7 +200,7 @@ fn function_ref_and_resource_members_are_readable() {
#[test]
fn memory_region_is_shared_by_handle_identity() {
let source = "## f\n\n**m = memory.new(1, 2)**\n\n**n = m**\n\n**`m[3] = 7`**\n\n*[n[3], m == n, memory.size(n)]*\n";
let source = "## @exec f\n\n`m = memory.new(1, 2)`\n\n`n = m`\n\n`m[3] = 7`\n\n`return [n[3], m == n, memory.size(n)]`\n";
let value = run_named(source, "f", &[]).unwrap();
assert_eq!(
value,
@@ -209,7 +210,7 @@ fn memory_region_is_shared_by_handle_identity() {
#[test]
fn memory_handle_cannot_reach_the_host_boundary() {
let source = "## f\n\n**m = memory.new(1, 1)**\n\n> fs.read(m)\n\n*0*\n";
let source = "## @exec f\n\n`m = memory.new(1, 1)`\n\n`fs.read(m)`\n\n`return 0`\n";
let error = run_named(source, "f", &[]).unwrap_err();
assert_eq!(error.code(), "runtime.type");
assert!(
@@ -223,7 +224,8 @@ fn memory_handle_cannot_reach_the_host_boundary() {
#[test]
fn memory_budget_is_enforced_without_changing_the_region() {
let program = build("## f\n\n**m = memory.new(1, 1)**\n\n*[m[0], memory.grow(m, 1)]*\n");
let program =
build("## @exec f\n\n`m = memory.new(1, 1)`\n\n`return [m[0], memory.grow(m, 1)]`\n");
let mut host = host();
let error = run_with_limits(
&program,
+152 -4
View File
@@ -13,7 +13,12 @@ use serde::{Deserialize, Serialize};
#[cfg(target_os = "linux")]
mod linux;
#[cfg(target_os = "linux")]
pub use linux::{marker_present, probe, verify_active, StrictOutcome};
pub use linux::probe;
/// Environment variable set by the strict supervisor for its re-executed
/// child. The marker alone never authorizes strict execution; [`inspect_strict_child`]
/// verifies the child identity before anything is trusted.
pub const STRICT_ACTIVE_ENV: &str = "MARQDO_STRICT_ACTIVE";
/// Execution mode requested on the command line.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
@@ -160,6 +165,97 @@ fn detect_strict() -> StrictIsolation {
}
}
/// Whether the current process is a verified strict child.
///
/// This is the **only** supported platform boundary for strict-child
/// detection. Linux performs the full identity verification; every other
/// platform refuses even a forged marker instead of treating it as a child.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum StrictChildState {
/// The process was not launched by the strict supervisor.
NotChild,
/// The process is the verified strict child and may run project code.
VerifiedChild,
}
/// Inspect whether this process is the strict supervisor's verified child.
///
/// Linux requires the marker **and** the full namespace/privilege identity
/// check in [`linux::verify_active`]. Non-Linux builds reject any marker:
/// without a verified OS isolation adapter there is no legitimate strict
/// child, so a forged `MARQDO_STRICT_ACTIVE` must fail closed.
pub fn inspect_strict_child() -> Result<StrictChildState, String> {
#[cfg(target_os = "linux")]
{
if !linux::marker_present() {
return Ok(StrictChildState::NotChild);
}
linux::verify_active().map(|()| StrictChildState::VerifiedChild)
}
#[cfg(not(target_os = "linux"))]
{
if std::env::var_os(STRICT_ACTIVE_ENV).is_some() {
return Err(
"strict active marker is unsupported on this platform; refusing to treat this \
process as a verified strict child"
.to_string(),
);
}
Ok(StrictChildState::NotChild)
}
}
/// Result of one supervised strict execution.
///
/// The outcome is a byte capture, **not** proof that the child completed
/// isolation verification: the caller must still route it through the public
/// strict contract (one JSON document in `--json` mode, no fallback).
#[derive(Debug, Clone)]
pub struct StrictOutcome {
/// Process exit code when the program exited normally.
pub exit_code: Option<i32>,
/// Terminating signal, when the program was killed by the supervisor,
/// a resource limit or the kernel.
pub signal: Option<i32>,
/// Captured stdout (bounded by the output limit).
pub stdout: Vec<u8>,
/// Captured stderr (bounded by the output limit).
pub stderr: Vec<u8>,
}
/// Stable classification of a strict control-plane failure.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum StrictFailureKind {
/// The runtime probe could not construct the isolation mechanism.
ProbeUnavailable,
/// The supervisor process could not be spawned.
SpawnFailed,
/// The supervisor exceeded the wall-clock budget.
Timeout,
/// A resource limit or signal terminated the supervised child.
ResourceLimit,
/// The requested launch is invalid (host root, document outside project).
InvalidRequest,
/// Capturing or reading the supervised output failed.
Io,
}
impl StrictFailureKind {
/// Stable machine-readable code consumed by `--json` error documents.
pub const fn code(self) -> &'static str {
match self {
StrictFailureKind::ProbeUnavailable => "strict.probe_unavailable",
StrictFailureKind::SpawnFailed => "strict.spawn_failed",
StrictFailureKind::Timeout => "strict.timeout",
StrictFailureKind::ResourceLimit => "strict.resource_limit",
StrictFailureKind::InvalidRequest => "strict.launch_failed",
StrictFailureKind::Io => "strict.launch_failed",
}
}
}
/// Failure to apply the requested sandbox mode.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum SandboxError {
@@ -170,11 +266,31 @@ pub enum SandboxError {
},
/// The strict supervisor could not launch, verify or supervise the child.
LaunchFailed {
/// Machine-readable classification.
kind: StrictFailureKind,
/// Actionable explanation.
message: String,
},
}
impl SandboxError {
/// Build a classified launch failure.
pub fn launch(kind: StrictFailureKind, message: impl Into<String>) -> Self {
SandboxError::LaunchFailed {
kind,
message: message.into(),
}
}
/// Stable machine-readable failure code for `--json` error documents.
pub const fn code(&self) -> &'static str {
match self {
SandboxError::StrictUnavailable { .. } => "strict.probe_unavailable",
SandboxError::LaunchFailed { kind, .. } => kind.code(),
}
}
}
impl std::fmt::Display for SandboxError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
@@ -182,7 +298,7 @@ impl std::fmt::Display for SandboxError {
f,
"refusing to run in strict mode: {reason}. Strict mode requires verified OS-level isolation and never falls back to an unrestricted process."
),
SandboxError::LaunchFailed { message } => {
SandboxError::LaunchFailed { message, .. } => {
write!(f, "strict isolation failed: {message}")
}
}
@@ -196,7 +312,7 @@ pub fn enforce(mode: SandboxMode, platform: &PlatformCapabilities) -> Result<(),
match mode {
SandboxMode::Strict => {
#[cfg(target_os = "linux")]
if verify_active().is_ok() {
if linux::verify_active().is_ok() {
return Ok(());
}
if platform.strict.is_available() {
@@ -246,7 +362,7 @@ pub fn launch_strict(command: &StrictHostCommand<'_>) -> Result<i32, SandboxErro
#[cfg(target_os = "linux")]
pub fn launch_strict_capture(
command: &StrictHostCommand<'_>,
) -> Result<linux::StrictOutcome, SandboxError> {
) -> Result<StrictOutcome, SandboxError> {
linux::launch_capture(command)
}
@@ -258,6 +374,17 @@ pub fn launch_strict(_command: &StrictHostCommand<'_>) -> Result<i32, SandboxErr
})
}
/// Capture a strict execution (unsupported elsewhere): every platform must
/// compile the same CLI routing code and fail closed instead of falling back.
#[cfg(not(target_os = "linux"))]
pub fn launch_strict_capture(
_command: &StrictHostCommand<'_>,
) -> Result<StrictOutcome, SandboxError> {
Err(SandboxError::StrictUnavailable {
reason: "strict OS isolation is only implemented on Linux".to_string(),
})
}
#[cfg(test)]
mod tests {
use super::*;
@@ -270,6 +397,27 @@ mod tests {
assert_eq!(SandboxMode::parse("nope"), None);
}
#[cfg(target_os = "linux")]
#[test]
fn ordinary_process_is_not_a_strict_child() {
if std::env::var_os(STRICT_ACTIVE_ENV).is_none() {
assert_eq!(
inspect_strict_child().expect("inspection must not fail without a marker"),
StrictChildState::NotChild
);
}
}
#[cfg(not(target_os = "linux"))]
#[test]
fn forged_marker_is_rejected_on_non_linux() {
std::env::set_var(STRICT_ACTIVE_ENV, "1");
let result = inspect_strict_child();
std::env::remove_var(STRICT_ACTIVE_ENV);
let error = result.expect_err("a forged marker must never authorize strict execution");
assert!(error.contains("unsupported"), "unexpected error: {error}");
}
#[test]
fn strict_refuses_when_isolation_is_unavailable() {
let platform = PlatformCapabilities {
+82 -66
View File
@@ -28,10 +28,7 @@ use std::path::{Path, PathBuf};
use std::process::{Child, Command, ExitStatus, Stdio};
use std::time::{Duration, Instant};
use crate::{SandboxError, StrictHostCommand};
/// Environment variable that marks a re-executed strict child.
pub const ACTIVE_ENV: &str = "MARQDO_STRICT_ACTIVE";
use crate::{SandboxError, StrictFailureKind, StrictHostCommand, StrictOutcome};
/// Path of the program inside the sandbox.
const SANDBOX_PROGRAM: &str = "/run/marqdo-program";
@@ -158,7 +155,7 @@ fn common_args() -> Vec<OsString> {
/// Verify that this process really is the strict child created by [`launch`].
pub fn verify_active() -> Result<(), String> {
if std::env::var(ACTIVE_ENV).ok().as_deref() != Some("1") {
if std::env::var(crate::STRICT_ACTIVE_ENV).ok().as_deref() != Some("1") {
return Err("strict active marker is absent".to_string());
}
// `--as-pid-1` makes the supervised program PID 1 in its own PID namespace.
@@ -204,21 +201,7 @@ pub fn verify_active() -> Result<(), String> {
/// Whether the current process claims to be a strict child. The marker alone
/// is not trusted; callers must use [`verify_active`].
pub fn marker_present() -> bool {
std::env::var(ACTIVE_ENV).ok().as_deref() == Some("1")
}
/// Result of one supervised strict execution.
#[derive(Debug, Clone)]
pub struct StrictOutcome {
/// Process exit code when the program exited normally.
pub exit_code: Option<i32>,
/// Terminating signal, when the program was killed by the supervisor,
/// a resource limit or the kernel.
pub signal: Option<i32>,
/// Captured stdout (bounded by the output limit).
pub stdout: Vec<u8>,
/// Captured stderr (bounded by the output limit).
pub stderr: Vec<u8>,
std::env::var(crate::STRICT_ACTIVE_ENV).ok().as_deref() == Some("1")
}
/// Launch `command` inside the strict sandbox and return its exit code,
@@ -251,7 +234,10 @@ fn finish(outcome: StrictOutcome) -> Result<i32, SandboxError> {
libc::SIGKILL => "strict child was killed by the resource supervisor".to_string(),
other => format!("strict child terminated by signal {other}"),
};
Err(SandboxError::LaunchFailed { message })
Err(SandboxError::launch(
StrictFailureKind::ResourceLimit,
message,
))
}
/// Probe function used by [`run`]. A function pointer keeps the production
@@ -277,43 +263,52 @@ fn run_with(
spawn_fn: SpawnFn,
) -> Result<StrictOutcome, SandboxError> {
if command.project_root == Path::new("/") {
return Err(SandboxError::LaunchFailed {
message: "strict mode refuses to expose the host root as the project".to_string(),
});
return Err(SandboxError::launch(
StrictFailureKind::InvalidRequest,
"strict mode refuses to expose the host root as the project",
));
}
let project_root =
fs::canonicalize(command.project_root).map_err(|error| SandboxError::LaunchFailed {
message: format!(
let project_root = fs::canonicalize(command.project_root).map_err(|error| {
SandboxError::launch(
StrictFailureKind::InvalidRequest,
format!(
"cannot resolve the strict project root `{}`: {error}",
command.project_root.display()
),
})?;
)
})?;
// The document (when supplied) must be inside the project root. It is
// rewritten to the sandbox path, so the host's absolute layout never leaks
// into the child and no other directory needs to be mounted.
let document = match command.document {
Some(document) => {
let absolute =
fs::canonicalize(document).map_err(|error| SandboxError::LaunchFailed {
message: format!("cannot resolve strict document: {error}"),
})?;
let relative =
absolute
.strip_prefix(&project_root)
.map_err(|_| SandboxError::LaunchFailed {
message: format!(
"strict document `{}` is outside the project root `{}`",
absolute.display(),
project_root.display()
),
})?;
let absolute = fs::canonicalize(document).map_err(|error| {
SandboxError::launch(
StrictFailureKind::InvalidRequest,
format!("cannot resolve strict document: {error}"),
)
})?;
let relative = absolute.strip_prefix(&project_root).map_err(|_| {
SandboxError::launch(
StrictFailureKind::InvalidRequest,
format!(
"strict document `{}` is outside the project root `{}`",
absolute.display(),
project_root.display()
),
)
})?;
Some(Path::new("/project").join(relative))
}
None => None,
};
probe_fn().map_err(|reason| SandboxError::LaunchFailed { message: reason })?;
let bwrap = bwrap_path().ok_or_else(|| SandboxError::LaunchFailed {
message: "bubblewrap disappeared after the probe".to_string(),
probe_fn()
.map_err(|reason| SandboxError::launch(StrictFailureKind::ProbeUnavailable, reason))?;
let bwrap = bwrap_path().ok_or_else(|| {
SandboxError::launch(
StrictFailureKind::ProbeUnavailable,
"bubblewrap disappeared after the probe",
)
})?;
let mut args = common_args();
@@ -335,7 +330,7 @@ fn run_with(
command.program.as_os_str().to_os_string(),
OsString::from(SANDBOX_PROGRAM),
OsString::from("--setenv"),
OsString::from(ACTIVE_ENV),
OsString::from(crate::STRICT_ACTIVE_ENV),
OsString::from("1"),
OsString::from("--chdir"),
OsString::from("/project"),
@@ -357,9 +352,10 @@ fn run_with(
args.push(argument.clone());
}
if !replaced {
return Err(SandboxError::LaunchFailed {
message: "strict launcher could not rewrite the document argument".to_string(),
});
return Err(SandboxError::launch(
StrictFailureKind::InvalidRequest,
"strict launcher could not rewrite the document argument",
));
}
let stdout = CaptureFile::create("stdout")?;
@@ -375,8 +371,11 @@ fn run_with(
limit,
cpu_limit,
)
.map_err(|error| SandboxError::LaunchFailed {
message: format!("cannot start the strict supervisor: {error}"),
.map_err(|error| {
SandboxError::launch(
StrictFailureKind::SpawnFailed,
format!("cannot start the strict supervisor: {error}"),
)
})?;
let status = match wait_with_deadline(&mut child, timeout) {
@@ -387,7 +386,7 @@ fn run_with(
let _ = child.wait();
let _ = stdout.read_all();
let _ = stderr.read_all();
return Err(SandboxError::LaunchFailed { message });
return Err(SandboxError::launch(StrictFailureKind::Timeout, message));
}
};
@@ -485,8 +484,11 @@ impl CaptureFile {
.write(true)
.create_new(true)
.open(&path)
.map_err(|error| SandboxError::LaunchFailed {
message: format!("cannot create strict capture file: {error}"),
.map_err(|error| {
SandboxError::launch(
StrictFailureKind::Io,
format!("cannot create strict capture file: {error}"),
)
})?;
Ok(Self {
path,
@@ -497,24 +499,32 @@ impl CaptureFile {
fn reopen(&self) -> Result<fs::File, SandboxError> {
self.file
.as_ref()
.ok_or_else(|| SandboxError::LaunchFailed {
message: "strict capture file is closed".to_string(),
.ok_or_else(|| {
SandboxError::launch(StrictFailureKind::Io, "strict capture file is closed")
})?
.try_clone()
.map_err(|error| SandboxError::LaunchFailed {
message: format!("cannot clone strict capture file: {error}"),
.map_err(|error| {
SandboxError::launch(
StrictFailureKind::Io,
format!("cannot clone strict capture file: {error}"),
)
})
}
fn read_all(&self) -> Result<Vec<u8>, SandboxError> {
let mut file = fs::File::open(&self.path).map_err(|error| SandboxError::LaunchFailed {
message: format!("cannot read strict capture file: {error}"),
let mut file = fs::File::open(&self.path).map_err(|error| {
SandboxError::launch(
StrictFailureKind::Io,
format!("cannot read strict capture file: {error}"),
)
})?;
let mut bytes = Vec::new();
file.read_to_end(&mut bytes)
.map_err(|error| SandboxError::LaunchFailed {
message: format!("cannot read strict capture file: {error}"),
})?;
file.read_to_end(&mut bytes).map_err(|error| {
SandboxError::launch(
StrictFailureKind::Io,
format!("cannot read strict capture file: {error}"),
)
})?;
Ok(bytes)
}
}
@@ -569,7 +579,8 @@ mod tests {
let _ = fs::remove_dir_all(&project);
match &error {
SandboxError::LaunchFailed { message } => {
SandboxError::LaunchFailed { kind, message } => {
assert_eq!(*kind, StrictFailureKind::SpawnFailed);
assert!(
message.contains("cannot start the strict supervisor"),
"unexpected launch diagnostic: {message}"
@@ -604,7 +615,12 @@ mod tests {
|_, _, _, _, _, _| unreachable!("host root must be rejected before spawning"),
)
.expect_err("host root must be rejected");
assert!(matches!(error, SandboxError::LaunchFailed { .. }));
match &error {
SandboxError::LaunchFailed { kind, .. } => {
assert_eq!(*kind, StrictFailureKind::InvalidRequest);
}
other => panic!("expected LaunchFailed, got {other:?}"),
}
assert!(error.to_string().contains("host root"));
}
}
+21 -5
View File
@@ -4,11 +4,11 @@
//! supervisor used by `marqdo run --mode strict`, then exercise direct
//! syscalls for filesystem, network, process, privilege and resource escapes.
//!
//! They require a host where [`marqdo_sandbox::probe`] succeeds (Bubblewrap
//! plus permitted user/mount/PID/net namespaces). On an incapable host they
//! are skipped **unless** `MARQDO_REQUIRE_STRICT_SECURITY=1` is set, which the
//! G-C security lane must set so a missing mechanism can never count as a
//! pass.
//! They are **opt-in** so an ordinary `cargo test --workspace` on a
//! namespace-capable but non-disposable host does not become a false G0 gate.
//! Set `MARQDO_REQUIRE_STRICT_SECURITY=1` (the G-C lane does; a missing
//! mechanism then fails) or `MARQDO_RUN_STRICT_SECURITY=1` (developer opt-in;
//! an incapable host skips).
//!
//! The helper path can be supplied with `MARQDO_SECURITY_HELPER` (used when
//! running this test binary inside the disposable security VM). Otherwise the
@@ -32,6 +32,15 @@ fn required() -> bool {
== Some("1")
}
/// The adversarial matrix is G3-only. Ordinary `cargo test --workspace` on a
/// developer host or GitHub-hosted runner must not turn a namespace-capable but
/// non-disposable environment into a red G0 gate. The G3 lane sets
/// `MARQDO_REQUIRE_STRICT_SECURITY=1`; a developer can opt in with
/// `MARQDO_RUN_STRICT_SECURITY=1`.
fn enabled() -> bool {
required() || std::env::var("MARQDO_RUN_STRICT_SECURITY").ok().as_deref() == Some("1")
}
/// Compile the escape helper exactly once even when the test binary runs its
/// tests in parallel.
fn helper() -> Option<PathBuf> {
@@ -65,6 +74,13 @@ fn helper() -> Option<PathBuf> {
}
fn setup() -> Option<(PathBuf, PathBuf, PathBuf)> {
if !enabled() {
eprintln!(
"skipping strict security lane: set MARQDO_REQUIRE_STRICT_SECURITY=1 \
(G3) or MARQDO_RUN_STRICT_SECURITY=1 to opt in"
);
return None;
}
if let Err(reason) = probe() {
if required() {
panic!("G-C security lane requires strict isolation but probe failed: {reason}");
+35 -3
View File
@@ -6,13 +6,45 @@ use serde::{Deserialize, Serialize};
/// Stable diagnostic codes. CI may depend on these strings; they must not be
/// renamed without an ADR because they are part of the CLI contract.
pub mod codes {
/// A Markdown AST node occupies an executable position but has an invalid shape.
/// Retired by RFC-LANG-0002: executable payload shapes are reported as
/// `sem.*` instead. Kept only for decoding historical Analysis/Trace JSON.
pub const MARKDOWN_INVALID_CONSTRUCT: &str = "markdown.invalid_construct";
/// A node that would be executable appears outside any function body and is
/// therefore inert document content.
/// Retired by RFC-LANG-0002: a payload with no owning Exec H2 is
/// `sem.exec_without_function`, and Knowledge content is inert without a
/// warning. Kept only for decoding historical Analysis/Trace JSON.
pub const MARKDOWN_INERT_EXECUTABLE: &str = "markdown.inert_executable";
/// Front matter is malformed or appears somewhere other than the document start.
pub const FRONTMATTER_INVALID: &str = "frontmatter.invalid";
/// A heading uses an invalid `@exec`/`@knowledge` mode marker.
pub const SEM_INVALID_MODE_MARKER: &str = "sem.invalid_mode_marker";
/// A mode declaration has no semantic heading name.
pub const SEM_EMPTY_UNIT_NAME: &str = "sem.empty_unit_name";
/// Duplicate canonical heading/unit name in one scope.
pub const SEM_DUPLICATE_UNIT_NAME: &str = "sem.duplicate_unit_name";
/// Executable payload appears under no Exec H2 function.
pub const SEM_EXEC_WITHOUT_FUNCTION: &str = "sem.exec_without_function";
/// A payload carrier contains a nested inline structure.
pub const SEM_PAYLOAD_NESTED: &str = "sem.payload_nested";
/// A payload carrier is empty in an Exec scope.
pub const SEM_EMPTY_PAYLOAD: &str = "sem.empty_payload";
/// A payload failed semantic statement validation.
pub const SEM_SYNTAX: &str = "sem.syntax";
/// `else` does not bind an open `if`.
pub const SEM_UNMATCHED_ELSE: &str = "sem.unmatched_else";
/// A second `else` appears for one `if`.
pub const SEM_DUPLICATE_ELSE: &str = "sem.duplicate_else";
/// `end` closes no open frame.
pub const SEM_UNMATCHED_END: &str = "sem.unmatched_end";
/// A control frame is not closed before the heading ends.
pub const SEM_UNCLOSED_CONTROL: &str = "sem.unclosed_control";
/// `params(...)` is not the first direct Exec H2 payload.
pub const SEM_SIGNATURE_POSITION: &str = "sem.signature_position";
/// A version directive such as `marqdo_semantics` is forbidden.
pub const SEM_UNSUPPORTED_VERSION_DIRECTIVE: &str = "sem.unsupported_version_directive";
/// `run` found no callable entry function.
pub const SEM_NO_ENTRY_FUNCTION: &str = "sem.no_entry_function";
/// A signature was inferred rather than declared.
pub const SEM_INFERRED_SIGNATURE: &str = "sem.inferred_signature";
/// An executable payload failed to parse as an expression or statement.
pub const EXPR_PARSE: &str = "expr.parse";
/// A function name is referenced but no such function exists.
+14 -1
View File
@@ -1,5 +1,18 @@
# Marqdo CLI 核心稳定化、协议冻结与独立生态准入实施规范
> **Historical / Superseded review baseline (`0adebe4`, 2026-11-20).** The
> source review and CI facts here are a snapshot, not current state. The current
> overview is [`architecture/CURRENT.md`](architecture/CURRENT.md); executed
> status is in [`STATUS.md`](STATUS.md).
>
> **语言语义已被取代:** 本文件的语言描述早于
> [`docs/architecture/RFC-LANG-0002-unified-semantics.md`](architecture/RFC-LANG-0002-unified-semantics.md)
> ([ADR-0052](adr/0052-unified-markdown-semantics.md))。现行唯一语言规范是
> 该 RFC 与 `spec/mapping.md` / `spec/expression.ebnf`;本文的旧分阶段路线、
> 问题清单和门槛均为历史输入,不表示已实现,也不得覆盖 RFC。下一阶段计划以
> [`STABLE_RELEASE_IMPLEMENTATION.md`](STABLE_RELEASE_IMPLEMENTATION.md) 为准。
> **文档状态:Proposed / 可直接用作下一阶段工程实施基线**
> **源码审查基线:** [`cflmy/marqdo-cli@0adebe42f14d72a2a275564d03ff9558f197766e`](https://github.com/cflmy/marqdo-cli/commit/0adebe42f14d72a2a275564d03ff9558f197766e),`master`,2026-11-20。
> **验证来源:** 对上述提交的 GitHub 源码、规范、测试、CI workflow 和 [CI run #44671025763](https://github.com/cflmy/marqdo-cli/actions/runs/44671025763) 日志的静态审阅;**未在本文写作环境重新本地编译、运行测试或验证 OS 沙箱**。
@@ -37,7 +50,7 @@
Marqdo 是以 Markdown 语法树为正式结构、具有确定执行语义的编程语言。其基础原则为“代码即文档,文档即知识库,知识即可执行”。它不是将 Markdown 翻译成 Python 后执行的工具。普通文档内容保持 Markdown 可读性,具有执行语义的结构由固定 Markdown profile 和 language mapping 解释,再由独立 Rust 解释器运行。
语言规范权威入口:`spec/markdown-profile.md`、`spec/mapping.md`、`spec/expression.ebnf`、`spec/evaluation.md`、`spec/references.md`。修改这些文件规定的语义属于**语言版本变更**,必须单独提出 ADR 和金样测试;本文件不会擅自改变它们。
语言规范权威入口(现行):`docs/architecture/RFC-LANG-0002-unified-semantics.md` 及其摘要 `spec/mapping.md`、`spec/expression.ebnf`、`spec/evaluation.md`、`spec/references.md`;`spec/markdown-profile.md` 仍固定 Markdown profile。(本句在写作时指向旧映射,已按 RFC 更新;下文其余叙述为历史。)修改这些文件规定的语义属于**语言版本变更**,必须单独提出 ADR 和金样测试;本文件不会擅自改变它们。
语言核心应独立完成 Markdown AST 解析、函数和参数推断、表达式与控制流、模块/显式导入、静态检查、确定的值及错误语义、Rust 原生执行、能力检查、依赖环境。编译/运行 **不需要** Knowledge、Proof 或 View。
+116 -107
View File
@@ -7,11 +7,15 @@
> **优先阅读**:本文件 → `spec/` 形式规范 → `tests/conformance/` 金样 → 相关模块源码
> **约定**:**[D]** 已确定的设计方向;**[P]** 本文提出的可实现细节,须经形式化测试和 ADR 批准后冻结;**[F]** 后期能力,不应阻塞首个独立运行版本。
> **语言映射状态(2026-11-21,必读)**:本文档原本是 M0.5/M0.6 期的架构蓝图,其中关于 Markdown 程序角色的描述(旧映射:`Strong`=执行、`Emph`=返回、有序列表=分支、无序列表=循环、引用块=调用、首个独立 CodeSpan=签名)**已被废止,仅作为历史设计记录**。现行唯一语言语义权威是
> [`docs/architecture/RFC-LANG-0002-unified-semantics.md`](architecture/RFC-LANG-0002-unified-semantics.md)(由 [ADR-0052](adr/0052-unified-markdown-semantics.md) 采纳),其规范摘要为 [`spec/mapping.md`](../spec/mapping.md) 与 [`spec/expression.ebnf`](../spec/expression.ebnf)。
> 本文 §0.1、§1.2–1.4、§2.2、§2.4、§3.1–3.2、§4.1、§5.2、§6.2、§8.1、§13、§14、§16.3、§17 已按现行映射改写或标注历史;其余章节(IR、ABI、安全、包管理、发布)仍是架构参考。
> **实现状态**:RFC 与 `spec/` 领先于 Rust 实现。当前前端、表达式解析器与标准库仍实现旧映射,执行源码的测试预期为红;见 [`docs/STATUS.md`](STATUS.md) 与 [`docs/migration/README.md`](migration/README.md)。不要把本蓝图或 RFC 示例当作现有 CLI 已经支持的证据。
> **示例标注**:本文未明确标注为“历史”的 Markdown/`.mq.md` 示例均按 RFC-LANG-0002 目标语法书写,属于示意片段,前端迁移前不能原样运行;标注为历史的片段保留旧映射,仅供决策考古。
**这不是对 `cflmy/marqdo` 的兼容性重构。本项目必须从语言目标出发,独立建立 Marqdo 工具链。** 旧仓库只作为经验、算法、测试和可能的代码复用来源;旧语法、旧 ABI、旧插件体系不因名称相同而自动成为新版本的兼容契约。
> **M0.6 规划说明**:`docs/NEXT_PHASE_IMPLEMENTATION.md`(Proposed)与 `docs/EXTENSIBILITY_AUDIT.md` 已在命令规模、能力注册、插件寻址、包身份、仓库解耦和发布门槛上提出新的目标方案;发生冲突时以该 M0.6 计划为准。`spec/`、已接受 ADR 与测试仍是已实现语言行为的规范来源,Proposed 内容不表示已实现。
>
> **当前阶段基线**:`docs/CORE_STABILIZATION_AND_ECOSYSTEM_GATE.md`(Proposed,审查基线 `0adebe4`)基于 M0.6 计划之后的提交重新审查 CI、锁图、源码身份、Analysis/Trace、Tool 与安全隔离,并定义 G-A/G-B/G-C 发布门槛。该文在**当前基线、问题清单和发布门槛**上取代上述两份文档;冲突时以其为准,但同样不表示已实现。
> **规划文档关系(2026-11-21 更新)**:语言语义只以 RFC-LANG-0002 与 `spec/` 为准。下一阶段工作与发布门槛以 [`docs/STABLE_RELEASE_IMPLEMENTATION.md`](STABLE_RELEASE_IMPLEMENTATION.md) 为准;`docs/CORE_STABILIZATION_AND_ECOSYSTEM_GATE.md`(审查基线 `0adebe4`)、`docs/NEXT_PHASE_IMPLEMENTATION.md`(M0.6)与 `docs/NEXT_RELEASE_CI_CONVERGENCE.md` 是历史规划输入,不表示已实现,也不能覆盖 RFC 的语言映射。当前执行状态见 [`docs/STATUS.md`](STATUS.md)。
---
@@ -19,7 +23,7 @@
1. **[D] Markdown 是语言的语法,而不只是程序的注释格式。** Marqdo 的合法源码是选定 Markdown 方言的严格子集;不要求任意 `.md` 自动变成可运行程序。
2. **[D] 标准 Markdown AST 是唯一的文档结构事实来源。** 使用成熟解析器;不用正则表达式重新识别 Markdown 块/强调/列表。可以构造程序 IR,但不能平行维护另一套“私有 Markdown AST”。
3. **[D] Markdown 节点的程序角色是确定的。** `Strong`、`Emph`、`OrderedList`、`BulletList`、`BlockQuote` 等在适用的程序区域有明确语义。解析失败即诊断;禁止“像代码就执行,否则当文本”的启发式。
3. **[D] Markdown 节点的程序角色是确定的。** 标题上的 `@exec`/`@knowledge` 声明(或从最近祖先标题继承)决定该区域是执行还是知识;执行区域内 `Strong`、`Emph`、行内 `Code` 是等价的单语句载荷,`if`/`else`/`while`/`end`/`return`/`params` 决定控制与签名,列表、引用块和段落只是容器。解析失败即诊断;禁止“像代码就执行,否则当文本”的启发式。现行权威为 RFC-LANG-0002;旧“`Strong`=执行、`Emph`=返回、列表=分支/循环、引用块=调用”映射已废止。
4. **[D] 自然语言可与执行节点同处一个段落或分支。** 叙述永远不触发执行,任何中文/英文词语都不能被猜成控制关键字。
5. **[D] Marqdo 自主执行,不翻译成 Python 作为正式后端。** 用 Rust 实现表达式求值、词法作用域、函数调用、返回、分支、循环和运行时错误;初期 tree-walk,后期可加字节码。
6. **[D] 表格、数学公式、代码围栏是原生文档值。** 公式计算和外部语言执行是 CLI 内置能力,但读取或渲染文档不会自动求值/执行它们。
@@ -30,7 +34,7 @@
### 0.1 设计理念与取舍
Marqdo 想要实现的不是 Python 的 Markdown 版本,也不是 Jupyter Notebook 的另一种封装,而是**让标准 Markdown 结构直接承担程序控制语义**:标题承载作用域、粗体承载执行、斜体承载返回、有序列表承载分支、无序列表承载循环、引用块承载调用。编程者通过有限约定撰写源文件,编译器对这些约定进行严格解释,普通 Markdown 阅读器仍能渲染。
Marqdo 想要实现的不是 Python 的 Markdown 版本,也不是 Jupyter Notebook 的另一种封装,而是**让标准 Markdown 结构直接承担程序作用域与载荷承载**:标题用 `@exec`/`@knowledge` 声明语义模式并沿祖先继承,只有 Exec H2 是可调用函数;执行区域内粗体、斜体与行内代码是等价的单语句载荷,程序操作由载荷中的 `if`/`else`/`while`/`end`/`return`/`params` 关键字决定;有序/无序列表与引用块只组织阅读顺序,不产生指令。普通 Markdown 阅读器仍能渲染源文件。(上述为现行 RFC-LANG-0002 映射;本蓝图早期“粗体承载执行、斜体承载返回、有序列表承载分支、无序列表承载循环、引用块承载调用”的表述属于历史设计,已废止。)
**不追求**“任意人类叙述皆可编译”;**不追求**“每种 Markdown 强调方式都仍有普通文档原义”;**不追求**“每一个 Markdown 节点都必须执行”;**不追求**“复杂程序完全不需要学习计算表达式”。
@@ -57,50 +61,53 @@ Marqdo 想要实现的不是 Python 的 Markdown 版本,也不是 Jupyter Note
- 单个解析器可能在 AST 中合并相邻引用行、把 Setext/ATX 标题统一为 `Heading`、把不同项目符号统一为 `BulletList`。语言语义针对**AST 节点**,规范书写使用固定符号,但不能依赖丢失的源码标记。
- 不从渲染后的 HTML 反推程序。输入 Markdown AST 是唯一解析来源,源位置信息应被保留用于 IDE 和诊断。
### 1.2 文档区域与执行区域
### 1.2 文档区域与执行区域(RFC-LANG-0002)
**[P] 作用域**:`# 名称` 建立顶层作用域,`## 名称` 建立函数;更深层标题在当前函数内部创建命名分节而**不**引入新运行时函数。Markdown AST 中标题通常是同级块,必须按 heading level 另建**范围索引**:一个 `##` 的范围延续到下一个同级或更高级标题之前。若没有上层 `#`,函数位于模块根。嵌套标题和代码围栏内的标题不产生顶层函数。
**[D] 标题模式**:H1–H6 的首个纯 Text 子节点可以以 `@exec` 或 `@knowledge` 加空白与非空名称开头,声明该标题范围的模式;未声明的标题继承最近祖先标题的模式,文档根为 Knowledge。模式声明不属于语义名称,`@exec`/`@knowledge` 不进入可调用名称。`# @exec 名称` 建立执行命名空间,`## @exec 名称` 建立**唯一可调用函数**;H2 Knowledge 只是知识单元,H3–H6 只是分节,永不创建可调用函数。
**[P] 执行上下文限定**:程序区域内,强/斜体节点参与执行;标题文字、链接标签、表格数据单元、代码围栏内容、数学内部、HTML 原文等**惰性叶节点**中的内联格式绝不执行。此约束基于祖先 AST 类型、不是表达式文本。出现在正文或列表体中的普通 `Paragraph` 可以混合 `Text` 与执行节点。
**[P] 执行上下文限定**:只有 Exec 范围(继承或显式)中的 `Strong`、`Emph`、行内 `Code` 才是载荷候选;每个闭合载体承载**恰好一条语句**,载体种类不决定操作。标题文字、链接标签与 URL、图片 alt、表格数据单元、代码围栏、数学、HTML、front matter 等惰性叶节点中的内容绝不执行。列表、引用块和段落只决定载荷的文档顺序,不赋予 Branch/Loop/Call 角色。
**[P] 函数范围与入口**:`main`(模块根下唯一入口函数)为 `marqdo run` 默认入口,导入模块的 `main` 不执行;`marqdo call file.mq.md 函数名` 选择明确目标。纯叙述函数调用后返回 `Unit`,不自动返回其段落内容。
**[D] 函数范围与入口**:只有 Exec H2 产生函数;其直接正文与后代 Exec H3–H6 分节按源序拼接为同一函数体,并在每个标题边界重置并要求控制栈为空。`main` 必须是 Exec H2 才能作为 `marqdo run` 默认入口;`marqdo call file.mq.md 函数名` 选择明确目标。无任何显式 `return` 的函数自然结束返回 `Unit`,不等于 `None`。
### 1.3 核心 Markdown AST 映射(强制无歧义)
### 1.3 核心 Markdown AST 映射(RFC-LANG-0002;强制无歧义)
| Markdown AST 节点 | 程序意义 | 必须满足的条件 | 非法时 |
| Markdown AST 节点 | 当前程序意义 | 必须满足的条件 | 非法时 |
|---|---|---|---|
| `Document` | 模块 | 文档根 | 编译错误 |
| `Heading(level=1)` | 命名空间/对象域 | 非空规范名称 | 编译错误 |
| `Heading(level=2)` | 函数 | 非空可寻址名称;作用域唯一 | 编译错误 |
| `Heading(level>=3)` | 分节/源码定位 | 必须位于所属模块或函数范围 | 不执行 |
| `Document` | 模块,根模式为 Knowledge | 文档根 | 编译错误 |
| `Heading(level=1)` | 命名空间/顶层单元(声明或继承模式) | 非空规范名称 | 编译错误 |
| `Heading(level=2, mode=Exec)` | **唯一可调用函数** | 非空可寻址名称;作用域唯一 | 编译错误 |
| `Heading(level=2, mode=Knowledge)` | 惰性知识单元 | 非空名称 | 不执行 |
| `Heading(level>=3)` | 所属函数/知识单元的分节(可本地覆盖模式) | 必须位于合法祖先范围 | 不创建函数 |
| `Text` | 叙述 | 不参与求值 | 不报错 |
| `Strong` | 执行语句 | 其完整纯文本/代码跨度载荷满足 `Statement` 文法 | **硬错误,不降级为强调** |
| `Emph` | 返回语句 | 载荷满足 `Expression` 文法;可嵌入段落 | **硬错误** |
| `Code` 行内 | 符号、形式条件、名称载荷 | 仅当位于声明/分支/循环头等规定位置才被解释 | 指定位置非法时报错 |
| `OrderedList` | 顺序优先的条件分支 | 每个 Item 有合法布尔条件头 | **硬错误,不降级为普通列表** |
| `BulletList` | 循环组 | 每个 Item 有合法条件或遍历头 | **硬错误** |
| `BlockQuote` | 调用 | 完整的调用节点结构;不得混普通引文 | **硬错误** |
| `Table` | 结构化值 | GFM 合法数据表 | 类型/约束错误 |
| `Strong` @ Exec | 单语句载荷载体 | 直接 Text/Code 子节点内容恰好一条语句;嵌套载体/复杂子节点报错 | **硬错误,不降级为强调** |
| `Emph` @ Exec | 单语句载荷载体 | 与 `Strong` 完全等价 | **硬错误** |
| `Code` 行内 @ Exec | 单语句载荷载体 | 与 `Strong`/`Emph` 完全等价;不享有签名特权 | 载荷非法时报错 |
| `Code` 行内 @ Knowledge/惰性区 | 文档代码文本 | 不执行 | 不报错 |
| `OrderedList` / `BulletList` | 纯文档容器 | 按源序递归提取其中的合法载荷;列表属性不影响指令 | **不产生隐含分支/循环** |
| `Paragraph` | 纯文档容器/载荷载体宿主 | 按源序提取合法载荷;纯叙述零指令 | 不报错 |
| `BlockQuote` | 纯文档容器 | 按源序递归提取合法载荷;引用本身不是调用 | **不产生隐含调用** |
| `Table` | 结构化值 | GFM 合法数据表;单元格内容不执行 | 类型/约束错误 |
| `CodeBlock` | `Code` 值 | 识别语言标记;未知语言允许存档 | 请求运行未知语言时报错 |
| `Math` | `Formula` 值 | 可渲染,不承诺均可符号解析 | 请求计算不受支持时诊断 |
| `Link` | 静态知识引用 | 可解析内外资源;普通外链不自动加载 | 按引用策略诊断 |
| `Image` / `Footnote` | 文档资源/来源 | 惰性 | 不执行 |
| `ThematicBreak` | 视觉分隔 | 惰性 | 不执行 |
**关于保留标记的代价**:在函数体中 `**自然语言强调**` 会尝试编译成语句并失败,`*普通斜体*` 会尝试返回并失败,`1. 步骤` 会被视为分支,`- 条目` 会被视为循环,`> 引言` 会被视为调用。作者需要使用普通段落、标题、公式或其他未保留结构表达这些内容。这是语言设计选择,不是解析器缺陷。
**关于保留标记的代价(RFC-LANG-0002)**:在 Exec 范围内 `**自然语言强调**`、`*普通斜体*` 和 `` `任意代码` `` 都会被当作一条语句载荷;若无法解析为合法语句,就是硬诊断,不会被猜成排版。需要纯视觉强调时,放进 `@knowledge` 分节或其他惰性区域。列表和引用块不再被强制解释为分支/循环/调用,普通文档结构可以在函数内自由使用。
**关键消歧约束**:不要采用“在 `Strong` 中能解析成语句才是执行,否则不执行”的规则。AST 节点类型及位置确定角色,角色确定后才解析内部表达式。错误必须立即报告。
**关键消歧约束**:不要采用“在 `Strong` 中能解析成语句才是执行,否则不执行”的规则。AST 节点类型及其模式/位置决定它是否是载荷,载荷必须按唯一文法完整解析;控制操作只由载荷内的关键字决定,不能由载体种类或容器种类决定。错误必须立即报告。旧的 Strong=执行/Emph=返回/List=分支/Quote=调用规则**不得**再作为现行设计。
### 1.4 Markdown 结构对齐的困难问题(必须写进测试)
### 1.4 Markdown 结构对齐的困难问题(必须写进测试;RFC-LANG-0002)
1. `Strong`/`Emph` 在一个 Paragraph 内可多次出现;按 DFS 文档顺序遍历并执行,但遇到 `Return` 立即短路剩余节点。`Strong`/`Emph` 相互嵌套或包含 `Link` 等非纯载荷时首版拒绝,避免 AST 内容被递归执行两次。
2. `OrderedList` 不能使用 `1.` 后的任何自然语言充当条件。每个列表项的**首个块**必须是 Paragraph,且它的第一个有效 inline 节点必须为 `CodeSpan(布尔表达式)`;其余同段文本只作说明。每个 Item 后续块按顺序构成分支体。分支从上到下选首个为 `true` 的条件,只执行该分支。无命中返回 `Unit` 并继续执行列表之后的节点。
3. `BulletList` 的结构头与分支类似;每个 Item 是一个独立的 `while` 型循环,循环头条件每轮重新求值,每轮执行其余 Item 块;多个 Item 按顺序执行多个循环。集合遍历优先采用标准库 `map/filter/fold` 或预定义迭代器;是否增加独立 foreach 头必须另写 ADR。
4. `BlockQuote` 默认表示**一次明确调用**。只接受**一个 Paragraph**,内容必须是完整 `CallExpr` 或 `Link + 后续实参`;不接受普通引文、多个段落或嵌套引用块。两次独立调用必须分成两个独立 `BlockQuote` AST 节点(需实测 CommonMark 的合并行为)。若作者需要在引用块内写叙述,首版不允许;避免调用/引证混淆。
5. 标题内部的 `Strong`、表格中的 `Strong` 不会因递归访问而执行。实现必须基于**上下文访问器**而非无条件遍历全树。
6. Markdown `*` 既是乘法运算符又是强调界定符。执行语句中推荐 `**a * b**`(包含空格)或稳定的 `**\`a*b\`**`(渲染突出一个行内代码表达式)。两者**都属于 Strong**,表达式载荷相同;如源码被 Markdown 解析为非法嵌套,则报出“请将表达式放入行内代码跨度”。检查器必须直接验证最终 AST,而不靠原始正则。
1. `Strong`/`Emph`/行内 `Code` 在 Exec 范围内都是单语句载体,彼此等价;必须提取完整外壳节点且只提取一次。嵌套 `Strong`/`Emph`/`Link`/数学等复杂子节点报 `sem.payload_nested`,不得部分执行或静默拼接。
2. `OrderedList`/`BulletList` 只是容器,`ordered`/`start`/`tight`/任务框选等属性不影响指令;编译器按源序递归提取其中载荷,没有条件头、没有隐含分支或循环。空投影(零语句)不等于产生 `List([])` 值。
3. `BlockQuote` 只是容器,没有调用语义;其中合法的独立载荷照常执行,纯引用正文零指令。旧 `>` 自动调用规则已废止。
4. 标题模式声明只在标题首个纯 Text 子节点以精确 `@exec`/`@knowledge` 开头时生效;大小写变体、拼错、冲突声明、空名称都是硬错误,不可退回祖先模式。Setext 标题只继承模式,不携带声明。
5. 标题内部的 `Strong`、表格单元格中的 `Strong`、链接标签中的载荷绝不执行;实现必须基于上下文访问器而不是无条件遍历全树。
6. Markdown `*`/`**` 仍是强调界定符。表达式中的乘方 `**` 只在载荷已提取后具有运算符意义;为避免 Markdown 吃掉运算符或把载荷切碎,推荐把表达式放进行内 CodeSpan(例如 `` `n = n ** 2` ``)。检查器必须直接验证最终 AST,不能靠原始正则。
7. CommonMark 的“紧凑/松散列表”、换行、嵌套列表及标点有复杂规则;定义的是**AST 的结构匹配**,而不是缩进空格个数。提供 `marqdo inspect --ast` 展示实际节点。
8. 数学公式与代码围栏**不因为位于函数内部而执行**。可通过所属函数的标题范围生成地址,但不能执行其内部文本。
8. 数学公式与代码围栏**不因为位于 Exec 标题内部而执行**。行内 `Code` 是载荷载体,围栏代码块是惰性 `Code` 值,两者不可混同。
9. 控制块(`if`/`while`)只能在同一标题的直接正文内闭合;跨 H3/H4 或跨越 Knowledge 分节的 `end` 必须分别报 `sem.unclosed_control` 与 `sem.unmatched_end`,不得跨边界“修复”。
---
@@ -117,21 +124,21 @@ Marqdo 想要实现的不是 Python 的 Markdown 版本,也不是 Jupyter Note
### 2.2 可选形式参数声明
**[D]** 默认允许从执行表达式静态推断参数,**不能**从自然语言、行内代码提及、标题措辞推断。**[P]** 若需要稳定的公开 API,在函数标题后第一个段落用单独的行内代码跨度声明可选签名:
**[D]** 默认允许从执行表达式静态推断参数,**不能**从自然语言、行内代码提及、标题措辞推断。**[P]** 若需要稳定的公开 API,在函数体第一条直接执行载荷写 `params(...)` 声明(RFC-LANG-0002):
```markdown
## 圆面积
## @exec 圆面积
`(r: Number, precision: Int = 2)`
`params(r: Number, precision: Int = 2)`
按照圆面积公式计算。
**结果 = pi * r * r**
`结果 = pi * r * r`
*结果*
`return 结果`
```
**固定规则**:仅标题后的第一个合法 `Paragraph(CodeSpan)` 可是签名;其他位置的代码跨度仍是普通符号。没有签名不代表需要从当前调用者环境中找变量。允许 `()`, 必选形参、类型注解、编译期常量默认值;首版不支持可变参数、解构形参、运行时默认表达式。
**固定规则(RFC-LANG-0002)**:`params(...)` 只能作为 Exec H2 的**第一条直接执行载荷**、最多一次、且不得位于列表或引用块内部;它是编译期元数据,不是运行时调用。其他位置的 `params` 报 `sem.signature_position`。允许必选形参、类型注解、编译期常量默认值;首版不支持可变参数、解构形参、运行时默认表达式。旧“函数标题后的独立 CodeSpan 即签名”规则已废止。
### 2.3 静态参数推断(必须区分于动态作用域)
@@ -148,10 +155,10 @@ Marqdo 想要实现的不是 Python 的 Markdown 版本,也不是 Jupyter Note
统一调用形式:`函数(...)`;零参数仍写 `函数()`;位置参数只对显式签名开放,具名实参允许覆盖声明顺序,禁止重复实参、未知实参、多余位置参数。参数表达式**按源码出现顺序、恰好求值一次**,再绑定到被调用函数的形参槽;调用者帧和被调用者帧隔离。默认实参值仅在缺失时使用,显式 `None` 不等于未提供。参数绑定失败必须按规范传播错误。
- `> 打印("你好")`:`BlockQuote` 表示调用语句;执行后结果丢弃,必要时由执行记录保留。
- `**面积 = 计算圆面积(r=3)**`:在粗体执行节点中的表达式调用,保存返回结果。
- `> [计算面积](./math.mq.md#计算面积)(r=3)`:调用被 Markdown 链接引用的目标;需对 Link 与后缀参数进行 AST 级匹配,不按行字符串拼接。
- `*面积*`:`Emph` 返回整个函数,嵌套在分支或循环中也必须向外传播到函数帧;不得只退出局部列表项。
- `> **打印("你好")`:调用来自载荷内的表达式,引用块本身不是调用;执行后结果丢弃,必要时由执行记录保留。
- `**面积 = 计算圆面积(r=3)**`:在粗体载荷中的表达式调用,保存返回结果;也可用 `` `面积 = 计算圆面积(r=3)` ``。
- `[计算面积](./math.mq.md#计算面积)`:只是静态引用;跨模块调用必须在显式导入/锁定闭包内以表达式调用完成,需对 Link 目标与后缀参数进行 AST 级匹配,不按行字符串拼接。
- `**return 面积**`、`*return 面积*` 或 `` `return 面积` ``:显式 `return` 退出整个 H2 函数,嵌套在分支或循环中也向外传播到函数帧;斜体/粗体本身不再表示返回。
- 函数自然结束返回 `Unit`;`None` 是显式空值,`Unit` 是无结果,二者必须区分。
- 初期**不支持同名重载**;默认值、命名空间和语言类型均不参与模糊候选选取。支持函数递归与互递归。
@@ -165,49 +172,41 @@ Marqdo 想要实现的不是 Python 的 Markdown 版本,也不是 Jupyter Note
### 3.1 表达式是小型独立文法,不是第二套 Markdown
**[D]** Markdown AST 确定“哪里可以执行”;执行节点的文本载荷使用一个**独立、严格、很小的表达式解析器**。不得把整个 `.mq.md` 转译成 Python。表达式候选文法(简化 EBNF):
**[D]** Markdown AST 确定“哪里可以执行”;每条执行载荷的文本使用一个**独立、严格、很小的语句/表达式解析器**(现行文法见 `spec/expression.ebnf`,权威为 RFC-LANG-0002)。不得把整个 `.mq.md` 转译成 Python。载荷必须恰好是一条语句并被完整消费:
```ebnf
Statement = Assignment | Expression ;
Assignment = Identifier, "=", Expression ;
Expression = OrExpression ;
OrExpression= AndExpression, { "or", AndExpression } ;
AndExpression = CompareExpression, { "and", CompareExpression } ;
CompareExpression = AddExpression, [ CompareOp, AddExpression ] ;
CompareOp = "==" | "!=" | "<" | "<=" | ">" | ">=" ;
AddExpression = MulExpression, { ("+" | "-"), MulExpression } ;
MulExpression = UnaryExpression, { ("*" | "/" | "//" | "%"), UnaryExpression } ;
UnaryExpression = ("not" | "-" | "+"), UnaryExpression | PowerExpression ;
PowerExpression = PostfixExpression, [ "**", UnaryExpression ] ;
PostfixExpression = Primary, { CallArgs | ".", Identifier | "[", Expression, "]" } ;
CallArgs = "(", [ Argument, { ",", Argument } ], ")" ;
Argument = [ Identifier, "=" ], Expression ;
Primary = Literal | Identifier | "(", Expression, ")" | ListLiteral | RecordLiteral ;
payload = statement , EOF ;
statement = if_stmt | else_stmt | while_stmt | end_stmt
| return_stmt | params_stmt | assignment | expression ;
if_stmt = "if" , expression ;
else_stmt = "else" ;
while_stmt = "while" , expression ;
end_stmt = "end" ;
return_stmt = "return" , expression ;
params_stmt = "params" , "(" , [ parameter_list ] , ")" ;
assignment = assign_target , "=" , expression ;
```
- 以上是**建议子集**,需要单独 `spec/expression.ebnf`、优先级表、歧义测试后冻结。`**` 既是 Markdown 强调界定符也是表达式乘方符号,但后者只出现在已提取的表达式载荷内部;若 Markdown 已把输入分割成意外 AST,使用行内代码跨度保证载荷保真。
- `if`/`while` 由显式 `end` 闭合,`else` 可选且最多一次;控制配对由前端独立控制栈完成,不属于表达式文法。没有 `elif`/`for`/`break`/`continue`/`try`/`end if`/分号多语句/行尾冒号。`if`/`while`/`return` 需要非空表达式并消费整个载荷。
- 表达式部分沿用既有优先级、字面量、调用、索引、赋值与 `Memory` 位置写入;`**` 只在已提取的载荷内是乘方符号,Markdown 强调边界仍由 Comrak AST 决定,必要时用行内 CodeSpan 保证载荷保真。
- 数值初期选择 `Int`(任意精度或严格有界的语义必须明确,**推荐任意精度 BigInt**)、`Float64`、`Bool`、`Text`、`None`、`Unit`、`List`、`Map`、`Table`、`Formula`、`Code`、`FunctionRef`、`Result`、`PluginHandle` 等。跨 ABI/JSON 边界保留类型标签和大整数的字符串形式,禁止 silently 精度损失。
- 只有 `Bool` 可用于条件。`and`/`or` 短路;数值除零、类型错误、未定义变量、越界均是结构化错误;禁止隐式字符串数字转换。字典访问和数组索引的越界语义必须标准化。
- 赋值为当前局部绑定更新,不隐式写入外层作用域;模块导入绑定只读。
### 3.2 控制流与程序区域
### 3.2 控制流与程序区域(RFC-LANG-0002)
**OrderedList = `Branch`**:项目按照 AST 顺序检查条件;首个为 `true` 的项执行一次。必须只有一个条件头 CodeSpan;描述文本可紧随条件,后续 Markdown 块皆可含叙述与执行。无匹配时继续下一节点。禁止先检查所有条件再选择(条件求值可能涉及错误;原则上条件必须是纯计算表达式,禁止在条件里产生 I/O 副作用)。
**载荷关键字,不是 Markdown 类型**:每条 Exec 载荷是一条语句;`if <expr>` 打开 `Branch`,`else` 切换最近尚未关闭的 `if` 的 else 分支,`while <expr>` 打开 `Loop`,`end` 关闭最近的 `if`/`while`,`return <expr>` 产生 `Return` 并退出整个 H2 函数。`else`/`end` 不产生运行时指令。控制栈在标题边界必须为空,`if`/`while` 不得跨标题闭合。
**BulletList = `Loop`**:每项代表一个 while-loop,条件每次迭代求值;条件必须是 `Bool`;循环体可含 Text、Paragraph、Strong、Emph、BlockQuote、嵌套 OrderedList/BulletList;每次执行都消耗 fuel。
**容器无语义**:`Paragraph`、有序/无序 `List`、`BlockQuote` 只是按源序递归提取合法载荷;`1.` 与 `-` 的替换、`>` 的包裹、列表紧/松属性都不改变指令。空容器投影为零语句,不是 `List([])` 值。
**BlockQuote = `Call`**:整体是一个 Call 结构;无效调用硬错误。调用表达式可以包括具名参数或标准 Markdown Link 目标。是否允许 `> [引用](#函数)` 无显式括号首版建议**不允许**。
**求值顺序**:同级块按 Markdown AST 文档顺序;段内载荷按源序;分支只执行命中臂;循环条件每轮重新求值;参数按源码出现顺序求值一次;函数帧隔离;导入模块无副作用初始化。错误传播停止当前控制流并保留调用栈/源码位置。
**Strong = `ExecStmt`**、**Emph = `Return`**:所有出现于程序上下文中的对应节点都按固定角色处理;AST 中 `Strong` 与 `Emph` 的拼写变体具有相同执行行为。返回沿所有嵌套控制流传播到所属函数边界。
**求值顺序**:同级块按 Markdown AST 文档顺序;段内执行节点按源序;分支只执行命中项;循环按项顺序执行;参数按源码出现顺序求值一次;函数帧隔离;导入模块无副作用初始化。错误传播停止当前控制流并保留调用栈/源码位置。
**基本示例**(M0.5 候选;新解释器尚未验证):
**基本示例**(RFC-LANG-0002 目标语法;前端尚未迁移,不能原样运行):
````markdown
# 数学
# @exec 数学
## 阶乘
## @exec 阶乘
数学定义为:
@@ -215,30 +214,32 @@ $$
n! = \begin{cases}1,& n\le1\\n(n-1)!,& n>1\end{cases}
$$
1. `n <= 1`,递归终点。
`if n <= 1`
*1*
`return 1`
2. `n > 1`,计算更小的问题。
`else`
**结果 = n * 阶乘(n=n-1)**
`return n * 阶乘(n=n-1)`
*结果*
`end`
## 倒计时
## @exec 倒计时
**n = 3**
`n = 3`
- `n > 0`,继续执行。
`while n > 0`
> 打印(n)
`打印(n)`
**n = n - 1**
`n = n - 1`
*n*
`end`
`return n`
````
上述阶乘函数通过自由变量推断形参 `n`,故自调用采用 `阶乘(n=n-1)`;倒计时的 `n` 在入口赋值后才读取,属于局部变量,不应被推断成参数。嵌套段落的强调节点在 Markdown AST 中须以金样确认。
上述阶乘函数通过自由变量推断形参 `n`(公开 API 推荐用 `params(n: Int)` 显式声明);倒计时的 `n` 在入口赋值后才读取,属于局部变量,不应被推断成参数。旧版“列表条件头 + 斜体返回”的示例属于历史设计,见 RFC-LANG-0002 §1.4 的迁移对照。
### 3.3 终止与图灵完备性
@@ -260,9 +261,9 @@ $$
|---|---|---|---|
| `Import` | YAML `import` | 模块加载/静态编译 | 否,模块只注册定义 |
| `Reference` | 普通 Markdown `[说明](目标)` | 文档索引阶段 | 否 |
| `Call` | `BlockQuote` 或执行表达式的 `CallExpr` | 静态解析并在执行时调用 | 是(函数有运行时行为) |
| `Call` | Exec 载荷中的表达式调用 | 静态解析并在执行时调用 | 是(函数有运行时行为) |
`[牛顿第二定律](#牛顿第二定律)` 只是引用;`> [计算作用力](#计算作用力)(m=2,a=3)` 才是明确调用。跨模块调用须在固定的模块/文件权限下解析,外网 HTTP 链接绝不通过导入自动下载和执行。外部文档链接可以纳入可选的网络知识索引,但**不得**改变本地程序求值。
`[牛顿第二定律](#牛顿第二定律)` 只是引用;明确调用必须在 Exec 载荷中写出表达式,例如 `` `print(计算作用力(m=2, a=3))` ``。引用块本身不再是调用,`> [计算作用力](#计算作用力)(m=2,a=3)` 若不处于 Exec 载荷内则只是文档引用。跨模块调用须在固定的模块/文件权限下解析,外网 HTTP 链接绝不通过导入自动下载和执行。外部文档链接可以纳入可选的网络知识索引,但**不得**改变本地程序求值。
### 4.2 规范身份与兼容
@@ -292,19 +293,21 @@ M0.5 的“知识单元”只等价于**可寻址标题范围 + 其正文 AST +
### 5.2 示例(候选 API)
````markdown
## 圆面积公式
## @knowledge 圆面积公式
$$
A = \pi r^2
$$
## 计算圆面积
## @exec 计算圆面积
依据[圆面积公式](#圆面积公式)代入半径。
**面积 = math.eval(ref("#圆面积公式"), {"r": r}, target="A")**
`params(r: Number)`
*面积*
`面积 = math.eval(ref("#圆面积公式"), {"r": r}, target="A")`
`return 面积`
````
`Formula` 可以按函数标题范围寻址。一个范围存在多段公式时,默认对象选择必须是明确规则(例如禁止无选择器,要求 `math.formulas(ref(...))` 列出候选;不选“第一条”)。数学可符号计算子集应该单独版本化。高级张量/量子数学可通过插件提供,不进入不可控的内核表面。
@@ -331,7 +334,7 @@ $$
### 6.2 例子(候选 API)
````markdown
## Python 校验
## @knowledge Python 校验
```python
import json, math, sys
@@ -339,11 +342,13 @@ args = json.load(sys.stdin)
print(json.dumps({"area": math.pi * args["r"] ** 2}))
```
## main
## @exec main
**结果 = foreign.call(ref("#Python 校验"), {"r": 2})**
`params(r: Number)`
> 打印(结果.area)
`结果 = foreign.call(ref("#python-校验"), {"r": r})`
`打印(结果.area)`
````
不能因为这个文件里存在 Python 代码围栏就自动创建 Python 进程;当 `foreign.call` 被执行且权限通过后才创建或复用受控内核。
@@ -441,9 +446,11 @@ import:
quantum: "@quantum"
---
# main
# @exec 示例模块
**面积 = geometry.圆面积(r=3)**
## @exec main
`面积 = geometry.圆面积(r=3)`
````
`capabilities` 是**项目提出的权限请求**,不等于宿主已批准的权限;导入和安装不能自动赋权。
@@ -766,7 +773,7 @@ marqdo sync --locked
- 固定 Markdown 解析器、扩展配置及版本;枚举节点级映射、惰性区域、表达式载荷抽取算法。
- 写 `spec/markdown-profile.md`, `spec/mapping.md`, `spec/expression.ebnf`, `spec/evaluation.md`。
- 先写 ≥100 个 AST 金样和负例:`Strong`/`Emph` 等价源码、嵌套列表、含叙述分支、标题边界、块引用合并、Markdown 中乘号/下划线/反引号、无效执行表达式必须硬错误。
- 先写 ≥100 个 AST 金样和负例:`Strong`/`Emph`/行内 `Code` 载荷等价源码、标题模式声明与继承、嵌套列表/引用容器(应为零指令或合法载荷)、控制块跨标题边界、Markdown 中乘号/下划线/反引号、无效载荷必须硬错误。
- 交付最小 `inspect --ast`,能输出节点、span 和解析来源。
- **验收**:AST 结构语义无歧义;任何执行位错误不静默退化为文本;渲染前后 Markdown 正常。
@@ -816,11 +823,11 @@ marqdo sync --locked
| 类别 | 目标 | 核心样例 |
|---|---|---|
| AST 金样 | Markdown AST 不被私有解析取代 | 等价粗斜体、混合段落、嵌套列表、引用合并 |
| 语义正例 | 正确程序可执行 | 参数、递归、嵌套分支/循环、调用值 |
| 语义反例 | 不进行启发式兜底 | 非法粗体、非法斜体、非布尔条件、错误调用 |
| AST 金样 | Markdown AST 不被私有解析取代 | 等价粗斜体、混合段落、嵌套列表、引用合并、`@exec`/`@knowledge` 前缀 |
| 语义正例 | 正确程序可执行 | 参数/`params`、递归、`if`/`else`/`while`/`end`、`return` 值 |
| 语义反例 | 不进行启发式兜底 | 非法载荷、悬空 `else`、缺失 `end`、非布尔条件、`params` 位置错误 |
| 不变性 | 叙述修改不影响程序 | 只改 Text/非执行资源,IR 与输出保持 |
| AST 等价性 | 等价 Markdown 结构语义相同 | `**a**` vs `__a__`、列表标记替代 |
| AST 等价性 | 等价 Markdown 结构语义相同 | `**a**`/`*a*`/`` `a` `` 载荷等价、列表/引用容器替换不改变指令 |
| 参数数据流 | 推断正确和诊断准确 | 自增输入、分支部分赋值、同名全局、错别字 |
| 引用/导入 | 可寻址、可复现 | 重名、断链、循环导入、模块版本冲突 |
| 数学 | 求值子集准确、超界报错 | 数值精度、导数、非法 LaTeX、不可解方程 |
@@ -907,7 +914,7 @@ Migration & rollout:
### 16.3 不允许的实现捷径
- 解析失败便把 `Strong`/`List` 当成普通 Markdown;
- 解析失败便把 Exec 载荷或容器当成普通 Markdown 并静默跳过;
- 用自然语言词(“若/执行/返回/对”等)决定是否启动控制节点;
- 为解决语法问题写大量文本级 regex 来覆盖 Markdown AST;
- 把 Marqdo 编译为 Python 作为唯一官方运行语义;
@@ -923,16 +930,18 @@ Migration & rollout:
## 17. 首批仍需通过 ADR 冻结的设计问题(不是阻止开发的借口)
> **状态更新(2026-11-21)**:本节 S-01/S-02/S-03/S-06 描述的旧映射冲突已由 [RFC-LANG-0002](architecture/RFC-LANG-0002-unified-semantics.md) 与 [ADR-0052](adr/0052-unified-markdown-semantics.md) 解决并废止;下表保留原始表述作为历史记录,其中“本文默认立场”不再是现行设计。仍未冻结的语义问题(参数推断策略、未来 `elif`/`for`/`break`、知识锚点等)需要单独的 Accepted ADR。
下列问题涉及语义精确性。团队可以按本文暂定规则编写参考实现和测试,但**不得在对外声称语言已稳定时留下空白**。
| ID | 问题 | 本文默认立场 | 冻结前需要的证据 |
| ID | 问题 | 本文默认立场(历史) | 冻结前需要的证据 / 现行状态 |
|---|---|---|---|
| S-01 | 直接运行 `Strong` 中的数学 `*` 与 CommonMark 强调冲突 | 强调强制执行,允许行内代码无损载荷 | Markdown AST 金样/错误诊断 |
| S-02 | 普通斜体与返回冲突 | 程序体中的 Emph 永远是返回,文档约束使用者 | 真实文档可读性与反例测试 |
| S-03 | `OrderedList`/`BulletList` 在无前缀 `#` 的顶层语义 | 只在函数体内执行;其他区域按文档处理 | 作用域与链接索引测试 |
| S-04 | 参数推断在部分赋值路径中的歧义 | 静态数据流;不确定时报错 | CFG 案例与 ABI 变更测试 |
| S-05 | 中文函数名与英文点号/括号表达式 | Unicode 标识符、固定优先级;含空格走链接引用 | 词法与调用金样 |
| S-06 | 用 Markdown Link + 参数列表作为调用 | BlockQuote 内接受 `Link + suffix`,固定 AST 形式 | 标准解析器真实解析结果 |
| S-01 | 直接运行 `Strong` 中的数学 `*` 与 CommonMark 强调冲突 | 强调强制执行,允许行内代码无损载荷 | 已由 RFC-LANG-0002 解决:载荷统一、建议用行内 CodeSpan;需 AST 金样 |
| S-02 | 普通斜体与返回冲突 | 程序体中的 Emph 永远是返回,文档约束使用者 | 已由 RFC-LANG-0002 解决:`return` 是关键字,斜体只是等价载体 |
| S-03 | `OrderedList`/`BulletList` 在无前缀 `#` 的顶层语义 | 只在函数体内执行;其他区域按文档处理 | 已由 RFC-LANG-0002 解决:列表只是容器,无隐含指令 |
| S-04 | 参数推断在部分赋值路径中的歧义 | 静态数据流;不确定时报错 | 仍需 ADR;CFG 案例与 ABI 变更测试 |
| S-05 | 中文函数名与英文点号/括号表达式 | Unicode 标识符、固定优先级;含空格走链接引用 | 仍需词法与调用金样 |
| S-06 | 用 Markdown Link + 参数列表作为调用 | BlockQuote 内接受 `Link + suffix`,固定 AST 形式 | 已废止:BlockQuote 不再调用;跨文件调用须显式导入/表达式调用,另有 ADR |
| S-07 | 多公式/代码围栏在同一章节下如何寻址 | 不自动选第一条;显式 selector/API | 引用稳定性与用户体验测试 |
| S-08 | 任意精度数值与跨语言/插件 JSON 边界 | BigInt 为精确整数、wire 有类型标签 | 数值和跨 ABI 测试 |
| S-09 | strict sandbox 不同平台的保障 | 不能落实就拒绝 strict | Linux/Win/macOS 安全功能矩阵 |
+12 -2
View File
@@ -1,6 +1,16 @@
# M0.6 扩展性审计:源码问题、改造路径与测试契约
> 源文档 `docs/NEXT_PHASE_IMPLEMENTATION.md` §1A 的审计摘录,供开发者按问题创建 PR;具体开发次序和发布门槛请看主文档。该审计之后的稳定化基线与 G-A/G-B/G-C 门槛见 `docs/CORE_STABILIZATION_AND_ECOSYSTEM_GATE.md`。
> **Historical / Superseded planning input.** 本审计是 M0.6 时期(审查基线
> `0116160`)的源码扩展性证据,不是当前语言语义或发布路线。语言的唯一现行
> 权威是
> [`docs/architecture/RFC-LANG-0002-unified-semantics.md`](architecture/RFC-LANG-0002-unified-semantics.md)
> ([ADR-0052](adr/0052-unified-markdown-semantics.md));下一阶段计划以
> [`docs/STABLE_RELEASE_IMPLEMENTATION.md`](STABLE_RELEASE_IMPLEMENTATION.md)
> 为准;执行状态见 [`docs/STATUS.md`](STATUS.md)。下文关于
> `Strong/Emph/List/Quote` 的“用户可见语义”表述描述的是已被 RFC-LANG-0002
> 废止的旧映射,仅作为当时审计记录保留。
> 源文档 `docs/NEXT_PHASE_IMPLEMENTATION.md` §1A 的审计摘录,供开发者按问题创建 PR;具体开发次序和发布门槛请看主文档。该审计之后的稳定化基线(历史)见 `docs/CORE_STABILIZATION_AND_ECOSYSTEM_GATE.md`;当前路线与发布门槛以 `docs/STABLE_RELEASE_IMPLEMENTATION.md` 为准。
## 1A. **扩展性审计(v3 增补,优先级高于后续阶段)**
@@ -98,7 +108,7 @@ Python/Node 可以继续作为内置 provider;新增 Julia/R/自定义内核
当前项目版本 `0.1.0`,**不对错误设计承担无限兼容义务**。开发者应优先保证未来长期正确性,而不是维护早期内部 enum 的表面稳定。推荐机制:
- **内部接口(可以直接破坏)**:`HostOp` 分发、`Builtin` 内部查找、`PluginSlot` 单 namespace、模块 loader、非公开 IR、CLI 命令内部结构;每次变更要有迁移 PR、替换后的测试和明确的 PR 说明。
- **用户可见语义(默认保护)**:`Strong/Emph/List/Quote` 语义、显式 YAML import、无括号调用的拒绝规则、链接不执行、函数参数推断等。若发现客观错误,也必须有 ADR、版本升级和一致性金样,而不能把“扩展性”当作随意改语法的理由。
- **用户可见语义(默认保护,现行映射见 RFC-LANG-0002)**:`@exec`/`@knowledge` 标题模式与继承、Exec H2 可调用性、`Strong`/`Emph`/行内 `Code` 单语句载荷等价、`if/else/while/end/return/params` 关键字、显式 YAML import、链接不执行、函数参数推断等。若发现客观错误,也必须有 ADR、版本升级和一致性金样,而不能把“扩展性”当作随意改语法的理由。(本审计原文写作时仍是旧 `Strong/Emph/List/Quote` 映射,已被 RFC-LANG-0002 废止。)
- **版本化公共协议(只能显式破坏)**:Tool Protocol、Analysis/Trace、插件 ABI、Value Wire、Lock 格式。新旧同时可用或明确不兼容均可,但不能把 schema 名不变、字段意义变了当作修复。
- **冗余命令(可以有序削减)**:`init`、`inspect`、`fmt`、`plugin inspect`、`remove`、`lock` 等仅作为现有 0.x 行为,保留要以维护收益证明;公开主线禁止因此增加更多命令。删除时提供对应的文档内替代方式,避免把重要能力一起删除。
@@ -0,0 +1,510 @@
# RFC-CI-0001:Marqdo CLI 最新提交缺口审查、Linux Strict 验证基础设施与跨平台/性能发布门禁
> **状态:Proposed / 工程实施指引(未执行代码修改)**
> 仓库:`https://github.com/cflmy/marqdo-cli`
> 审查冻结提交:`32d8fe1f7afcdd7d7316c6692c442f5b26a24db2`(`master`,2026-11-21)
> 审查依据:该 SHA 的 GitHub Actions 运行 `44883104060`、各 Job 日志、源码、`docs/STATUS.md`、安全 VM 文档、CI 配置及性能脚本。
> **发布结论:DO NOT RELEASE。**
> 本文区分 **源码确认**、**该 SHA 远程实测**、**旧 SHA 记录** 和 **建议待实施**。本文件不宣称已在 Linux VM 上重新测试,不声称 Windows/macOS 已修复,不改动 GitHub 仓库。
## 0. 第一次阅读者应先知道的事情
Marqdo CLI 是 Rust 工作区实现的 Markdown 原生编程语言。当前 `.mq.md` 已经破坏性迁移到统一语义:多级标题可声明 `@exec` / `@knowledge`;仅 Exec H2 可调用;Strong/Emph/CodeSpan 承载相同的一条程序语句;载荷中的 `if/else/while/end/return/params` 决定程序语义;普通列表、引用块不再强制代表控制流。**这次语言语义在 Linux 本地已有 `9c1723d` 的 507 个通过测试记录,不等于最新 SHA 的跨平台与隔离安全验收。**
仓库还具有三个相关但独立的机制:
1. `pure/project/strict`:执行权限模式。`strict` L1 限定纯 Marqdo、原生数学与经授权的 stdout;不允许外部文件、网络、Foreign、Plugin、Trace 等扩权。
2. `marqdo-sandbox`:Linux 上用 Bubblewrap 构造用户、挂载、PID、网络等命名空间,带内存/进程/CPU/输出限制;在环境不可用时 **fail closed**。Mac/Windows 不提供该 strict 隔离。
3. `.github/workflows/ci.yml`:GitHub-hosted Linux + Windows + macOS 测试与打包;`.github/workflows/security-linux.yml`:需要具备命名空间能力的专用自托管 Linux VM 安全测试;`.woodpecker.yml`:Linux Docker 常规开发测试。**Linux 容器通过 Cargo 测试,不等于 Linux strict 隔离通过。**
本文目标:修复当前实际阻断,可靠地在 Linux 运行/编排严格隔离测试,取得同 SHA 的各平台证据,建立不被不稳定主机噪声污染的性能回归与发布体系。**本方案不修改语言语法,不迁出仓库标准库,不重新发明 Sandbox。**
## 1. 最新 SHA 的证据:不是简单的“UNVERIFIED”
### 1.1 该提交对应的公共 CI
- GitHub Actions:`https://github.com/cflmy/marqdo-cli/actions/runs/44883104060`
- 执行时间:2026-11-21 08:03–08:05 UTC,`head_sha=32d8fe1...`。
- `schema instance validation + consumers`:**PASS**。
- `fmt + clippy + test (Linux x86_64)`:**FAIL**,fmt 与 clippy 通过,`cargo test --workspace` 中 `strict_mode_enforces_isolation_or_refuses_closed` 失败。报错为 `stdout is not a single JSON document: EOF while parsing a value at line 1 column 0`,即在 strict `--json` 失败路径读到空 stdout。
- `runtime contract (windows-latest)`:**FAIL / compile E0425**:`crates/cli/src/commands/run.rs:67,70` 调用了只在 Linux 编译的 `marqdo_sandbox::marker_present`、`verify_active`。
- `runtime contract (macos-latest)`:**FAIL / compile E0425**:同上。
- `release candidate artifact`:**SKIPPED**,因此无本 SHA 的候选包验证。
- `Linux strict-security (G-C)` disposable VM:**本 SHA 无远程通过记录**。历史 VM 在旧源码 `70d916d` / `8af7262` 有 7/7 对抗案例证据,不可沿用为本次提交验收。
**重要区别:** Windows、macOS 在本 SHA 上“已经运行但失败”,应标为 `FAIL`;strict VM 未运行,才标为 `UNVERIFIED`。对 Linux GitHub-hosted 普通 runner 的失败,不能通过直接重跑旧 VM 结果消除;必须修复 CLI JSON 协议的失败路径。
### 1.2 精确定位与原因
| ID | 代码/证据位置 | 已确认问题 | 后果 | 优先级 |
|---|---|---|---|---|
| B01 | `crates/cli/src/commands/run.rs:67/70`;`crates/marqdo-sandbox/src/lib.rs:15–16` | CLI 直接引用 `#[cfg(target_os="linux")]` 导出的函数 | Mac/Windows 无法编译 | **P0** |
| B02 | `crates/cli/src/commands.rs::launch_strict_process`;`crates/cli/tests/integration_cli.rs` | strict 监督器/启动失败与 JSON 输出路径没有统一规范;非零退出或 `Err` 可能只写 stderr | `--json` 消费端收到空文档,本 SHA Linux CI 失败 | **P0** |
| B03 | `strict_is_available` 与实际 `run` 的独立探测 | probe 与 launch 有时间窗口;在资源波动主机上条件可能变化 | 测试易依赖时序,容易把可用性与拒绝行为混为一谈 | **P0** |
| B04 | `.github/workflows/security-linux.yml` | 只有手动和每周计划触发,依赖 `self-hosted,linux,marqdo-security` 标签 | 未形成每个候选 SHA 自动且可信的 G-C 证据 | **P0** |
| B05 | `.github/workflows/ci.yml` 和 `docs/STATUS.md` | Windows/macOS 已存在 GitHub-hosted 矩阵,但部分文档仍描述未配置 runner | 文档状态与实际仓库不一致,误导发布判断 | **P1** |
| B06 | `docs/architecture/CURRENT.md` 第 3 节接口表 | 新的 fs request/DNS/TLS 等“已实现”行与旧的“未实现”行同时存在 | 当前架构单一事实源发生自相矛盾 | **P1** |
| B07 | `COMPATIBILITY.md`、`docs/ci.md`、`docs/NEXT_RELEASE_CI_CONVERGENCE.md` | 含旧 SHA 基线、过时的 Windows/macOS runner 状态 | 使用者无法判断当前 SHA 是 PASS/FAIL/UNVERIFIED | **P1** |
| B08 | `tools/perf/bench.py` | 仅 7 个样例、每项 3 次、无隔离或稳定性量测、结果校验仅成功标记 | 只能作旧 P6 基线,不能作性能回归门禁 | **P1** |
| B09 | `docs/security-vm.md` | VM 文档存在,但镜像使用 `current` 可变地址、运行者可选手动拷贝目录 | 可复现性和源码证据链不够强 | **P1** |
| B10 | `tests/security/run-lane.sh` | 具备 7/7 对抗矩阵和正向 CLI 合约,但没被同 SHA 发布总门禁严格消费 | 老 VM 通过记录不能保证新发行版有效 | **P1** |
### 1.3 不能误报为缺失的能力
- `@exec/@knowledge`、统一载荷、v2 语句解析、当前 IR、Linux 本地语法测试:已有实现与旧同日记录;应在本 SHA 重测,而不是重新实现。
- `stdio/stdin/stdout/stderr`、文件/目录能力、JSON/CSV、HTTP 字节协议、DNS/TCP/UDP/TLS、环境/时间/随机/进程等:`docs/architecture/CURRENT.md` 记录有实现和产品测试。不要因旧文档片段再次宣布“全部缺失”。
- Linux strict L1 的 Bubblewrap Supervisor 与 7 项对抗测试已经存在:需求是 **当前 SHA、可信 VM、真实执行、不跳过、可追溯结果**。
- Windows/macOS strict 不是本次目标:不应为了过测试而谎称提供操作系统隔离;在这些系统上仍必须明确拒绝 `--mode strict`。
## 2. 推荐目标架构:四种 lane、一个 SHA、一份发布裁决
```text
Git commit SHA 32d8fe1...
├── G0: Linux GitHub-hosted / Woodpecker: fmt, clippy, compile, unit, integration, docs, stdlib
├── G1: Windows 原生 runner: 编译、通用语义、资源与 CLI;strict 必须 fail-closed
├── G2: macOS 原生 runner: 编译、通用语义、资源与 CLI;strict 必须 fail-closed
├── G3: disposable Linux VM: Bubblewrap 真隔离、无权限逃逸、正反路径、L1 对抗矩阵
└── G4: 固定 Linux 性能节点: 正确性校验 + 性能统计 + 固定硬件可比性
│
└── Gate Aggregator:校验每项证据 tested_sha == 目标 SHA
├── any failure / missing / skipped / SHA mismatch -> DO NOT RELEASE
└── all required PASS + clean-install + approval -> RELEASE CANDIDATE
```
**职责分离:** G0/G1/G2 验证行为与可移植性,G3 验证隔离**是否成立**,G4 验证实现变化是否引入显著性能回退。不能把 G0 的 namespace 失败称为 G3 已失败;也不能把 G3 的 QEMU/TCG 时间当成性能指标。
## 3. P0 修复一:跨平台条件编译,绝不伪造隔离
当前 `crates/marqdo-sandbox/src/lib.rs` 只在 Linux 导出 `marker_present()` 和 `verify_active()`,但 `commands/run.rs` 无条件调用。**Rust 中 `if cfg!(target_os="linux")` 不会阻止另一分支被类型检查**,不能这样修。
**推荐将平台判断集中在 sandbox crate,提供单一公共 API:**
```rust
pub enum StrictChildState {
NotChild,
VerifiedChild,
}
pub fn inspect_strict_child() -> Result<StrictChildState, String> {
#[cfg(target_os = "linux")]
{
if !marker_present() { return Ok(StrictChildState::NotChild); }
verify_active().map(|_| StrictChildState::VerifiedChild)
}
#[cfg(not(target_os = "linux"))]
{
// On non-Linux, even a forged marker must never authorize strict.
if std::env::var_os("MARQDO_STRICT_ACTIVE").is_some() {
return Err("strict active marker is unsupported on this platform".into());
}
Ok(StrictChildState::NotChild)
}
}
```
这是一段 **API 设计草案**,实施时应检查模块导入与类型可见性,不可未经编译直接粘贴交付。`run.rs` 只处理 `NotChild/VerifiedChild/Err`,所有 Linux 名称只出现在 sandbox 的 `#[cfg]` 区域。`VerifiedChild` 仍必须由现有 PID 1 / uid map / NoNewPrivs / CapEff / net namespace 验证取得,不能仅凭环境变量。
**验证清单**:
1. Linux `cargo check --workspace --all-targets --locked`;Mac 和 Windows 使用各自原生环境运行同样命令。
2. `cargo test --workspace --locked --no-fail-fast` 在三平台编译并执行;禁止通过 `#[cfg]` 跳过所有跨平台行为测试。
3. 两个非 Linux 平台:`marqdo run examples/hello.mq.md --mode strict` 非零、清晰错误、不出现任何用户程序输出;伪造 `MARQDO_STRICT_ACTIVE=1` 同样拒绝。
4. Linux:原有 strict 子进程验证与伪造 marker 拒绝保持不变。
5. `--json` 模式无论三平台何种拒绝路径都产生 **一个**合法 JSON 错误文档。
## 4. P0 修复二:strict 的 `--json` 合约和 probe→launch 时序
### 4.1 目前错误具体在哪里
`commands/run.rs` 调用 `launch_strict_process(&root, file, settings)`;其在 `commands.rs` 使用 `marqdo_sandbox::launch_strict()`,正常时透传子进程 stdout/stderr,错误时只 `eprintln!("error: ...")`。同时 `launch_strict()` 的封装可能将 Bubblewrap 的非零退出原样向上传递。这样如果 runtime probe 曾可用,但实际 bwrap 无法建命名空间,`--json` 就可能收到空 stdout,并触发当前 CI 中的 `json_stdout` panic。
**这里必须区分两种失败:**
- **控制平面失败**:probe、supervisor 启动、namespace 构造、超时、捕获输出失败或没有可验证的子进程输出。必须 fail closed,不能运行项目;在 `--json` 输出规范化错误。
- **程序平面失败**:真正进入 verified strict child 并返回用户程序错误。应保留子进程输出的合法 JSON,不得再包裹一份 JSON;真实程序失败码应保留。
### 4.2 实施方案
使用已有 `launch_strict_capture` / `StrictOutcome` 做父进程输出路由,或将等价的捕获接口用于 CLI;不要把结构化输出完全留给失败中的 bwrap 进程。
- `launch_strict_process` 接收 `json: bool` 与 `command_name: &str`,所有控制平面错误统一走 `emit_strict_failure_json`。输出至少:`{ "command":"run", "success":false, "error": "...", "code":"strict.launch_failed" }`。最终字段与 `--json` 既有对外 schema 保持兼容;不要在输出中泄露宿主路径、用户数据或不受控的长 stderr。
- `StrictOutcome` 是成功运行子进程后的字节捕获,不表示已经完成隔离验证。遇到非零退出而没有合法 JSON 时,父进程应按 supervisor failure 或受控非 JSON 程序错误规范化,不得在 strict `--json` 模式输出空内容或多个 JSON。
- 一旦子进程产生合法程序响应,父进程转发 **恰好一份** JSON;拒绝 stdout 附带无关文本。`--json` 下任何 CLI 错误路径都必须保持同一协议约定。
- 记录内部诊断区分 `strict.probe_unavailable`、`strict.spawn_failed`、`strict.namespace_failure`、`strict.child_invalid`、`strict.timeout`、`strict.resource_limit`,但对外要避免泄漏宿主环境敏感信息。错误码命名在 ADR 中冻结。
- 对 `probe OK → launch 失败`、`bwrap 退出 1 且 stdout 为空`、`spawn 返回 Err`、`strict child 报运行时错误`、`超时/信号退出`分别注入故障测试,校验退出码、结构化错误、**没有任何用户代码副作用**。
### 4.3 重写易受时序影响的测试
当前 `strict_is_available()` 会单独 `inspect --platform`,再根据这个独立时间点判断后续 `run`;这是合理探测但不能当永久保证。
拆成:
- **每个平台都必须跑的 fail-closed contract**:使用可注入 supervisor 的单元/集成测试,不依赖宿主是否允许 namespace;故障必须拒绝且输出合法 JSON。**不能以 `if unavailable { skip }` 作为通过。**
- **普通 Linux hosted runner 的 smoke**:如果环境不支持 Bubblewrap,验证 `strict unavailable` 的标准拒绝;如果支持则执行正例;如果 probe 后能力丢失,验证拒绝路径,不允许 fallback。
- **专用 G3 VM 的强制正例**:显式要求 `strict available`、`run --mode strict` 必须成功、7 项对抗案例必须全部执行;任意 skip 就失败。现有 `MARQDO_REQUIRE_STRICT_SECURITY=1` 已是正确机制,应继续沿用。
**绝不能**为使 GitHub-hosted runner 变绿而移除 strict 测试、把 JSON 断言降级为“只要退出非零”、关闭用户命名空间保护或将 strict 自动降级为 project 模式。
## 5. Linux disposable-VM strict lane 的现实实现
### 5.1 为什么优先 VM 而不是 Docker
`bwrap` 需要新建用户、挂载、PID、网络命名空间。普通 Docker CI 可能已经受 seccomp/AppArmor/容器权限限制。即使 `cargo test` 在容器通过,也不能说明用户进程能在同样条件下启动新的命名空间。仓库原有 `docs/security-vm.md` 与 `tests/security/run-lane.sh` 已用 **普通用户** 在 Ubuntu disposable VM 中完成历史 7/7 验证;应在此基础上工程化。GitHub-hosted 普通 Ubuntu runner 对本项目可变的 namespace 资源也不稳定,不应承担强制 G3 正向证据。
### 5.2 建立两类 Linux 机器
**Security VM(G3)**:每个待发布 SHA 启动一台临时 Ubuntu 虚拟机;普通用户、不暴露主机秘密、Bubblewrap/AppArmor 按发行版官方配置;尽可能 KVM 加速,缺少 `/dev/kvm` 时可用 QEMU/TCG 完成隔离**正确性**验证,但 TCG 时间不能用作性能依据。VM 使用独立可销毁的 qcow2 overlay;每次任务销毁整个 overlay,不复用可能被污染的 writable home/cache。
**Performance runner(G4)**:独立固定的 Linux 硬件或 KVM VM;保持 CPU 型号、微码、内核、Rust toolchain、构建参数、负载条件与发行版一致;不参与 untrusted PR 与安全对抗代码;不与 G3 共用性能基线。G4 可以在较稳定的硬件上运行,不必跑 Bubblewrap 对抗测试。
### 5.3 可复现实例:Host、Guest、检查与测试
推荐 Ubuntu 主机:
```bash
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
qemu-system-x86 qemu-utils genisoimage openssh-client
# 若具备 KVM:
ls -l /dev/kvm || true
```
**重要:** 建议根据仓库 `docs/security-vm.md` 创建 cloud-init NoCloud seed 和独立登录密钥;下列为**构建新镜像的示意**,不是在已有机器直接清盘的命令。网络下载的 Ubuntu cloud image 应事先下载并对照官方发布校验值,记录实际 `sha256sum` 与 `uname -a`,而不是永久依赖 `.../current/...` 可变目标。
```bash
# 以下路径应在专用工作目录内执行,不可对生产盘运行。
qemu-img create -f qcow2 -F qcow2 -b ubuntu-base-pinned.qcow2 \
strict-${SOURCE_SHA}.qcow2 24G
# 通过事先准备的 cloud-init seed(普通 dev 账号)启动隔离 VM;
# KVM 可用则 -accel kvm;否则 -accel tcg,thread=multi。
```
在 **guest** 内,管理员只为初次准备镜像安装系统依赖;测试使用普通 `dev`:
```bash
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
bubblewrap build-essential python3 git curl ca-certificates
id -u # 测试时必须不是 0
uname -a
bwrap --version
systemd-detect-virt --container || true
cat /proc/self/uid_map
findmnt -t cgroup2 || true
bwrap --unshare-user --unshare-net --ro-bind / / -- /bin/true
```
**不能**全局禁用 AppArmor、调低安全约束或用 `sudo cargo test` 来“让测试通过”。Ubuntu 版本对 unprivileged user namespace 的策略不同,遵循 `docs/security-vm.md` 的 packaged Bubblewrap/AppArmor 配置;如果机制不可用,G3 标为 **FAIL/UNAVAILABLE**,修环境而不修掉测试。
为保证准确的 SHA 对应关系,宿主只把目标 `git archive` 或固定提交的 checkout 传入 guest。推荐由受信任的管理端构建归档:
```bash
export SOURCE_SHA=32d8fe1f7afcdd7d7316c6692c442f5b26a24db2
# 在 Git repo 且已 fetch 到 SOURCE_SHA 的前提下:
git rev-parse HEAD
git archive --format=tar "$SOURCE_SHA" > source.tar
sha256sum source.tar > source.tar.sha256
# 把 tar 和 sha256 通过受控 SSH/SCP 传入临时 VM;guest 验证后解包
```
在 **guest** 使用普通账户运行:
```bash
set -euo pipefail
export MARQDO_REQUIRE_STRICT_SECURITY=1
export CARGO_TERM_COLOR=never
# 必须提前验证固定源码的 SHA 和传输 SHA256
cargo test -p marqdo-sandbox --test security_linux --locked -- --test-threads=1
./tests/security/run-lane.sh
# 正式 RC 还应独立构建 --release --locked,并对 release binary
# 做 strict 正例、非法权限、伪造 marker、预算拒绝的 smoke。
```
`run-lane.sh` 已经会执行 `security_linux`,上例中单独再调用一次用于先快速定位对抗测试;正式 CI 可以去掉重复以节省时间,**但不能删去任何安全场景**。
### 5.4 安全测试矩阵:不能只测“启动成功”
必须保持现有 7 项:
1. 严格子进程 UID/PID1/NoNewPrivs/CapEff/uid_map/net namespace 真正生效。
2. 无法读到宿主 `/etc/shadow`、`/root`、其他用户目录,却能读挂载的 `/project` 许可内容。
3. 无法写宿主目录或只读项目;私有 `/tmp` 不影响宿主。
4. `/proc`、capability 提升、`setuid`、`mount`、再次 `unshare` 等路径被拒绝。
5. 非 loopback 出站网络及原始 socket 被拒绝。
6. 文件描述符、子进程数量受到监督。
7. 内存、CPU、墙钟、stdout 输出预算触发安全拒绝。
叠加当前语言/资源语义的新回归:`@knowledge` 中的任意可疑载荷绝不产生 Host effect,`@exec` 中未经授权 `resource.open(file/tcp)` 拒绝;`stdin/stderr` 的 strict 权限符合当前 L1 明确契约;`--allow-env`、`--allow-process`、`--allow-random`、`--allow-net`、Plugin、Foreign、Worker、Trace 均被严格拒绝;`MARQDO_STRICT_ACTIVE` 伪造标记失败;缺少 bwrap 与 probe→launch race 均 fail closed;`--json` 的成功、失败都恰好输出一个文档。
### 5.5 接入 CI 时最关键的安全边界
`security-linux.yml` 当前使用 `runs-on: [self-hosted, linux, marqdo-security]`,但 GitHub 官方安全指导特别提醒:**公开仓库的持久自托管 runner 会受到来自 PR 代码的严重威胁**。不要直接把常驻 runner 放在拥有 GitHub Token、SSH 私钥、内网资源或重要文件的 Linux 主机上。规范见 `https://docs.github.com/en/actions/reference/security/secure-use`。
建议:
- GitHub-hosted Windows/macOS/G0 **保持现有 hosted runner**,不必迁移到 Linux Docker。
- G3 仅接收经过保护分支审查后的 SHA 或由管理员指定 SHA 的手动验证;**不让任意 fork PR 在受信任自托管 VM 上执行**。
- 每个 G3 job 启动独立临时 VM(或使用生命周期真正一次性的自托管 runner);无持久磁盘、无主机挂载、无宿主 Docker socket、无 SSH/Git/云凭证、无发布 token;最小 `permissions: contents: read`,`actions/checkout` 使用 `persist-credentials: false`。
- Runner 注册凭证应由受信任的受控 orchestrator 短期签发,不应进入项目源码或客户机磁盘快照;执行后销毁整个 VM、归档日志和检查结果。若无法安全提供 runner,请先运行**独立 Linux VM 实验**并报告“本地 VM PASS、远程 G3 UNVERIFIED”,切勿伪造绿色 CI。
- 单独记录 `source_sha`、`runner_image_sha256`、`guest_kernel`、`bwrap_version`、`rustc -vV`、`git archive_sha256`、`runner_kind`、`outcome`、`artifact_sha256`、`run_url`。所有这些都应写入一个 schema 化的 `strict-evidence.json`。
**触发与绑定:** 当前 security workflow 只有 `workflow_dispatch/schedule`,无法自动证明发布候选 SHA。建议在可信 `master` push 后触发专用 VM job,或者由 release orchestrator 请求显式的 `source_sha`。**尤其注意:手动 workflow_dispatch 所在工作流的 `GITHUB_SHA` 不一定等于输入的被测试源码 SHA;必须分别记录 `workflow_sha` 与 `tested_sha`,并校验 checkout/归档等于 `tested_sha`。**
### 5.6 推荐 `security-linux.yml` 修改要点(示意,不是现成完整文件)
```yaml
name: Linux strict security lane (G-C)
on:
push:
branches: [master]
workflow_dispatch:
inputs:
source_sha:
description: "Exact reviewed commit SHA"
required: true
type: string
permissions:
contents: read
jobs:
strict-security:
if: github.event_name != 'pull_request'
runs-on: [self-hosted, linux, marqdo-security]
timeout-minutes: 45
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event_name == 'workflow_dispatch' && inputs.source_sha || github.sha }}
persist-credentials: false
- name: Bind tested SHA and run G-C
shell: bash
run: |
set -euo pipefail
tested_sha="$(git rev-parse HEAD)"
echo "TESTED_SHA=$tested_sha"
test "$(id -u)" -ne 0
./tests/security/run-lane.sh
# 后续调用安全证据脚本生成 strict-evidence.json,
# 包含 tested_sha、环境、结果、工件摘要与流水线链接
```
前提:**在 runner 连接 workflow 前已经完成 ephemeral VM provisioning 和信任隔离**。上述 YAML 本身不会使持久 self-hosted runner 安全;如果没有完成此基础设施,先保持手动隔离 VM 的证据并拒绝发布。
## 6. Windows/macOS:Linux 能做什么、不能做什么
**直接回答:不能仅在一台 Linux 主机上完成真实的 macOS 与 Windows 原生 CI 认证。** Linux 可负责调度、打包源码、跨目标检查部分编译问题、核验日志和合并结果,但不能用 `cargo check --target`、Wine、模拟执行替代真实原生 runner 对文件系统、动态库、路径分隔符、权限、线程、进程与协议的测试。
**最低成本方案:继续使用已经配置的 GitHub-hosted `windows-latest`、`macos-latest`。** 当前这两个 Job 已运行,问题是 `E0425`,不是“无 runner”;先修 B01,再运行同 SHA 相关后续提交的原生 matrix。
Windows 原生应覆盖:正常 `--mode project/pure`、CLI run/call/check、UTF-8 和 CRLF、锁与路径、标准库、Tool/Trace、非 Linux strict fail-closed;明确未支持的 Windows Native/Worker ABI 要验证 **拒绝而非莫名崩溃**。
macOS 原生应覆盖:上述 portable 语义、动态插件 host target、路径大小写与 `dylib`,非 Linux strict fail-closed。若有承诺 Linux+macOS Native 插件,应建立真实 dylib 调用与 ABI 错误测试;不要以“Cargo 编译成功”替代运行验证。
如果组织希望**所有设备统一由 Linux 内部调度**:Windows 可以在 Linux/KVM 上管理真实 Windows guest(需有效 OS 授权、镜像安全与许可),注册 guest runner;macOS 则应使用真实受支持的 Apple/macOS runner 或 GitHub-hosted macOS。不能在 Linux 上把 macOS 交叉编译当作“原生 lane”。
跨平台 CI 修复并绿灯后,才在最新提交上记录确切 Job URL、OS、架构、编译器、成功测试数量和 unsupported 能力列表。不要将这项证明等同于 G3 strict 安全证明。
## 7. 性能稳定性:单独建立可比的 Linux 性能基线
### 7.1 已有脚本的真实边界
现有 `tools/perf/bench.py` 用 `/usr/bin/time` 和 `time.perf_counter` 对七个 workload 各跑 **3 次**,输出中位运行时间与峰值 RSS。它检查进程退出码和 JSON `success`,但没有验证每个工作负载的正确输出、没有预热、没有环境噪声报告、没有隔离/固定 CPU、没有历史可比基线、没有性能回归置信区间。**这个脚本是可重复实验起点,不是长期性能稳定性保障。**
### 7.2 分离三种性能指标
| 类别 | 目标 | 宿主 | 测量方法 |
|---|---|---|---|
| 语言核心吞吐 | 解释器、表达式、内存/容器/标准库算法 | 固定 G4 Linux | workload 完整输出校验 + wall、CPU、RSS |
| CLI 用户体验 | `check/run/inspect` 冷启动、解析、载入 stdlib | 固定 G4 Linux | fresh process p50/p95、启动时间与加载时间;冷热分组 |
| strict 安全开销 | Bubblewrap probe、re-exec、映射/检查、严格模式运行时间 | 固定隔离性能节点(与常规 G4 区分) | 同输入 pure 与 strict 比值;只作性能评估,不替代 G3 安全证据 |
**不要在 QEMU/TCG 上做最终性能门禁**;其不稳定和极高模拟开销与解释器性能无关。若在 KVM guest 中对性能做长期追踪,固定相同 CPU 型号、vCPU 拓扑、资源配额和宿主负载;不要把不同物理机器或任意公共 GitHub-hosted runner 的耗时直接比较。
### 7.3 新性能基准工具要求
建议扩展 `tools/perf/bench.py` 或替换为 `tools/perf/runner.py`,但不要在本轮重构中引入第二套业务标准库。
1. **固定二进制**:`cargo build --release --locked -p marqdo-cli`,记录 SHA、`rustc -vV`、`Cargo.lock` digest、release profile 与二进制 SHA256。基准期间禁止重新编译。
2. **确认正确性**:每个 workload 事先记录预期 wire JSON 的 digest / 值、stdout;任一结果不一致直接报 `CORRECTNESS_FAIL`,不得拿错误运行的“更快时间”计入性能。
3. **预热**:至少 5 次 warmup,正式每项至少 20 次采样,顺序采用固定随机种子或交错设计,以避免热度/频率漂移造成系统性偏差;冷启动任务单独统计,不混入温态样本。
4. **报告统计**:p50/median、p95、MAD/离散度、最大 RSS、CPU user/sys 时间;记录样本总数、超时/错误比例、环境状态与输出摘要。对极短小样本增加批量重复,避免计时器粒度主导结果。
5. **环境控制**:宿主上使用非 root 性能账户;固定 CPU affinity(如 `taskset -c <隔离核心>`,前提是该核心确实可用)、固定性能模式/频率策略仅在专用宿主且经管理员允许时设置。检查温度/节流、并发负载、CPU governor 和内核版本;若严重干扰则结果记 `NOISY` 重新采样,不偷偷删掉慢样本。
6. **性能变更统计**:同一机器同一配方上与预先批准的基线比较。新项目初期可以把 p50 回退 ≥15% 或峰值 RSS 回退 ≥20% 作为**需要调查的建议阈值**,并非绝对通用的自动回滚线。基线稳定并经历史数据校准之后再启用 hard gate;使用 MAD 或 bootstrap 区间避免偶发假阳性。
7. **跨平台构建不等于性能验证**:Windows/macOS matrix 重点测功能;若要宣称它们的性能稳定,另建各自稳定的性能 runner/baseline。
8. **双向检验**:对于有意大改解析器/Stdlib 的提交,记录吞吐增益、内存占用变化和 cold-start 代价,不因单项变快而忽略整体退化。
### 7.4 推荐工作负载(在已有 7 项之外补齐)
- **Parse/check**:真实大 Markdown 文件、1000+ heading/mode 切换、中文 Span、GFM 表格、复杂引用。
- **Runtime**:递归与迭代阶乘、hash table、大整数、`Memory` 增长/边界、嵌套 JSON、CSV。
- **Stdlib**:UTF-8 大字符串、JSON 反序列化、Table join/group/sort、HTTP 字节 chunked。
- **I/O**:stdout 多次小块/单次大块、文件读写、TCP loopback(单独统计 I/O 与 OS 影响)。
- **strict**:同一纯 workload 在 normal/pure 与 strict 上对照、supervisor 启动开销、受限输出;不在生产安全 runner 上跑压力极限长测试。
示例基准结果 schema(字段名称正式实施时冻结):
```json
{
"schema": "marqdo.perf.v2",
"source_sha": "<full 40-hex commit>",
"binary_sha256": "sha256:<digest>",
"platform_id": "linux-x86_64-fixed-node-01",
"os_kernel": "<uname -r>",
"rustc": "<rustc -vV>",
"profile": "release",
"workload": "json-parser",
"valid_output_sha256": "sha256:<expected-output>",
"warmups": 5,
"samples": 20,
"median_ms": 0,
"p95_ms": 0,
"mad_ms": 0,
"max_rss_kb_median": 0,
"status": "MEASURED"
}
```
数值 `0` 仅是占位示意,实际报告**不得使用假数据**;字段必须由真实测量填充。JSON 中不能把 guest / KVM / TCG 的结果混为同一基线。
## 8. 统一发布证据与状态治理
### 8.1 Evidence manifest:可验证的五元组
对每一个待发布提交,建立 `docs/evidence/ci/<SHA>/manifest.json`(或 CI artifact 同路径),包含:
```json
{
"schema": "marqdo.release-gates.v1",
"target_sha": "<full commit SHA>",
"gates": {
"linux": {"state": "PASS|FAIL|UNVERIFIED", "tested_sha": "<SHA>", "run_url": "<URL>"},
"windows": {"state": "PASS|FAIL|UNVERIFIED", "tested_sha": "<SHA>", "run_url": "<URL>"},
"macos": {"state": "PASS|FAIL|UNVERIFIED", "tested_sha": "<SHA>", "run_url": "<URL>"},
"strict_vm": {"state": "PASS|FAIL|UNVERIFIED", "tested_sha": "<SHA>", "run_url": "<URL>"},
"performance": {"state": "PASS|WARN|FAIL|UNVERIFIED", "tested_sha": "<SHA>", "run_url": "<URL>"}
},
"release_decision": "DO_NOT_RELEASE"
}
```
正式工具中应将 `state` 设计为枚举(而不是把以上 `PASS|FAIL...` 字符串当数据)。校验所有 `tested_sha` 与 `target_sha` 的相等性、run conclusion/job 状态、测试完整数量、artifact digest、构建二进制 digest,以及 `strict_vm` 的实际 VM/非 root/7/7 证据。**跳过、超时、取消、排队未启动、缺失、老 SHA 均不算 PASS**。
### 8.2 强制决策规则
- `G0` Linux CI **必须 PASS**;Mac/Windows 原生支持矩阵所承诺的测试必须 **PASS**。
- `G3` strict VM **必须 PASS**,才能发布包含 `strict L1 supported` 的候选。若选择移除 strict 产品宣称,应有独立的 API/版本策略与负责人批准,本 RFC 不把这当作自动逃生路径。
- 性能基线 G4 至少必须 `MEASURED` 且没有未解决的确定性大回退;未达到统计可比性的报告记 `UNVERIFIED` 或 `NOISY`,不能写 PASS。
- RC 构建与 clean-install smoke 绑定同一 commit SHA,包含 stdlib digest 验证、二进制 checksum 与完整 LICENSE/README/specs;平台未覆盖的功能必须明确标注 `UNSUPPORTED`。
- 发布最后由负责人审批;绝不让文档的 `PASS local` 自动变成 `PASS remote`。
### 8.3 清理文档状态漂移
只保留三个职责明确的事实源:
1. `docs/architecture/CURRENT.md`:唯一的**当前架构和 API 支持矩阵**。删除已经被后续实现推翻的重复旧行,不能在同一表同时写“已实现”和“未实现 DNS/TLS”。
2. `docs/STATUS.md`:**提交/SHA + 运行平台 + 结果 + CI URL/本地日志**,新数据写在顶部;历史章节明确标记旧 SHA。
3. `COMPATIBILITY.md` 和 `docs/releasing.md`:从前两者与 CI evidence 自动/受控整理支持声明,不允许复制过去版本的绿色状态。
`docs/ci.md` 要改掉“Windows/macOS 原生 runner 未配置”的绝对表述:GitHub-hosted matrix **确实已运行,本 SHA 编译失败**;Woodpecker 本身没有 Windows/macOS agent 是另一件事。`docs/NEXT_RELEASE_CI_CONVERGENCE.md` 标记旧基线为 Historical,不再作为当前 green 证明。`docs/security-vm.md` 给出当前 SHA 的 VM 重跑流程,旧数据只作复现参考。
## 9. 分阶段开发方案:具体文件、PR 与合并门禁
### PR-1:修复平台编译与 strict 结构化拒绝(最先做)
**主要文件**:`crates/marqdo-sandbox/src/lib.rs`、`crates/cli/src/commands/run.rs`、`crates/cli/src/commands.rs`、`crates/cli/tests/integration_cli.rs`、`crates/marqdo-sandbox/src/linux.rs`(仅必要的 outcome 分类与可测试注入)、相关负向测试。
**做法**:统一跨平台 `StrictChildState`、保证所有 `strict --json` 失败返回单 JSON、保持非 Linux fail closed、为 probe/launch race 注入确定性测试。禁止动语言解析器或 Host 的权限边界。
**合并门禁**:Linux fmt/clippy/test PASS、Windows/macOS 原生 matrix 通过编译和 portable tests;GitHub-hosted Linux namespace 受限情况下仍规范拒绝;证明 project 代码从未执行。
### PR-2:G3 Linux VM 可重建与本 SHA 安全证据
**主要文件**:`docs/security-vm.md`、`tests/security/run-lane.sh`、`tests/security` 下的回归场景、`tools/ci/strict-evidence.py`(新)、`.github/workflows/security-linux.yml`,以及 runner provisioning **受信任基础设施**(必要时独立私有仓库,不放生产 SSH key)。
**做法**:固定 VM 基镜像及 digest、无 root 测试、7/7 + 新语义负向案例、未运行零跳过、生成证据 JSON、按 SHA 触发并上传日志。Runner 一次一销毁。
**合并门禁**:`tests/security/run-lane.sh` 对 **当前最终候选 SHA** 真正 PASS;包含足够的测试日志、VM 环境摘要和运行 URL。Security VM 跑在容器内或以 root 通过不合格。
### PR-3:Linux 性能样本与可比基准
**主要文件**:`tools/perf/bench.py`(扩展或原位替换)、`tools/perf/workloads.json`、`tools/perf/expected/`、`tools/perf/README.md`、`docs/performance.md`(新)、独立性能 CI 工作流。
**做法**:固定 runner、Release 二进制 SHA、5 warmups/20 samples、先正确性后计时、p50/p95/MAD/RSS、环境 manifest;按可信历史样本建立 baseline。不得误把 TCG 模拟负载当发布性能。
**合并门禁**:全部 workload 正确、报告 schema 可机读、能重复运行并比较相同硬件同一构建配置;达到统计阈值的真实回退需调查解决。
### PR-4:按同一 SHA 聚合发布门禁并清理文档
**主要文件**:`.github/workflows/ci.yml`、`docs/STATUS.md`、`docs/architecture/CURRENT.md`、`COMPATIBILITY.md`、`docs/ci.md`、`docs/releasing.md`、`docs/security-gate.md`、`docs/evidence/`、`crates/cli/tests/docs_consistency.rs`。
**做法**:聚合每个平台最新通过状态,禁止 skipped/老 SHA 证据复用;删除文档重复冲突行;RC clean-install 与 `SOURCE_SHA` / stdlib digest 绑定。增强 docs consistency,以防出现“已实现 TLS”和“无 TLS”的同页冲突。
**合并门禁**:所有 release gate 都能明确输出 `PASS/FAIL/UNVERIFIED`,明确记录所测 SHA;不能手工把旧证据改成当前 PASS。
## 10. 必须增加的自动化测试清单
### 10.1 跨平台与 JSON
- [ ] Windows/macOS `cargo check --workspace --all-targets --locked` 和 `cargo test --workspace --locked` 均执行成功。
- [ ] 非 Linux 不编译 Linux 专用标记函数的直接访问;伪造 marker 不绕过保护。
- [ ] strict `--json`:无 bwrap、probe 拒绝、spawn 报错、namespace 建立失败、超时、被 kill、成功执行、执行期 RuntimeError、Foreign/Plugin flags 拒绝都只产生一份 JSON。
- [ ] 不能从 `--json` 错误路径输出未经编码的 Bubblewrap 文本作为完整标准输出。
- [ ] 逻辑相同的普通输出路径没有 stdout/stderr 渠道混乱;项目输出不得在拒绝后出现。
### 10.2 strict VM
- [ ] 本 SHA guest `id -u != 0` 且 `systemd-detect-virt --container` 未把它识别成被禁止的受限容器。
- [ ] strict probe 可用;7/7 对抗测试强制运行、全部通过、0 ignored/skipped。
- [ ] 正向 `run/call/json` 成功;无权限文件/网络/扩展申请 fail closed;伪造 marker 失败。
- [ ] `@knowledge` 中载荷无副作用,不突破权限;`@exec` 中不授权资源会拒绝。
- [ ] 同 SHA debug 与 release 二进制 smoke 均 PASS;退出码和 JSON 对外协议固定。
- [ ] 新镜像、运行日志、证据 manifest、`tested_sha`、run URL 全部可追溯。
### 10.3 性能与资源稳定性
- [ ] 现有七 workload 预期输出已验证,不仅是 `success=true`。
- [ ] 大型 Markdown cold check、复杂 JSON/CSV、Table、I/O、Memory、循环等有覆盖。
- [ ] 基准二进制和环境哈希固定;独立性能 runner 上建立至少三次不同日期/同硬件的基线后再冻结阈值。
- [ ] 报告至少包含 warmup/sample 次数、p50/p95/MAD、RSS、失败率、环境噪声信息。
- [ ] strict VM 对抗时延与普通性能指标分离;不会把 QEMU TCG 时间与 KVM/native 时间混比。
- [ ] release 命中明确阈值时需要 review,不能靠一条 noisy 采样直接判回归或通过。
## 11. 工程开发规范与禁止事项
- 修改安全边界必须先加不允许逃逸的负向测试;修改 CLI 错误协议先改测试契约再实现。
- 非 Linux 上不可编译的 symbol 只能存在于 `#[cfg]` 对应模块内,**不使用运行时 `cfg!` 规避**。
- 新安全机制默认 `DENY`,可用性由实际探测决定;probe 成功不代表后续 launch 永远成功。
- 严格模式不得自动降级为 `pure/project`,不得保留原宿主目录的真实写权限;不得全局关闭 AppArmor、seccomp 或 namespace 限制以取得绿色测试。
- 不在生产机/常驻 privileged Docker 直接运行对抗安全测试。普通业务测试与 G3 隔离测试分开维护。
- 不把 Secret、runner registration token、SSH key、环境变量凭证、完整宿主绝对路径打到公共日志或 evidence。
- 现有标准库坚持 `.mq.md`,本轮工作不新增专用 Rust JSON/CSV/打印内置算法,也不将 stdlib 强行拆仓库。
- CI 结果与性能证据要有固定 schema、SHA、文件摘要;无法核验的历史记录标记为历史,不声称本 SHA 通过。
- 文档必须能被新开发者按顺序运行;关键命令说明 host/guest、预期退出码与什么行为意味着 FAIL。
## 12. 最终验收与结论
**P0 阻断清单(全部完成才能讨论 RC)**:
1. 修复 `E0425`,Windows/macOS 原生 job 在当前候选 SHA 通过全部已承诺的 portable 语义与 CLI 测试。
2. 修复 GitHub-hosted Linux 上 strict 失败时 `--json` 输出为空的缺陷,证明 launch race 仍 fail closed。
3. 用真正的非 root disposable Linux VM 重跑全部 G-C 7/7 + 当前语义的 strict 正反例,产生 **本 SHA** 证据。
4. 引入同 SHA 证据聚合和 clean-install 检查,缺失或失败立即 `DO NOT RELEASE`。
5. 性能基准具备正确性校验与固定宿主比较能力;任何真实、可重复、未经批准的严重性能回退必须解决或公开记录为阻断项。
**当前评定**:语法重构不是当下主阻断;新的远程 CI 已经给出了精确故障,Windows/macOS **FAIL**、Linux 主测试 **FAIL**、G3 VM **UNVERIFIED**、RC **SKIPPED**。修复优先级应是 **P0 跨平台条件编译 → P0 strict JSON/失败分类 → P1 disposable VM 同 SHA 正向隔离证据 → P1 性能稳定性与 release evidence**。
### 参考源码与证据
- 最新审查 SHA:`https://github.com/cflmy/marqdo-cli/commit/32d8fe1f7afcdd7d7316c6692c442f5b26a24db2`
- 远程测试:`https://github.com/cflmy/marqdo-cli/actions/runs/44883104060`
- 跨平台直接引用问题:`crates/cli/src/commands/run.rs`,`crates/marqdo-sandbox/src/lib.rs`
- JSON 路由/测试:`crates/cli/src/commands.rs`、`crates/cli/tests/integration_cli.rs`
- 安全 VM:`docs/security-vm.md`、`docs/security-gate.md`、`tests/security/run-lane.sh`
- CI:`.github/workflows/ci.yml`、`.github/workflows/security-linux.yml`、`.woodpecker.yml`
- 性能:`tools/perf/bench.py`
- 实现状态与文档:`docs/STATUS.md`、`docs/architecture/CURRENT.md`、`COMPATIBILITY.md`
- GitHub 官方自托管 runner 风险说明:`https://docs.github.com/en/actions/reference/security/secure-use`
- Bubblewrap 项目:`https://github.com/containers/bubblewrap`
---
*备注:文档中列出的代码与 YAML 是设计和实施模板,未在当前环境对最新 SHA 执行编译、安全 VM、性能测试;通过状态只引用实际可见的 GitHub Actions 日志或仓库已有的历史记录。*
@@ -1,5 +1,23 @@
# RFC-CM-0002:Marqdo 最小计算内核、标准库自举与系统能力迁移
> **Design input / Proposed.** This RFC records target semantics, not
> implemented behavior. The implemented first slice is described in
> [`architecture/CURRENT.md`](architecture/CURRENT.md).
> **Syntax note — RFC-LANG-0002 is the current language contract.** The
> compute-kernel / stdlib-ownership / resource model in this RFC is still the
> current design input. Its `.mq.md` examples and Markdown-role wording were
> migrated to the single current syntax
> ([RFC-LANG-0002](architecture/RFC-LANG-0002-unified-semantics.md),
> [ADR-0052](adr/0052-unified-markdown-semantics.md)): `## @exec` heading
> modes, `params`, `return`, `if/else/end`, `while/end`, and one-statement
> payloads carried by `Strong`, `Emph` or inline `Code`. Any remaining old
> spelling (for example "`##/**/*` roles") is historical and is **not** an
> active language. The frontend now implements this syntax; examples that use
> APIs this RFC only proposes remain design targets, not implemented features.
> See [`STATUS.md`](STATUS.md).
> **状态**:Draft / 架构与迁移提案,未批准、未实施。所有代码均为目标语义草案,不能当作已运行示例。
> **日期**:2026-11-20
> **依赖与基线**:在《RFC-CM-0001:Marqdo 自主计算机器、可寻址内存与系统交互边界》之上细化,不代替其中的多内存区域与资源边界原则。
@@ -52,7 +70,7 @@ Marqdo 语义核心 资源调
- **G2 系统构造性**:只要基础 OS 资源端点已存在并获授权,Marqdo 应自行实现 `print/read_line/copy_file/HTTP parser/CLI commands` 等高层 API。
- **G3 内核稳定**:未来 100 个独立标准库函数,不意味着增加 100 个内置函数或触及 Rust 编译器/解释器。
- **G4 语义统一**:高级函数运行在与普通程序**同一个**值、内存、调用、错误、来源映射和权限体系下,没有“标准库专用魔法规则”。
- **G5 兼容现有 Markdown 语义**:`##/**/*`、链接/表格惰性、函数与模块寻址、执行顺序不被重写。
- **G5 兼容当前 Markdown 到程序的映射**:按 RFC-LANG-0002,执行由 `@exec` 模式与载荷关键字决定;链接/表格保持惰性;函数与模块寻址、执行顺序不被重写。
- **G6 工程品质**:至少能构造可扩展容器、网络字节协议、异构表格、持续交互工具,同时明确性能瓶颈和安全保证。
- **G7 明确安全边界**:内部虚拟内存权限与资源访问权限分别验证;不能把“运行在 Rust 内”误写为“自动内存安全”,也不能把普通 Worker 等同于 OS 隔离。
@@ -372,18 +390,18 @@ std.io.print(value)
- 标准输出内容与编译器日志、trace、`--json` 通道保持隔离。
- 首版可支持保持 `print` 完全一致,同时引入 `write_text` / `write_bytes` 作为标准库独立能力;用户未授权写标准输出时,仍必须拒绝。
示例(**目标 `.mq.md`,非已实现语法,`std` 对应正常模块导入**):
示例(**目标 `.mq.md`,按 RFC-LANG-0002 书写、非已实现语法,`std` 对应正常模块导入**):
````markdown
## 打印
## @exec 打印
先将任何可打印值格式化为文本,再编码为 UTF-8 字节。
**显示文字 = format.display(值)**
`显示文字 = format.display(值)`
**数据 = text.utf8_encode(显示文字 + "\n")**
`数据 = text.utf8_encode(显示文字 + "\n")`
> io.write_all(io.stdout(), 数据)
`io.write_all(io.stdout(), 数据)`
````
这些 `format.display`、`text.utf8_encode` 和 `io.write_all` 必须有真实的 `.mq.md` 定义;其最终字节写入依赖已有授权 `Stream.write` 而不是新的 `HostOp::PrintJson` 等。
@@ -529,40 +547,46 @@ F ∈ L(K,E)
**工程性能评估**:记录 `wall_time / peak_RSS / CPU / fuel / bytes_moved / allocations / copy_count`,与 Rust 原生和其他参考语言比较。初期不设无依据的“必须达到 80% 原生速度”的硬指标;若性能瓶颈来自逐字节 Rust↔Marqdo 调用/隐藏复制,先优化执行路径、批量传输、字节码/JIT,再讨论原生加速。任何 O(1) 原语若退化为 O(n) 隐式全表复制,必须单独报告。
### 7.6 三种验收样例(目标伪代码,非现有可运行程序)
### 7.6 三种验收样例(按 RFC-LANG-0002 书写的目标伪代码,非现有可运行程序)
**A. 用 Marqdo 实现堆分配器而不调用 Rust `malloc` 算法**:
````markdown
## 分配
## @exec 分配
内存管理器保存 region、next、free-list 等元数据。
1. `存在足够的空闲块`
> 复用空闲块并更新元数据
2. `没有足够的空闲块`
> 需要时增长(region)
> 更新区域中的尾指针
`if 存在足够的空闲块`
*分配得到的偏移量*
`复用空闲块并更新元数据`
`else`
`需要时增长(region)`
`更新区域中的尾指针`
`end`
`return 分配得到的偏移量`
````
**B. 打印与读取由标准库完成**:
````markdown
## main
## @exec main
**输入 = io.read_line()**
`输入 = io.read_line()`
**输出 = "收到:" + 输入**
`输出 = "收到:" + 输入`
> io.print(输出)
`io.print(输出)`
````
**C. 同一个 GFM 表格运行不同的 `.mq.md` 算法**:
````markdown
## 分析数据
## @exec 分析数据
| 名称 | 分数 |
| --- | ---: |
@@ -571,12 +595,12 @@ F ∈ L(K,E)
表格是数据,不在渲染时执行;函数通过显式数据引用取得逻辑 Table。
**结果 = table.filter(输入表, 过滤函数)**
`结果 = table.filter(输入表, 过滤函数)`
*结果*
`return 结果`
````
B/C 中的函数名、输入参数和文档表格访问均仅为目标接口示意,须在真正实现时遵守冻结的 `##` 函数声明、参数推断/导入和文档资源寻址规则。**不能为了示例好看就把尚不存在的语法当成现实。**
B/C 中的函数名、输入参数和文档表格访问均仅为目标接口示意,须在真正实现时遵守冻结的 `## @exec` 函数声明、参数推断/导入和文档资源寻址规则。**不能为了示例好看就把尚不存在的语法当成现实。**
---
## 8. 重构目标架构、仓库组织与实际代码改动
+38 -21
View File
@@ -1,16 +1,33 @@
# RFC-CM-0001:Marqdo 自主计算机器、可寻址内存与系统交互边界
> **状态**:设计提案(Draft,未实施,未批准)
> **日期**:2026-11-20
> **建议归档路径**:`docs/architecture/RFC-CM-0001-compute-machine.md`
> **验证基线**:`cflmy/marqdo-cli`,Git 提交 `a113d894962d8c00eb02efb42e0e73f8b084a1fa`(检索于 2026-11-20)
> **Design input / Proposed.** This RFC records target semantics, not
> implemented behavior. The implemented first slice is described in
> [`architecture/CURRENT.md`](architecture/CURRENT.md).
> **Syntax note — RFC-LANG-0002 is the current language contract.** The
> compute / memory / resource / ABI model in this RFC is still the current
> design input. Its `.mq.md` examples and Markdown-role wording were migrated
> to the single current syntax
> ([RFC-LANG-0002](architecture/RFC-LANG-0002-unified-semantics.md),
> [ADR-0052](adr/0052-unified-markdown-semantics.md)): `## @exec` heading
> modes, `params`, `return`, `if/else/end`, `while/end`, and one-statement
> payloads carried by `Strong`, `Emph` or inline `Code`. Any remaining old
> spelling (for example "`##` function, `**` execute, `*` return") is
> historical and is **not** an active language. The frontend now
> implements this syntax; examples that use APIs this RFC only proposes remain
> design targets, not implemented features. See [`STATUS.md`](STATUS.md).
> **状态**:设计提案(Draft,未实施,未批准)
> **日期**:2026-11-20
> **建议归档路径**:`docs/architecture/RFC-CM-0001-compute-machine.md`
> **验证基线**:`cflmy/marqdo-cli`,Git 提交 `a113d894962d8c00eb02efb42e0e73f8b084a1fa`(检索于 2026-11-20)
> **兼容性**:涉及 `Value`、表达式赋值、运行时、主机能力和线协议的变更,必须通过 ADR、规范版本与相应测试;本 RFC 不声明任何新接口目前可用。
---
## 摘要
Marqdo 的优势不应只是用 Markdown 排版代码,而应是:在不更换 Markdown 语法角色的前提下,以**自身定义的计算语义**构造算法、数据结构、运行时工具和知识制品。现有解释器已提供整数、浮点、文本、列表、映射、控制流和函数调用,但尚未提供**由 Marqdo 程序控制的可寻址字节存储**;其赋值左侧仅支持标识符,系统交互集中在一组固定 `HostOp` 与外部插件能力中。这妨碍了用 `.mq.md` 自己编写内存分配器、二进制协议、共享节点结构、虚拟机器等低层算法。
Marqdo 的优势不应只是用 Markdown 排版代码,而应是:在不更换自身计算模型的前提下,以**自身定义的计算语义**构造算法、数据结构、运行时工具和知识制品。现有解释器已提供整数、浮点、文本、列表、映射、控制流和函数调用,但尚未提供**由 Marqdo 程序控制的可寻址字节存储**;其赋值左侧仅支持标识符,系统交互集中在一组固定 `HostOp` 与外部插件能力中。这妨碍了用 `.mq.md` 自己编写内存分配器、二进制协议、共享节点结构、虚拟机器等低层算法。
**提议:以“托管值 + 多个可增长的线性内存区域 + 不透明系统资源”为计算机器的三个基础领域。**
@@ -40,12 +57,12 @@ Marqdo 必须能在不借助 Python、JavaScript 或宿主语言提供专门算
### 1.3 设计原则
**P1 语义最小而充分**:首先增加无法由现有表达式自然实现的基础机制,不先增加高级 API 或新关键词。
**P2 高级能力在语言中生长**:新增算法/数据结构不修改解释器、不修改 `HostOp`、不需要重新编译 CLI。
**P3 读写位置统一**:索引读取和索引赋值遵循同一类“位置”语义;`Memory` 的读取/写入是基础例子,普通列表的值语义不得无声改变。
**P4 内外边界诚实**:内存操作是语言计算;操作系统 I/O 依赖权限、平台接口和底层实现。
**P5 可读性**:普通 Markdown 文档仍然可读;`##`/`**`/`*`/列表/blockquote 的现有角色不因本 RFC 改变。
**P6 安全与可验证性**:将可验证的保证限定在区域边界和权限边界;不对自定义分配器的块级错误作虚假保证。
**P1 语义最小而充分**:首先增加无法由现有表达式自然实现的基础机制,不先增加高级 API 或新关键词。
**P2 高级能力在语言中生长**:新增算法/数据结构不修改解释器、不修改 `HostOp`、不需要重新编译 CLI。
**P3 读写位置统一**:索引读取和索引赋值遵循同一类“位置”语义;`Memory` 的读取/写入是基础例子,普通列表的值语义不得无声改变。
**P4 内外边界诚实**:内存操作是语言计算;操作系统 I/O 依赖权限、平台接口和底层实现。
**P5 可读性**:普通 Markdown 文档仍然可读;按 RFC-LANG-0002,执行语义只由 `@exec` 模式与载荷中的 `if`/`else`/`while`/`end`/`return` 关键字决定,普通 `##`/`**`/`*`/列表/blockquote 不内建指令角色。
**P6 安全与可验证性**:将可验证的保证限定在区域边界和权限边界;不对自定义分配器的块级错误作虚假保证。
**P7 零隐式权限**:能够导入模块、持有字符串或解析文档,不代表拥有文件、网络或设备访问权。
## 2. 现有仓库事实及需要保留的设计
@@ -339,28 +356,28 @@ Memory 支持字节缓冲区与数据布局,但 GPU 计算仍需要执行后
这一方案故意**只修改最少的表达式语义**。
- `##` 函数、`**` 执行、`*` 返回、Markdown 列表与引用规则继续维持规范。
- 函数由 `## @exec 名称` 声明;执行/返回/条件/循环由载荷中的 `if`、`else`、`while`、`end`、`return` 关键字决定(RFC-LANG-0002);Markdown 列表与引用只是文档容器。
- `[]()` 是**文档/知识的静态寻址**;动态 Memory 的字节偏移属于执行期位置,两者不强行共享 URI 语法。
- 普通 GFM 表格是文档值;可用于初始化/展示内存快照,但不会自动变成内存分配器。
- `memory.new(...)` 与 `resource.open(...)` 都可按普通函数调用表达;不新增 `memory`、`pointer`、`asm`、`register` 这样的语法关键字。
- 内存状态变化可选输出可读 GFM Table 供 View/Proof 使用;证明必须以抽象执行语义为依据,而不是假设表格文本本身就是执行存储。
示例(**候选语法,当前尚不可运行**):
示例(**目标语法,按 RFC-LANG-0002 书写;当前尚不可运行**):
````markdown
## 研究一个字节缓冲区
## @exec 研究一个字节缓冲区
此函数演示“可引用的存储区域”与普通 Markdown 叙述并存。
**区域 = memory.new(1, 4)**
`区域 = memory.new(1, 4)`
**`区域[8] = 42`**
`区域[8] = 42`
**另一个名字 = 区域**
`另一个名字 = 区域`
**`另一个名字[9] = 99`**
`另一个名字[9] = 99`
*区域[8] + 区域[9]*
`return 区域[8] + 区域[9]`
````
预期结果 141。注意这里 `另一个名字 = 区域` 分享 Memory 身份;一般列表/表格并不因此被改造成共享字节缓冲区。
@@ -383,7 +400,7 @@ Memory 支持字节缓冲区与数据布局,但 GPU 计算仍需要执行后
- `marqdo-value`:新增不可伪造的 Memory 值表示;必要时添加 ResourceHandle,注意现有 PluginHandle 不能随意直接复用为 OS 资源句柄。
- `marqdo-expression`:`AssignTarget` 的 Local / Index 变体;保留原有表达式读取。
- `marqdo-frontend`:更新表达式 AST 对应 IR 和定位信息;不改变 Markdown 语义映射。
- `marqdo-frontend`:更新表达式 AST 对应 IR 和定位信息;不改变 RFC-LANG-0002 定义的模式/载荷映射(语法迁移是独立的全仓库工作)。
- `marqdo-runtime`:Memory 实例生命周期/增长/边界检查/预算;执行位置写入;参考解释器。
- `marqdo-host`:逐步提供可注册资源操作的能力边界;必须保持旧 `HostOp` 兼容窗口,不能一次砍断所有旧调用。
- `marqdo-package` / CLI:发布标准库兼容信息,继续利用已存在的 path/Git pinned lock;资源授权仍由真实操作者提供。
@@ -420,7 +437,7 @@ marqdo-stdlib/
### R1:最小可寻址存储(核心里程碑)
- `Memory` 区域:创建、大小、增长、读取一字节、写入一字节;实现总/单区域预算、错误恢复与 trace。
- 支持 `**\`m[8] = 42\`**` 这类现有 Markdown 负载中的索引赋值。
- 支持 `` `m[8] = 42` `` 这类现有 Markdown 载荷中的索引赋值。
- 不开放裸指针、不引入新寄存器、不改旧 List/Map/Table 的赋值与复制行为。
- 运行跨区域隔离与别名测试;报错不能导致原有区域被污染。
+54 -23
View File
@@ -1,5 +1,19 @@
# Marqdo CLI M0.6(扩展性审计修订 v3)— 轻量核心、开放能力注册与独立生态实施规范
> **Historical planning input (M0.6).** This plan is superseded by
> [`STABLE_RELEASE_IMPLEMENTATION.md`](STABLE_RELEASE_IMPLEMENTATION.md)
> for next-phase work and is not current status. See
> [`architecture/CURRENT.md`](architecture/CURRENT.md) for the tree as it is today.
> Keep this file as design history; do not use it as an implementation claim.
>
> **语言语义已被取代:** 本文件出现的 `Strong`/`Emph`/有序列表/无序列表/引用块
> 程序角色是旧映射,已被
> [`docs/architecture/RFC-LANG-0002-unified-semantics.md`](architecture/RFC-LANG-0002-unified-semantics.md)
> ([ADR-0052](adr/0052-unified-markdown-semantics.md))废止。现行唯一语言规范
> 是该 RFC 与 `spec/mapping.md` / `spec/expression.ebnf`;下列 M0.6 路线、
> 审计与验收均为历史输入。§3 的旧语法示例已替换为目标语法并标注历史。
> **文档性质**:面向没有参与任何历史讨论的开发者的独立架构决策、实施计划及验收契约。
> **建议仓库路径**:`docs/NEXT_PHASE_IMPLEMENTATION.md`
> **设计状态**:**Proposed / v3 破坏性架构纠偏**(需要维护者批准并通过验收;基于源码进行的扩展性审计要求:Tool Protocol、可调用函数、外部内核、ABI 导出、模块来源均消除不必要的硬编码扩展瓶颈)。
@@ -13,7 +27,7 @@
本文件用于告诉陌生开发者“下一步为什么做、具体做什么、哪些不要做、怎样验收”,**不是**另起一套取代现有实现的语言规范。建议依次阅读:
1. `docs/CORE_STABILIZATION_AND_ECOSYSTEM_GATE.md`:后续稳定化、协议冻结与独立生态准入的当前阶段规范(Proposed);先用它刷新基线与发布门槛。
1. `docs/CORE_STABILIZATION_AND_ECOSYSTEM_GATE.md`:历史稳定化/协议冻结规范(Proposed,审查基线 `0adebe4`);语言与路线已被 RFC-LANG-0002 和 `STABLE_RELEASE_IMPLEMENTATION.md` 取代,仅作历史输入。
2. 本文件:产品边界、M0.6 实施次序与发布条件。
3. `docs/STATUS.md`:M0.5 当前实现的阶段性记录;该文件可能因后续修复提交而局部滞后,存在冲突时以当前源码和回归测试为准,并同步修正状态表。
4. `spec/markdown-profile.md`、`spec/mapping.md`、`spec/expression.ebnf`、`spec/evaluation.md`、`spec/references.md`:**现有语言行为的规范来源**。
@@ -35,7 +49,7 @@
- 对外输出**稳定、最小、版本化**的数据契约,不公开整个私有 IR 并让三个外部仓库依赖 Rust 内部结构。
- **用户入口统一,工程仓库独立**:通过同一 `Tool Protocol` 安装任意合规清单后自动得到 `marqdo <command>`。`view`、`knowledge`、`prove` 只是示例,不在 CLI 中注册硬编码的特殊名字、发布地址或适配器。
- 拒绝为了“以后可能有用”增加顶层命令;先用真实消费者需求、测试和 ADR 证明必要性。
- **开放扩展点,但不开放语言语法的任意重定义**:允许注册外部函数、工具、内核、依赖来源;禁止插件重新解释 `Strong`、`Emph`、列表或引用的核心语义。
- **开放扩展点,但不开放语言语法的任意重定义**:允许注册外部函数、工具、内核、依赖来源;禁止插件重新解释标题模式、载荷载体、关键字控制流或容器语义(RFC-LANG-0002;旧 `Strong`/`Emph`/列表/引用映射已废止)。
- **允许破坏性修正错误架构**:现处 `0.1.0` 早期阶段,凡未对外冻结或存在显著扩展阻塞的内部 API 可重构;涉及公开 ABI、wire、清单或语言行为时必须显式升级版本、加负例和迁移说明。
## 1. 本次真实仓库审查(不是假定的空仓)
@@ -177,7 +191,7 @@ Python/Node 可以继续作为内置 provider;新增 Julia/R/自定义内核
当前项目版本 `0.1.0`,**不对错误设计承担无限兼容义务**。开发者应优先保证未来长期正确性,而不是维护早期内部 enum 的表面稳定。推荐机制:
- **内部接口(可以直接破坏)**:`HostOp` 分发、`Builtin` 内部查找、`PluginSlot` 单 namespace、模块 loader、非公开 IR、CLI 命令内部结构;每次变更要有迁移 PR、替换后的测试和明确的 PR 说明。
- **用户可见语义(默认保护)**:`Strong/Emph/List/Quote` 语义、显式 YAML import、无括号调用的拒绝规则、链接不执行、函数参数推断等。若发现客观错误,也必须有 ADR、版本升级和一致性金样,而不能把“扩展性”当作随意改语法的理由。
- **用户可见语义(默认保护,现行见 RFC-LANG-0002)**:`@exec`/`@knowledge` 模式与继承、Exec H2 可调用性、`Strong`/`Emph`/行内 `Code` 载荷等价、`if/else/while/end/return/params`、显式 YAML import、链接不执行、函数参数推断等。若发现客观错误,也必须有 ADR、版本升级和一致性金样,而不能把“扩展性”当作随意改语法的理由。(原文的 `Strong/Emph/List/Quote` 是旧映射,已废止。)
- **版本化公共协议(只能显式破坏)**:Tool Protocol、Analysis/Trace、插件 ABI、Value Wire、Lock 格式。新旧同时可用或明确不兼容均可,但不能把 schema 名不变、字段意义变了当作修复。
- **冗余命令(可以有序削减)**:`init`、`inspect`、`fmt`、`plugin inspect`、`remove`、`lock` 等仅作为现有 0.x 行为,保留要以维护收益证明;公开主线禁止因此增加更多命令。删除时提供对应的文档内替代方式,避免把重要能力一起删除。
@@ -268,38 +282,55 @@ docs/
- Analysis/Trace 导出器读取核心已产生的数据,禁止维护另一套符号解析器。
- 外部工具不能反向要求核心增加 `knowledge`, `proof`, `view` 专用类型或运行逻辑。
## 3. 语言行为在本次迭代中保持冻结
## 3. 语言行为在本次迭代中保持冻结(历史章节)
### 3.1 Markdown 结构的唯一语义
> **历史说明:** 本节写于 RFC-LANG-0002 之前,原文描述的 `Strong`/`Emph`/列表/
> 引用块映射已被废止。原文的“本次迭代冻结旧映射、函数体外惰性告警”不再是
> 现行设计;现行规则是标题模式 + 统一载荷 + 关键字控制流,非法载荷为硬错误
> 且旧 `.mq.md` 源不可执行。下面保留改写后的示例仅用于说明目标语法;它不是
> 本 M0.6 计划的实现承诺。
在 **level-2 函数体内部**(当前 ADR-0002):`Strong` = 计算/赋值;`Emph` = 退出函数并返回;`OrderedList` = 有序首匹配分支;`BulletList` = 条件循环;`BlockQuote` = 显式调用;`Text` = 惰性叙述;`CodeSpan` 在分支/循环头的特定结构位置表示条件;`Link` = 静态引用;数学节点与代码围栏 = 原生惰性资源。一级标题给命名空间,二级标题给函数,三级及以上标题给文档分节。
### 3.1 Markdown 结构的唯一语义(RFC-LANG-0002)
函数体外不执行程序;现有 ADR-0002 规定可疑“执行节点”惰性并告警,而不是直接在模块根运行。除非另提 ADR,不要偷偷改为硬错误。源码同义 Markdown 结构(例如 `**x**` / `__x__`)按同一 AST 语义执行;也不因为正文中文有“如果/返回/执行”就推断控制流。
在 Exec 范围(显式 `@exec` 或从祖先继承)内:标题声明/继承 `@exec` 与
`@knowledge` 模式;只有 Exec H2 是可调用函数;`Strong`、`Emph`、行内 `Code`
是等价的单语句载荷载体;`if`/`else`/`while`/`end`/`return`/`params(...)`
决定控制与签名;`Text` = 惰性叙述;`Link` = 静态引用;数学节点与代码围栏 =
原生惰性资源;有序列表、无序列表与引用块只是容器,不产生指令。一级标题给
命名空间/默认模式,二级标题给函数(Exec)或知识单元(Knowledge),三级及
以上标题只给文档分节并继承所属函数。
函数体外不产生执行投影;Exec 载荷若没有所属 H2 函数是
`sem.exec_without_function` 硬错误,绝不提升为模块级执行。源码同义 Markdown
结构(例如 `**x**` / `__x__`)在载荷提取上等价;不因为正文中文有“如果/返回/
执行”就推断控制流。
```markdown
# 数学
# @exec 数学
## 阶乘
## @exec 阶乘
输入 `n`,计算非负整数的阶乘。
1. `n <= 1`,递归基。
`params(n: Int)`
*1*
`if n <= 1`
2. `n > 1`,继续递归。
`return 1`
**结果 = n * 阶乘(n=n-1)**
`else`
*结果*
`return n * 阶乘(n=n-1)`
`end`
```
期望:`marqdo call factorial.mq.md 数学.阶乘 --arg n=5` 返回 `120`。实际放置测试时以现有 `examples/factorial.mq.md` 为准并追加金样,**不要为了执行把示例转换成 Python**。
期望(目标语法,前端尚未迁移):`marqdo call factorial.mq.md 数学.阶乘 --arg n=5` 返回 `120`。实际放置测试时以迁移后的示例与金样为准,**不要为了执行把示例转换成 Python**。
### 3.2 函数与参数推断必须严格而可检查
- 标题保持自然名称,如 `## 计算圆面积`,**不要把参数名强制放进标题**;标题变更可能影响 Markdown 锚点,参数变更不应顺便破坏链接。
- 显式签名放在约定的标题后独立 `CodeSpan` 段落,例如 `` `(r, precision=2)` ``;无签名允许静态形参推断。
- 显式签名写成 Exec H2 的第一条直接载荷 `params(r, precision=2)`(或带类型 `params(r: Number, precision: Int = 2)`);无签名允许静态形参推断。旧“标题后独立 CodeSpan `(r, precision=2)`”签名已被 RFC-LANG-0002 废止。
- 参数推断源于执行表达式及控制流的自由变量/确定赋值分析,不从叙述文本或 LLM 中猜测;可见未初始化局部变量应报错。
- 推断签名默认只接受具名调用,显式签名可接受位置调用;无参数调用仍用括号 `()`;调用的解析器只有一套。
- 重复形参、错误实参、未解析别名、变量读写歧义产生稳定诊断码。
@@ -310,9 +341,9 @@ docs/
- 表达式解析来自 `marqdo-expression`,不是“普通 Markdown 括号里藏 Python”。
- 既有 ADR-0007 采用任意精度整数及类型化 wire v1;浮点、布尔和 `Unit` 语义以 `spec/evaluation.md` 为准。变更必须同步 ABI/Foreign/wire 的兼容性测试。
- `OrderedList` 的首个条件头为符合结构规定的 `CodeSpan`;同一段落里条件后的叙述和后续缩进段落不参与条件求值。无 `CodeSpan` 的清单在函数内是非法分支,不回退成普通清单。
- `BulletList` 的每个 item 是一个 `while` 条件与循环体;不默认把 `- [ ]` 任务清单当普通 UI 清单。
- 引用块一次只表达一个显式调用;链接形式调用是否跨文件支持由 `spec/references.md` 目前的限制决定,不能为了满足文档示例绕过模块 loader。
- 条件不再由列表首段 CodeSpan 表达:写 `if <expr>` … `end`,列表只是容器。旧“`OrderedList` 首段 CodeSpan 条件头”规则已被 RFC-LANG-0002 废止;普通列表在 Exec 范围内按源序提取合法载荷,零语句投影是合法惰性结构。
- 循环写 `while <expr>` … `end`;不再把每个 bullet item 当作独立 while。`- [ ]` 任务清单与普通列表一样没有隐含指令。
- 引用块不是调用;调用必须来自 Exec 载荷中的表达式。跨文件调用是否支持由 `spec/references.md` 的显式导入限制决定,不能为了满足文档示例绕过模块 loader。
- 公式和代码块必须先构建惰性值,只有 `math.*`、`foreign.*` 或明确 Host API 才能执行;不自动运行任意外部语言。
### 3.4 引用的最小承诺
@@ -625,7 +656,7 @@ Core 不包含 Lean、Proof IR、证明驱动器或证书存储。新的证明
### 12.3 `marqdo-view`
默认按原 Markdown 阅读,不执行;使用 Analysis v1 对源码 Strong/Emph/分支/循环和函数做精确标注。存在真实 Trace v1 时,可选择播放调用帧与历史结果;若用户选择运行,View 须明确发起 `marqdo run/call` 并尊重沙箱与权限。它不实现第二个解释器、第二个静态语义分析器,不将未运行的语义节点当成已执行结果。
默认按原 Markdown 阅读,不执行;使用 Analysis v1 对源码标题模式、载荷、`if`/`while`/`return` 控制流和函数做精确标注。存在真实 Trace v1 时,可选择播放调用帧与历史结果;若用户选择运行,View 须明确发起 `marqdo run/call` 并尊重沙箱与权限。它不实现第二个解释器、第二个静态语义分析器,不将未运行的语义节点当成已执行结果。
### 12.4 对外工具的独立性条件
@@ -701,13 +732,13 @@ Core 不包含 Lean、Proof IR、证明驱动器或证书存储。新的证明
| EXT-PACKAGE | 相同包多版本、项目跨目录、全局缓存移动、Git pin、离线同步、锁漂移 | `ModuleId` 依赖物理路径,知识/证明制品不可复用 |
| EXT-KERNEL | 注册从未内置的语言标签、缺 provider、超时、非授权进程、wire 版本冲突 | Foreign 不可真正扩展或会绕开 Host |
| EXT-ABI | 多 namespace、重复 export、跨语言 C/Rust 示例、ABI 主版本拒绝、Worker crash、capability broker、伪造句柄与重放拒绝 | C ABI 宣称可扩展但只支持固定单一路径 |
| EXT-SEMANTIC | 修改普通叙述后的执行 IR 稳定;外部组件不能改变 `Strong/List` 等 AST 意义 | 将“扩展”错误扩大到语言执行规则 |
| EXT-SEMANTIC | 修改普通叙述后的执行 IR 稳定;外部组件不能改变标题模式/载荷/容器等 AST 意义 | 将“扩展”错误扩大到语言执行规则 |
| EXT-CONSUMER | Core 公开 JSON 在 Rust 私有 IR 改动之后仍与独立客户端兼容;Reference 非执行 | 外部仓库将被迫跟随 Core 频繁升级 |
| 测试域 | 正例 | 必须负例/反例 | 测试位置 |
|---|---|---|---|
| Markdown AST | Strong/Emph/list/quote/heading、CJK、多行、GFM table、Math | 强调嵌套、缺失条件、task-list 错位、parser 版本漂移 | `tests/conformance/ast` + profile tests |
| Markdown AST | heading modes、payload carriers(Strong/Emph/Code)、lists/quotes、CJK、多行、GFM table、Math | 嵌套载荷、模式拼写、缺失 `end`、task-list 错位、parser 版本漂移 | `tests/conformance/ast` + profile tests |
| 语言求值 | 阶乘、递归、嵌套分支、函数调用、命名实参、返回 | 未初始化、重复参数、非法返回、fuel/stack 超限 | `tests/conformance/semantics` + negative |
| 叙述不变性 | 只修改普通正文,IR/值不变 | 文本中插 `if/return` 不改变结果 | `crates/cli/tests/properties.rs` |
| 函数接口 | 推断形参、显式签名、跨模块别名 | 名称冲突/模糊/缺少参数/错误签名 | frontend + CLI tests |
@@ -778,7 +809,7 @@ cargo build -p marqdo-cli
维护者可以用以下清单判断是否允许另一名未参与设计讨论的开发者独立推进项目:
- [ ] 能解释为什么 Marqdo 的 Strong/Emph/List/Quote 具有固定控制流,而 Text 不执行。
- [ ] 能解释为什么 Marqdo 的模式/载荷/关键字控制流是确定的,而 Text 与容器不执行。
- [ ] 知道唯一 Markdown parser 在哪里;不会编写第二套行正则解析器。
- [ ] 能在 `spec/mapping.md` 定位函数、分支、循环和返回的约束。
- [ ] 能在 `crates/marqdo-runtime` 找到原生求值,在 `marqdo-host` 找到所有副作用检查。
+66 -32
View File
@@ -1,5 +1,27 @@
# Marqdo CLI:CI 收敛、语言边界回归与稳定发行实施规范
> **Historical / Superseded baseline (`c9f01aa8`, 2026-11-20).** The CI SHA
> and run below are a snapshot. Refresh the remote SHA and CI evidence before
> use. The current overview is
> [`architecture/CURRENT.md`](architecture/CURRENT.md); executed status is in
> [`STATUS.md`](STATUS.md). Next-phase work is governed by
> [`STABLE_RELEASE_IMPLEMENTATION.md`](STABLE_RELEASE_IMPLEMENTATION.md).
>
> **语言语义已被取代:** 本文件的 Markdown/`**` 语义讨论写于 RFC-LANG-0002 之前;
> `Strong`=语句、`Emph`=返回、有序列表=分支、无序列表=循环、CodeSpan 条件头、
> BlockQuote=调用的旧映射已被
> [`docs/architecture/RFC-LANG-0002-unified-semantics.md`](architecture/RFC-LANG-0002-unified-semantics.md)
> ([ADR-0052](adr/0052-unified-markdown-semantics.md))废止。§3 的旧语法示例已替换
> 为目标语法并标注历史;现行规范见 `spec/mapping.md` / `spec/expression.ebnf`。
> **CI status refresh (2026-11-21):** the GitHub-hosted Windows/macOS matrix
> exists in `.github/workflows/ci.yml`; it is not "unconfigured". The platform
> `E0425`/`E0433` defect and the strict `--json` empty-stdout defect are fixed in
> the RFC-CI-0001 series (ADR-0053). Those fixes do not turn this file's
> `c9f01aa8` rows into current evidence: re-run the matrix on the candidate SHA
> and record the result in `docs/evidence/ci/<SHA>/manifest.json`.
>
> **文档性质:** 可直接入库的执行规范 / Release Convergence Plan。供未阅读任何历史聊天的维护者、贡献者与 DeepSeek Harness 使用。
>
> **仓库:** `https://github.com/cflmy/marqdo-cli`;开发者的本地 Git 远端可能是 Gitee,不应混淆远端提交、GitHub 镜像与真实 CI 状态。
@@ -49,7 +71,7 @@ marqdo-markdown:固定 Comrak/CommonMark/GFM/Math profile(唯一 Markdown
│ Canonical MdNode + byte Span
▼
marqdo-frontend:Heading 作用域/导入/参数推断/Markdown AST→IR
│ 条件头、分支、循环、调用、引用、源码位置
│ 标题模式、载荷、控制关键字、引用、源码位置
▼
marqdo-ir + marqdo-expression:可执行 AST/强类型表达式语法
│
@@ -178,23 +200,28 @@ bwrap: Creating new namespace failed: Resource temporarily unavailable
## 3. Markdown 与乘方:从疑虑收缩为正式回归测试
### 3.1 已确定的语义,不得擅自改变
### 3.1 已确定的语义,不得擅自改变(已按 RFC-LANG-0002 更新)
Markdown 源码经过冻结 Comrak profile。`##` 定义函数;普通函数段落中的 `Strong` 是可执行语句;`Emph` 是返回表达式;有序列表是分支,无序列表是循环;列表项第一段的**首个 CodeSpan** 是条件表达式;后续普通文字可解释目的;BlockQuote 是显式调用。完整计算表达式由独立 Marqdo 表达式解析器解释。
> **历史说明:** 本节原文写于旧映射时期(`##` 无条件函数、`Strong`=语句、
> `Emph`=返回、有序列表=分支、无序列表=循环、首段 CodeSpan=条件、
> BlockQuote=调用)。该映射已被 RFC-LANG-0002 废止,原文中的“不要以修复乘方
> 为由改语法”等表述是当时语境;现行规则如下。
**用户本地实测**已经说明:适当空格使 Markdown 按预期生成 AST,乘方可以正确运行。故这一轮**不应以“修复乘方”为由重新设计强/斜体语法**。Markdown 标记本身具有分隔符词法规则:紧凑写法造成不同 AST 属于需要准确文档化的语言边界,而不是一律归咎解释器。
Markdown 源码经过冻结 Comrak profile(Markdown profile 不变)。现行映射:标题以 `@exec`/`@knowledge` 声明模式并从祖先继承;只有 Exec H2 是可调用函数;`Strong`、`Emph`、行内 `Code` 是等价的单语句载荷载体;`if`/`else`/`while`/`end`/`return`/`params(...)` 决定控制与签名;列表、引用块、段落只是容器。完整计算表达式由独立 Marqdo 表达式解析器解释。
Markdown 标记仍具有分隔符词法规则:紧凑写法造成不同 AST 属于需要准确文档化的语言边界,而不是一律归咎解释器。因此目标语法推荐把表达式放进独立的行内 CodeSpan 载荷;`**` 只有在载荷提取后才作为乘方运算符。
```markdown
## main
## @exec main
**n = 2**
`n = 2`
**`n = n ** 2`**
`n = n ** 2`
> print(n)
`print(n)`
```
以上是建议的、对 `**` 乘方运算最稳妥的可读写法:外围 Strong 定义执行语句,内部 CodeSpan 避免 Markdown 把计算运算符看作强调分隔符。对于本机已经验证能形成正确 AST 的空格写法,也应以 AST 金样记录,不应禁止。
以上为目标语法示例(前端尚未迁移,不能原样运行):独立行内 CodeSpan 作为载荷,避免 Markdown 把计算运算符看作强调分隔符。对于能形成正确 AST 的写法,也应以 AST 金样记录。
**正式判断标准:** `marqdo inspect --ast` 的 Comrak AST 是否对应规范结构;其后 `check`/`run` 是否按该结构执行。不得为“猜测用户想写什么”而增加原始 Markdown 正则修复器,也不得因源码像表达式就跳过 Markdown AST。
@@ -212,38 +239,40 @@ Markdown 源码经过冻结 Comrak profile。`##` 定义函数;普通函数段
| 编号建议 | 源码表达 | 预期 | 证明什么 |
|---|---|---|---|
| 37 | `*` + CodeSpan(`2 ** 3 ** 2`) + `*` | `Int 512` | Markdown→右结合解析→求值 |
| 38 | `*` + CodeSpan(`-2 ** 2`) + `*` | `Int -4` | 一元负号优先级 |
| 39 | `*` + CodeSpan(`(-2) ** 2`) + `*` | `Int 4` | 括号与优先级 |
| 40 | `*` + CodeSpan(`2.0 ** -1`) + `*` | `Float 0.5` | 浮点负指数 |
| 41 | `**` + CodeSpan(`n = 2 ** 3`) + `**`,然后返回 `n` | `Int 8` | Strong + CodeSpan 嵌套路径 |
| 42 | 有序列表条件 `` `n ** 2 == 4` `` + 叙述 + 引用调用 | 只调用真分支 | 条件 Markdown / 段落 / 控制流 |
| 43 | 无序列表条件 `` `n ** 2 < limit` `` + 叙述 + 递增 | 预期迭代和返回 | 循环、条件重新求值、乘方 |
| 37 | 独立 CodeSpan 载荷 `` `2 ** 3 ** 2` `` | `Int 512` | Markdown→右结合解析→求值 |
| 38 | 独立 CodeSpan 载荷 `` `-2 ** 2` `` | `Int -4` | 一元负号优先级 |
| 39 | 独立 CodeSpan 载荷 `` `(-2) ** 2` `` | `Int 4` | 括号与优先级 |
| 40 | 独立 CodeSpan 载荷 `` `2.0 ** -1` `` | `Float 0.5` | 浮点负指数 |
| 41 | 载荷 `` `n = 2 ** 3` `` 后 `` `return n` `` | `Int 8` | 赋值 + return 关键字 |
| 42 | `` `if n ** 2 == 4` `` + 打印 + `` `else` `` + 打印 + `` `end` `` | 只执行真分支 | 关键字分支 / 载荷 / 控制流 |
| 43 | `` `while n ** 2 < limit` `` + 递增 + `` `end` `` + `` `return n` `` | 预期迭代和返回 | 循环、条件重新求值、乘方 |
例如,给 42 建立下述源文件;相应 expected 的 `stdout` 应为 `乘方正确\n`,返回值应为 `Unit`,实际 JSON 具体类型按已存在 wire-v1 金样约定构造:
例如,给 42 建立下述(目标语法)源文件;相应 expected 的 `stdout` 应为 `乘方正确\n`,返回值应为 `Unit`,实际 JSON 具体类型按已存在 wire-v1 金样约定构造:
```markdown
## main
## @exec main
**n = 2**
`n = 2`
1. `n ** 2 == 4`,正确分支的自然语言说明。
`if n ** 2 == 4`
> print("乘方正确")
`print("乘方正确")`
2. `n ** 2 != 4`,另一分支的说明。
`else`
> print("乘方错误")
`print("乘方错误")`
`end`
```
对 41:
```markdown
## main
## @exec main
**`n = 2 ** 3`**
`n = 2 ** 3`
*n*
`return n`
```
### 3.4 必须新增的负向/边界测试
@@ -253,18 +282,23 @@ Markdown 源码经过冻结 Comrak profile。`##` 定义函数;普通函数段
| `2 ** -1`,纯 Int | 类型错误;不是神奇地得到 `0.5` | Runtime error fixture |
| `2 ** 100001` | `resource.exhausted`(当前指数限额) | Runtime error fixture |
| `2 **` | 解析器诊断非法表达式,正确源码 span | 负向 fixture |
| 列表头 `` `n ** 2 ==` `` | 条件表达式语法错误,定位到原 Markdown 源码中缺失操作数的位置 | Frontend / CLI error |
| `if` 条件 `` `n ** 2 ==` `` | 载荷语法错误,定位到原 Markdown 源码中缺失操作数的位置 | Frontend / CLI error |
| 多反引号 CodeSpan 包含 `**` | 仍能正确映射完整表达式与字节 Span | AST + frontend |
| `**n=n**2**` 等缺少空格的 Markdown 边界 | 固定 Comrak 真实 AST,不推断用户意图,不自动更正 | AST golden + 文档例子 |
| `**n = n ** 2**`(有空格) | 在实际 profile 中确认 AST,再据此固定接受/拒绝结果 | AST + CLI |
| 普通叙述段落出现 `` `n ** 2` `` | 不执行计算、不被作为分支条件 | 结构/副作用性质测试 |
| `@knowledge` 段落出现 `` `n ** 2` `` | 不执行计算、不产生 IR | 结构/副作用性质测试 |
| 代码围栏中出现 `**` | 保持惰性,不会触发运算 | AST + inert fixture |
| 纯说明段落的修改 | 可执行 IR/实际结果不变 | property test |
| 强行嵌套 Markdown 强调导致非法 payload | 给出诊断,不静默换另一计算规则 | negative test |
**条件头被吞掉的可执行节点:** 当前 `map.rs::condition_of()` 使用 `&item.children[1..]` 形成分支体,首段里第一个 CodeSpan 后若出现 `Strong` / `Emph`,这些可执行标记可能被整体忽略。建议在原有“首个 CodeSpan + 说明文本”语法上增加一个严格检查:首段剩余结构仅允许惰性叙述(按已冻结 profile 判定),保留其原始 AST 与 SourceSpan;若出现其他可执行角色,报 `markdown.invalid_construct`,不能无声丢弃。先补复现 fixture,再制定必要的精确规则;**无需更换 CodeSpan 条件头的正式语法**。
> **历史说明:** 以下两段描述的是 RFC-LANG-0002 之前的条件头/CodeSpan
> 实现细节(`map.rs::condition_of()`、`markdown.invalid_construct`)。在现行
> 映射中列表不再有条件头,载荷统一用 `sem.*` 诊断;这两段只作为旧实现考古,
> 不得据此恢复旧语法。
**CodeSpan 源码位置:** `condition_of()` 当前使用 `PayloadSegments::single(&literal, condition_node.span)`,须回归验证反引号是否计入字节跨度;若诊断偏移,修正 span 映射而不是改 Markdown AST。要求单/双反引号、中文前缀、换行和多字节 UTF-8 的准确来源测试。
**条件头被吞掉的可执行节点(旧实现,历史):** 旧 `map.rs::condition_of()` 使用 `&item.children[1..]` 形成分支体,首段里第一个 CodeSpan 后若出现 `Strong` / `Emph`,这些可执行标记可能被整体忽略。旧建议是在“首个 CodeSpan + 说明文本”语法上增加严格检查;该语法已被废止,目标语法改由关键字控制流表达分支。
**CodeSpan 源码位置(历史):** 旧 `condition_of()` 使用 `PayloadSegments::single(&literal, condition_node.span)`,须回归验证反引号是否计入字节跨度。该回归要求作为 span 映射经验保留;新 frontend 需为所有载荷载体(Strong/Emph/Code)提供准确字节跨度。
### 3.5 正确的测试组织与执行命令
@@ -383,7 +417,7 @@ ADR-0021 已实现 Bubblewrap supervisor,Linux strict L1 在专用 Ubuntu 26.0
### 7.1 关闭条件:最短可衡量路径
- **第一个工作包完成:** G1~G3 全绿(解决旧 CI 三处失败);不再存在 E0433 / 环境依赖的错误断言。
- **第二个工作包完成:** G5/G6 全绿(乘方、Strong/CodeSpan、列表叙述、负向边界、源 Span);语言语义不需重新设计。
- **第二个工作包完成(历史验收,旧映射):** G5/G6 全绿(乘方、Strong/CodeSpan、列表叙述、负向边界、源 Span);该结论对应旧映射,语言语义已由 RFC-LANG-0002 重写。
- **第三个工作包完成:** G7/G8 全绿或清楚地标为非承诺子能力;独立 Tool 消费无硬编码。
- **第四个工作包完成:** G9 由专用 VM 自托管 CI 在同 SHA PASS,或经负责人明确决定仅发行 G-B 并真正关闭 G-C 声明。
- **第五个工作包完成:** G10/G11 PASS、Release Candidate 清洁安装,再经批准完成 G12。
@@ -486,7 +520,7 @@ Approved by / time:
(2) Ubuntu 普通 CI 下 strict probe 与实际启动竞态/资源不足的 fail-closed
契约,并在独立安全 VM 保证严格隔离正向执行与 7/7 攻击矩阵;
(3) 增加完整 Markdown AST、乘方、分支/循环条件、源 Span、惰性与
错误边界的黄金测试和端到端语义测试,不擅自改 CodeSpan 条件语法;
错误边界的黄金测试和端到端语义测试;条件头/CodeSpan 旧语法已废止,目标语法见 RFC-LANG-0002;
(4) 验证 lock v3、Analysis/Trace、完全通用的 Tool、ABI/Worker/Foreign
的支持矩阵和独立消费者;
(5) 完成全平台 CI、同 SHA 的 G-C 证据、Release Candidate 干净安装、
+10
View File
@@ -1,5 +1,15 @@
# Marqdo CLI 稳定版本开发与发布实施总规范
> **Historical roadmap note / superseded language syntax.** 本文件是稳定版本
> 规划快照(编写基线 `e966b766`,2026-11-20),不是执行状态记录。其语言前提
> 早于
> [`docs/architecture/RFC-LANG-0002-unified-semantics.md`](architecture/RFC-LANG-0002-unified-semantics.md)
> ([ADR-0052](adr/0052-unified-markdown-semantics.md)):旧 `Strong`=执行、
> `Emph`=返回、列表=分支/循环、引用块=调用的映射已被废止;语言语义只以该 RFC
> 与 `spec/mapping.md` / `spec/expression.ebnf` 为准。发布计划/门槛内容仍可作为
> 下一阶段规划的参考输入,但必须按当前 HEAD、`docs/STATUS.md` 与迁移台账刷新后
> 才能当作现行门禁;不得据此声称新语法或任一平台已完成。
> **定位:** 可直接提交到 `cflmy/marqdo-cli` 的 `docs/STABLE_RELEASE_IMPLEMENTATION.md`,并完整交由 DeepSeek Harness 执行。
> **状态:** Proposed / 下一阶段唯一总体实施计划(必须以实际代码、规范和测试验证结果修正状态)。
> **编写基线:** `cflmy/marqdo-cli`,默认分支 `master`,远程 HEAD `e966b7661f03ee7462df959917f5630353e66b10`(2026-11-20)。
+249 -46
View File
@@ -1,22 +1,144 @@
# Marqdo Implementation Status
Snapshot: **2026-11-20**, workspace version `0.1.0`, no release tag.
Snapshot: **2026-11-21**, workspace version `0.1.0`, no release tag.
Current architecture overview:
[`architecture/CURRENT.md`](architecture/CURRENT.md).
Authoritative CI convergence plan:
[`NEXT_RELEASE_CI_CONVERGENCE.md`](NEXT_RELEASE_CI_CONVERGENCE.md).
[`NEXT_RELEASE_CI_CONVERGENCE.md`](NEXT_RELEASE_CI_CONVERGENCE.md) (historical
baseline at `c9f01aa8`; refresh before use).
Support matrix: [`../COMPATIBILITY.md`](../COMPATIBILITY.md).
This file records what was actually executed on the current tree. "Implemented"
does not mean "verified on every platform"; the evidence column says which
commands were run and where. The historical M0.5 snapshot is preserved at
commands were run and where. Do not copy a running status into another plan;
[`architecture/CURRENT.md`](architecture/CURRENT.md) is the single current
overview. The historical M0.5 snapshot is preserved at
[`history/STATUS-M0.5-snapshot.md`](history/STATUS-M0.5-snapshot.md) and must
not be read as current status.
Status is generated from executed commands and logs; a gate row without a command,
log or source path is a defect, not a state.
## 0. Language migration completed (RFC-LANG-0002)
The repository now has **one** language contract, implemented end to end:
[`architecture/RFC-LANG-0002-unified-semantics.md`](architecture/RFC-LANG-0002-unified-semantics.md)
(accepted by [ADR-0052](adr/0052-unified-markdown-semantics.md)), with the
normative summary in [`../spec/mapping.md`](../spec/mapping.md) and
[`../spec/expression.ebnf`](../spec/expression.ebnf).
Landed on `master` as one linear series (the former `semantics-v2-refactor`
branch was fast-forwarded and deleted; only `master` remains):
| Step | Commit | Content |
|---|---|---|
| 1 | `4983542` | docs/spec/ADR/READMEs, historical banners, migration guide and inventory |
| 2 | `e029853` | frontend/IR/runtime implementation, `sem.*` diagnostics, migrated stdlib + digest, examples, fixtures and Rust-embedded sources |
| 3 | `ffd1cc9` | expression-owned payload statement grammar, extra negative/CLI coverage |
| 4 | `bbeb1de` | status/compatibility/README/migration docs flipped to the implemented state + green evidence |
| 5 | `b38b769` | function-wide `params(...)` position (across heading segments) + two negatives |
| 6 | `361a6cc` | removed the dead `Stmt::Call`/`CallSite`/`CallTarget` IR |
| 7 | `d5002ac` | stale pre-RFC role docs and review-aid fixes |
| 8 | `9fb97bc` | signatures recognized only from Exec payloads (Knowledge sections inert) |
| 9 | `a3f9770` | unmatched `end` diagnostics point at the `end` payload |
| 10 | `0bbe0f6` | Setext headings never declare a mode; dead helper removed |
| 11 | `b3429cb` | diagnostics sorted by source position then code |
| 12 | `9c1723d` | coverage for the removed syntax-selector front-matter probe |
The breaking behaviour is:
* H1–H6 headings declare `@exec`/`@knowledge`; unmarked headings inherit the
nearest ancestor and the document root is Knowledge;
* only an Exec H2 is callable; an H2 Knowledge unit and H3–H6 sections are not
functions, and a payload with no owning Exec H2 is `sem.exec_without_function`;
* `Strong`, `Emph` and inline `Code` are interchangeable one-statement carriers;
nested carriers and empty payloads are hard errors;
* `if`/`else`/`while`/`end`/`return`/`params` come from the payload and lower to
the existing `Branch`/`Loop`/`Return`/`Assign`/`Eval` IR; lists, block quotes
and paragraphs are presentation containers;
* control frames cannot cross a heading segment;
* `params(...)` is legal only as the first direct Exec H2 payload across the
whole function (a later `### @exec` segment cannot add a signature after
earlier code ran); without it the existing free-variable inference applies and
inferred signatures keep the named-arguments-only rule;
* old syntax (`Strong` execute / `Emph` return / list branch/loop / quote call)
is **not** executable, and `marqdo_semantics`/`syntax` front-matter selectors
are rejected with `sem.unsupported_version_directive`.
### Verification at `9c1723d` (Linux x86_64, Rust 1.100.0)
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked --no-fail-fast PASS
507 passed, 0 failed, 2 ignored (62 green test targets)
python3 tools/pack-stdlib.py PASS (digest verified)
```
Full evidence: [`evidence/2026-11-21-semantics-v2-green.md`](evidence/2026-11-21-semantics-v2-green.md).
The intermediate "text migrated, implementation pending" record is kept as
[`evidence/2026-11-21-semantic-docs-migration.md`](evidence/2026-11-21-semantic-docs-migration.md)
and is superseded by the green gate.
**Not claimed:** the **remote** G3 security artifact is still missing. The
Linux/Windows/macOS native matrix is green, G4 is measured on the designated
Woodpecker agent, and the G3 mechanism is PASS in a local disposable VM, but
the RFC does not promote a local VM run to a remote gate. See §0.1, §4 and
[`../COMPATIBILITY.md`](../COMPATIBILITY.md). The release decision remains
**DO NOT RELEASE**.
## 0.1 CI/security convergence (2026-11-21)
The RFC-CI-0001 series fixes the release-blocking CI defects that the
`32d8fe1` GitHub run exposed. Commits on `master`:
| Step | Commit | Content |
|---|---|---|
| 1 | `f4a404b` | platform-neutral `StrictChildState`/`inspect_strict_child`; macOS/Windows no longer reference Linux-only functions (`E0425`) |
| 2 | `f817cce` | strict capture/routing: one JSON document on every `--json` path, stable failure codes, ADR-0053, deterministic routing tests |
| 3 | `c9f509e` | architecture guard against direct marker access; Windows cross-target check |
| 4 | `990e984` | G-C lane writes SHA-bound `strict-evidence.json`; workflow binds checkout/archive to the tested SHA and uploads evidence |
| 5 | `d2267e9` | correctness-verified `marqdo.perf.v2` sampler and fixed-node workflow |
| 6 | `0d7e11b` | SHA-bound release-evidence aggregator, docs drift cleanup, consistent `windows-latest`/`macos-latest` statement |
| 7 | `512b9f5` | evidence-generation failure fails the lane; workflow archive/verification fixes; G-C re-run |
| 8 | `beb68a9` | G-C evidence/status update and the cross-platform JSON smoke exit-code fix |
| 9 | `3a29fce` | Windows PATH lookup applies `PATHEXT`; ANSI-stripped native failure annotations |
| 10 | `b898c41` | macOS parser-nesting test stack; portable process product fixture (CLI `--version` instead of `/bin/echo`) |
| 11 | `4265231` | CLI runs on an explicit 8 MiB thread (Windows 1 MiB overflow); macOS temp-path canonicalization; G3 adversarial tests opt-in |
| 12 | `a24161c` | Windows foreign temp paths stay non-verbatim (Node `lstat` rejection) |
| 13 | `5fcc111` | reject rooted Windows dependency paths (`has_root`, not `is_absolute`) |
Executed evidence:
| Lane | Result | Evidence |
|---|---|---|
| Linux fmt/clippy/test/build (Woodpecker) | **PASS** | pipelines `#9`–`#12`, `#14`, `#16`, `#17`, `#21`–`#25` on the series; `#13`/`#15`/`#19` were superseded/cancelled by the next push, not failures |
| Linux + Windows + macOS native matrix (GitHub) | **PASS** | [run 30](https://github.com/cflmy/marqdo-cli/actions/runs/44896671929) on `5fcc111`: `fmt + clippy + test (Linux x86_64)`, `runtime contract (windows-latest)`, `runtime contract (macos-latest)`, and schema consumers all PASS |
| G3 strict isolation | **PASS (local disposable VM) / remote UNVERIFIED** | `5fcc111`, `dev` uid 1000, QEMU/TCG, 7/7 + strict JSON/inert-`@knowledge`; [`evidence/2026-11-21-strict-gc-5fcc111.md`](evidence/2026-11-21-strict-gc-5fcc111.md) |
| G4 performance | **MEASURED / WARN (provisional)** | Woodpecker agent 1, `1199c88`: 5 warmups + 20 samples, 9/9 `VERIFIED`, CPU 0 pinned; two runs -10.9%..+7.1% p50; shared powersave host caveat; [`evidence/2026-11-21-perf-woodpecker-agent-1.md`](evidence/2026-11-21-perf-woodpecker-agent-1.md) |
Release decision: **DO NOT RELEASE.** Linux, Windows and macOS native jobs are
green on the same SHA, the strict isolation mechanism PASSes locally as a
normal user, and G4 is `MEASURED`/`WARN` on the designated Woodpecker agent.
The remaining release gate is the **remote** G3 artifact whose `tested_sha`
matches; the RFC explicitly requires it (and the local VM PASS is not promoted
to a remote claim). The perf gate is provisional because the agent is a shared
powersave host; see [`performance.md`](performance.md).
The manifest contract is in
[`evidence/ci/README.md`](evidence/ci/README.md); `tools/ci/release_evidence.py`
reports `DO_NOT_RELEASE` for any missing/failed/SHA-mismatched gate.
## 1. Remote baseline and current evidence
GitHub `master` is still `c9f01aa8fb127a2dcc125103ef794832984774ad`; the latest
public CI is run
[44692692318](https://github.com/cflmy/marqdo-cli/actions/runs/44692692318)
and is **red**:
The local language baseline for this snapshot is `9c1723d` (2026-11-21,
semantics v2 implementation plus follow-up fixes). The later
documentation and test migration is recorded by its own commits; re-run the gate
before making a release claim.
The old GitHub `master` `c9f01aa8fb127a2dcc125103ef794832984774ad`
and public CI [run 44692692318](https://github.com/cflmy/marqdo-cli/actions/runs/44692692318)
are **historical (2026-11-20), not current**, and that run was red:
- Linux `fmt + clippy + test`: FAIL —
`strict_mode_enforces_isolation_or_refuses_closed` hit
@@ -26,11 +148,16 @@ and is **red**:
- Schema instance validation + consumers: PASS.
- Release candidate artifact: SKIPPED (upstream failure; tag/dispatch only).
The remote SHA must be refreshed from CI after a push; do not read `c9f01aa8`
as the current tree.
This checkout's private remote is `https://git.cflmy.cn/cflmy/marqdo-cli` with
Woodpecker CI at `https://ci.cflmy.cn` (`.woodpecker.yml`). The Linux Docker
pipeline runs fmt/clippy/test/release-build. GitHub native lanes, Windows/macOS
runners and a CI-attached strict VM runner are still unavailable; the G-C
evidence below remains local VM evidence.
Woodpecker CI at `https://ci.cflmy.cn` (`.woodpecker/main.yml`). The Linux Docker
pipeline runs fmt/clippy/test/release-build; the docs-consistency gate runs in
`cargo test --workspace`. The GitHub-hosted native matrix is now green at
`5fcc111` ([run 30](https://github.com/cflmy/marqdo-cli/actions/runs/44896671929)).
A CI-attached strict VM runner is still unavailable; the G-C evidence below
remains local VM evidence.
Convergence code/security SHA: **`8af72620f12e2494a59830a23abb393768b610e9`**
(the following docs commit only adds this report and release material; code is
@@ -42,9 +169,14 @@ byte-identical).
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
430 passed, 0 failed, 2 ignored; includes 7/7 Linux security tests
491 passed, 0 failed, 2 ignored; includes 7/7 Linux security tests
```
The documentation and test migration is local work; re-run this block after it
lands and update the count from the log rather than copying the number.
`docs_consistency`, `architecture_boundaries` and `stdlib_product` run inside
this workspace suite.
### G-C security lane (disposable Ubuntu 26.04 VM, ordinary user `dev`)
Source: `git archive 70d916d` extracted in the VM (not the dev container).
@@ -67,19 +199,25 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
## 2. Work package status
> **Historical table note.** Most rows in this section record work executed
> before the RFC-LANG-0002 syntax migration. Language-source rows (fixtures,
> control mapping, semantic goldens) were re-verified after the rewrite at
> `9c1723d` (§0); platform/security rows keep their original evidence dates and
> are not refreshed by the language migration.
| # | Work package | State | Evidence |
|---|---|---|---|
| 0 | Windows AST line endings | Fixed locally; **Windows CI UNVERIFIED** | `.gitattributes`, CRLF fixture |
| 1 | macOS native plugin fixtures | Fixed locally; **macOS CI UNVERIFIED** | host-target manifest + dylib flags |
| 2 | Cross-platform strict `cfg` | **Fixed locally; native CI UNVERIFIED** | `#[cfg]` dispatch + non-Linux regression test |
| 3 | Strict probe→launch race | **Fixed locally; GitHub Linux CI UNVERIFIED** | fail-closed CLI contract + probe-OK/launch-EAGAIN seam test |
| 0 | Windows AST line endings | **Windows CI PASS** (`5fcc111`) | `.gitattributes`, CRLF fixture |
| 1 | macOS native plugin fixtures | **macOS CI PASS** (`5fcc111`) | host-target manifest + dylib flags |
| 2 | Cross-platform strict `cfg` | **Native CI PASS** (`5fcc111`) | `#[cfg]` dispatch + non-Linux regression test |
| 3 | Strict probe→launch race | **Linux CI PASS** (`5fcc111`) + deterministic seam | fail-closed CLI contract + probe-OK/launch-EAGAIN seam test |
| 4 | Parallel security matrix isolation | **Fixed** | unique paths + helper `OnceLock`; 7/7 in parallel workspace run |
| 5 | Markdown/power/control-flow corpus | **Done locally** | 160 AST, 73 semantic, 52 compile-negative, 21 runtime-negative fixtures |
| 6 | Condition header + source spans | **Fixed** | executable tails rejected; literal-inside-backtick mapping tests |
| 5 | Markdown/power/control-flow corpus | **Green on semantics v2** | 160 AST + 73 semantic + 49 compile-negative + 2 warning + 21 runtime-negative fixtures; `conformance_ast`/`semantics`/`negative` pass |
| 6 | Payload/control source spans | **Green on semantics v2** | UTF-8 spans, control-expression offsets and heading boundaries covered by fixtures and CLI tests; old condition headers were removed |
| 7 | Independent Tool consumers | **Done locally** | `sample-view`/`sample-knowledge`/`sample-proof` E2E test |
| 8 | lock v3 reproducibility | **Implemented; tamper test added** | `package_env` PASS |
| 9 | Analysis/Trace/Tool/ABI/Foreign | **Implemented; supported-matrix bounded** | protocol tests, plugin/worker/foreign tests |
| 10 | Linux strict L1 | **VM PASS at 70d916d; strict needs a dedicated CI runner** | 7/7 + positive CLI + strict resource refusal; general Linux Docker CI is green |
| 10 | Linux strict L1 | **Local VM PASS at 5fcc111; remote G3 runner still missing** | 7/7 + positive CLI + strict JSON/inert-`@knowledge`; [`strict-gc-5fcc111`](evidence/2026-11-21-strict-gc-5fcc111.md) |
| 11 | Linux strict L2 | **NOT IMPLEMENTED / UNSUPPORTED** | strict rejects Worker/Foreign/native ABI |
| 12 | Windows Native/Worker ABI | **UNSUPPORTED** | no Windows ABI artifact/claim |
| 13 | RC artifact | **Linux candidate only; no release** | clean-install + checksums; macOS not built here |
@@ -88,9 +226,10 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
## 3. Verified capabilities
- Frozen Markdown profile `marqdo.md.v1` with 160 AST goldens and explicit
CRLF byte semantics.
- Native Rust interpreter with 73 semantic goldens, 52 compile-error fixtures,
and runtime error source attribution.
CRLF byte semantics (unchanged by the semantic rewrite).
- Native Rust interpreter with the unified semantics: 73 semantic fixtures,
49 compile-error fixtures, 2 warning fixtures and 21 runtime-error fixtures
all green at `9c1723d`; AST profile unchanged (160 goldens).
- Native `**` power is pinned end-to-end: right associativity, unary-minus
precedence, parentheses, float negative exponents, branch/loop conditions,
Strong+CodeSpan payloads, and inert prose/code fences.
@@ -133,7 +272,15 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
`打印/print` to the bundled `io.写值` when the program uses it; stdlib
modules are injected lazily; resource operations are trace `effect` events
and `HostOp::Print` no longer runs on that path. Strict permits only
stdio/stdout; file/network resources remain refused.
stdio/stdout; stdin/stderr/file/network resources remain refused.
- Stdio streams and console productization (ADR-0040): `stdio/stdin` is
read-only over one shared run cursor; `stdio/stdout`/`stdio/stderr` are
write-only with separate host buffers and output-budget accounting; the CLI
copies non-terminal process stdin at composition and embeds program stderr as a
separate JSON string (or writes it to CLI stderr). `stdlib/io/stream.mq.md`
checks short writes and close errors; `stdlib/io/console.mq.md` implements
line input and print through the resource ABI; `stdlib/text/utf8.mq.md`
provides a validating UTF-8 decoder.
- Full-value stdlib formatting (ADR-0029): FunctionRef `module/name` and
PluginHandle `plugin/export` are read-only observation members (host id
private); `stdlib/format.mq.md` reproduces builtin `打印`/`repr` for all 15
@@ -169,9 +316,22 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
- Table stdlib algorithms (CM-A08, ADR-0032): `stdlib/data/table.mq.md`
provides select/filter/sort/append/set/join/group/`转GFM` over the logical
`Value::Table`; inputs are never mutated; GFM AST laziness and physical T0
storage are unchanged.
storage are unchanged. `std:fs/file` provides bounded `读文本`/`写文本`/`复制文件`
over the existing `resource.*` ABI plus `存在`/`元数据`/`列出目录`/`创建目录`/
`删除`/`重命名`/`追加文本`/`读取片段`/`遍历` over `resource.request`
(ADR-0041/0045); `data/json`/`data/csv`
provide recursive JSON (128-depth bound, multi-value) and quoted single/multi-row
CSV product parsers; `path` and a `regex` subset are pure `.mq.md`; `datetime` is pure UTC ISO-8601 and
`clock.现在` reads `clock.now` in project mode (ADR-0042); `env.读取` reads
only exact `--allow-env NAME` grants in project mode (ADR-0043); `process.运行`
runs an exact `--allow-process PROGRAM` grant with 5 s timeout and 1 MiB caps
(ADR-0044, not an OS sandbox); `random.字节` reads explicitly granted
`random.bytes` entropy (ADR-0046). File watch, binary offset reads, streaming
incremental JSON/CSV streaming, pure-mode regex groups/alternation and regex
look-around/backreferences remain open; project mode has host-backed groups
and captures (ADR-0050).
- Bundled standard library (CM-A10, ADR-0028): `stdlib/` ships
`format`/`text`/`io`; explicit `std:` imports resolve only inside the bundle;
`format`/`text`/`io`/`fs`/`data`/`path`/`regex`/`datetime`/`clock`/`env`/`process`/`random`/`compress`/`net/http`; explicit `std:` imports resolve only inside the bundle;
a SHA-256 digest over `*.mq.md` is verified before loading; Analysis records
`project.stdlib`; a frozen-binary test asserts the CLI hash is unchanged after
running a `std:io` project and that tampering is rejected. `打印/print` is
@@ -181,13 +341,33 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
([evidence](evidence/2026-11-20-stdlib-package.md)). The stdlib intentionally
stays in this repository for now (ADR-0039); extraction is a future option
when size/release cadence requires it, not a pending gate.
- Resource streams (RFC-CM CM-A06, ADR-0025): opaque host-issued `Resource`
handles with `resource.open/read/write/close`; bytes move only through
`Memory`; stdout write is capability/output-budget checked and project-file
- Resource streams (RFC-CM CM-A06, ADR-0025/0026/0040): opaque host-issued
`Resource` handles with `resource.open/read/write/close`; bytes move only
through `Memory`; stdin is read-only over one shared run cursor; stdout and
stderr are separate capability/output-budget checked buffers; project-file
read/write is path/grant checked; file writes buffer until `close` and
never flush on drop; a `.mq.md` chunked copy loop reproduces a file without a
Rust `copy_file`; strict permits only the stdio console channel and refuses
file/TCP/provider resources (CM-A11 lane PASS). Semantic fixture 66 and CLI/host tests.
Rust `copy_file`; `resource.request` adds non-streaming `fs.exists`/
`fs.metadata`/`fs.list`/`fs.mkdir`/`fs.remove`/`fs.rename` (project mode,
grant-checked, recoverable fs Result); strict permits only the stdio stdout
console channel and refuses stdin/stderr/file/TCP/provider resources (CM-A11
lane PASS). Semantic fixture 66 and CLI/host tests.
- Test and documentation migration (review RFC-CM P0/P2):
`crates/cli/tests/docs_consistency.rs` pins one current overview, the
status/index links and SHA-pinned evidence; `crates/cli/tests/architecture_boundaries.rs`
pins the workspace graph, `marqdo-runtime -> marqdo-host`, the bundled stdlib
digest and the stdio/file/TCP endpoints; `crates/cli/tests/stdlib_product.rs`
runs every public stdlib module (`clock/compress/console/csv/datetime/demo/env/format/fs/http/io/json/path/process/random/regex/stream/table/text/utf8`)
through `std:` imports over `stdlib/tests/*.mq.md`. `tools/pack-stdlib.py`
now sorts `.mq.md` by the same relative path key as the CLI. Source/test baseline
`0d40de7`; later docs-only commits keep code/test bytes identical. Evidence:
[`docs-and-test-migration`](evidence/2026-11-21-docs-and-test-migration.md).
- Module-boundary refactor (review RFC-CM P1): CLI split into
`commands/{sandbox,run,check,inspect,init,package,tool}.rs`; host split into
`host/{resource_ops,providers,effects,dispatch}.rs` plus `process.rs`.
`commands.rs` keeps shared project/runtime helpers and `host.rs` keeps the
`Host` trait methods and capability checks; this is a behavior-preserving
split, not a new crate or backend.
- Minimal structure observation/construction (RFC-CM CM-A04, ADR-0023):
value-semantics `list.append/concat`, `map.keys/has/get/set/remove`,
Unicode scalar `text.codepoints`/`text.from_codepoint`, and non-negative
@@ -195,26 +375,31 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
## 4. Open blockers and limits
1. **No native GitHub/Windows/macOS CI evidence.** The private Woodpecker
Linux Docker pipeline (fmt/clippy/test/release-build) is green, but Windows
and macOS native lanes are still unavailable. G1/G2/G3 remain FAIL until a
real native run exists.
1. **Native matrix is green; remote G3/G4 are not.** GitHub run 30 on
`5fcc111` PASSes Linux, Windows and macOS, and Woodpecker #25 is green. The
remaining release blockers are a remote G3 security artifact and a
`MEASURED` G4 report on the fixed node.
2. **G-C is VM-verified but not CI-signed.** The Woodpecker Docker pipeline is
not an OS-isolation lane; no self-hosted security runner exists yet.
not an OS-isolation lane; no ephemeral self-hosted security runner has run
`security-linux.yml` for this SHA.
3. Windows Native/Worker ABI is explicitly UNSUPPORTED (no Windows `cc`
fixture lane, Unix-gated worker tests).
4. `fs.write`, `net.request`, `process`, `clock`, `random` are not executed
broker services; unsupported requests must return `capability.unavailable`.
5. Resource streams cover project-file read/write, stdout, compatible print,
trusted in-process providers and IP-literal TCP; directories, process,
device, DNS/TLS/HTTP, UDP and provider packaging remain open. Strict permits
only stdio and refuses the rest (CM-A11 lane PASS).
5. Resource streams cover project-file read/write (including `std:fs/file`
read/write/copy/append/read_at/walk), non-streaming `resource.request`
`fs.*` metadata/directory operations, stdout, compatible print, trusted
in-process providers and TCP/UDP/listener/TLS endpoints with DNS
resolution after an exact host grant. File watch, device and provider
packaging remain open. Strict permits only stdio and refuses the rest (CM-A11 lane
PASS).
6. uv/npm/venv provisioning is not implemented. Only a pre-provisioned,
digest-pinned directory with a regular-file executable is supported.
7. No Windows/macOS RC archive exists from this host. The release-candidate
workflow can build macOS on GitHub, but it has not run for this SHA.
7. No tagged Windows/macOS RC archive exists. The native **build/test** lanes
PASS at `5fcc111`; the release-candidate job is tag/dispatch gated and has
not produced those archives.
8. No release tag/binary publication. The release decision is **DO NOT
RELEASE** until the native CI matrix and a security-lane CI URL exist.
RELEASE** until the remote G3 artifact and the MEASURED G4 report exist.
## 5. RFC-CM compute-machine status
@@ -225,8 +410,8 @@ behavior. This table records only what is in this tree now.
|---|---|---|
| P0 freeze (ADRs, spec deltas, baseline) | **P1 + partial P2 slice done** | ADR-0022 freezes CM-A01/CM-A02/CM-A03; ADR-0023 freezes CM-A04 (minimal List/Map/Text/Table observation and construction); ADR-0024 freezes CM-A05 (recoverable `Result` map convention); `spec/expression.ebnf`, `spec/evaluation.md`, `spec/value-wire.md`, `spec/analysis-v2.md`, `spec/security.md` updated; existing AST/value goldens unchanged |
| P1 Memory + place assignment | **Implemented locally** | `Value::Memory`, run-scoped `MemoryStore` (64 KiB pages, 64 MiB total / 16 MiB region), `memory.new/size/grow`, `m[i]` read, `m[i] = byte`; semantic fixtures 45–47, runtime negatives 13–18, `examples/memory-buffer.mq.md`; evidence [`evidence/2026-11-20-memory-p1.md`](evidence/2026-11-20-memory-p1.md) |
| P2 Marqdo-written allocator/data structures + minimal structure observation | **PASS (local pure lane); stdlib remains in-repo by design** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), a quoted/escaped CSV row parser (68), table group (69), an Int matrix multiply (70), and a sparse-matrix multiply (73); the first-fit allocator also detects double free (`alloc.double_free`). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). Local gate PASS: `crates/cli/tests/p2_closure.rs` runs 21 C01–C06/C11/C12 fixtures plus the stdlib table self-test in `--mode pure` (no foreign/plugin/import grants), scans sources for foreign/plugin/import shortcuts and compares value/stdout to goldens; evidence [`p2-closure-gate`](evidence/2026-11-20-p2-closure-gate.md). The bundled `stdlib/` tree, `std:` imports, digest verification, Analysis identity and frozen-CLI closure test are implemented (ADR-0028). The stdlib intentionally remains in-repo (ADR-0039); a separate repository is a future extraction option, not a gate. Linux Docker CI is green; a strict security runner and Windows/macOS lanes are still pending. |
| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict permits only stdio/stdout and refuses file/network resources. Fixture 66 + CLI/host tests. CM-A07 **Compat-C** is implemented: when a program calls `打印/print`, the loader injects the bundled `stdlib/io.mq.md` and the runtime routes to `io.写值` (all 15 value types, UTF-8 via `Memory`, `resource.write(stdout)`); `HostOp::Print` and the legacy `Value::display()` print path are removed, and a missing bundle fails with `kernel.unavailable`. Strict permits only the existing stdio console channel. Providers are registered through ADR-0033 (in-process) and ADR-0035 (worker helper). Still missing: directories, C ABI provider binding, provider packages, strict provider support. |
| P2 Marqdo-written allocator/data structures + minimal structure observation | **PASS (local pure lane); stdlib remains in-repo by design** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), a quoted/escaped CSV row parser (68), table group (69), an Int matrix multiply (70), and a sparse-matrix multiply (73); the first-fit allocator also detects double free (`alloc.double_free`). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). Local gate PASS: `crates/cli/tests/p2_closure.rs` runs 21 C01–C06/C11/C12 fixtures plus the stdlib table self-test in `--mode pure` (no foreign/plugin/import grants), scans sources for foreign/plugin/import shortcuts and compares value/stdout to goldens; evidence [`p2-closure-gate`](evidence/2026-11-20-p2-closure-gate.md). The bundled `stdlib/` tree, `std:` imports, digest verification, Analysis identity and frozen-CLI closure test are implemented (ADR-0028). The stdlib intentionally remains in-repo (ADR-0039); a separate repository is a future extraction option, not a gate. Linux/Windows/macOS native CI is green at `5fcc111`; a remote G3 security runner and a fixed G4 performance node are still pending. |
| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026/0040 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdin read, separate stdout/stderr write, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict permits only stdio/stdout and refuses stdin/stderr/file/network resources. `stdlib/io/stream.mq.md`, `stdlib/io/console.mq.md` and `stdlib/text/utf8.mq.md` are product-level `.mq.md` APIs; fixture 66 + CLI/host tests. CM-A07 **Compat-C** is implemented: when a program calls `打印/print`, the loader injects the bundled `stdlib/io.mq.md` and the runtime routes to `io.写值` (all 15 value types, UTF-8 via `Memory`, `resource.write(stdout)`); `HostOp::Print` and the legacy `Value::display()` print path are removed, and a missing bundle fails with `kernel.unavailable`. Strict permits only the existing stdio console channel. Providers are registered through ADR-0033 (in-process) and ADR-0035 (worker helper). Still missing: directories, C ABI provider binding, provider packages, strict provider support. |
| P4 Table logical/physical migration | **PASS (local gate); T1/T2/T3 future** | ADR-0032/CM-A08: `Value::Table` semantics and GFM AST laziness unchanged; `stdlib/data/table.mq.md` provides select/filter/sort/append/set/join/group/`转GFM` as value-returning `.mq.md` functions with input immutability; zero Rust table algorithms were added. Cross-storage equivalence is pinned by fixture 72 (row Table vs `.mq.md` columnar store, identical headers/rows/cells). The optional T1/T2/T3 physical-storage/column-store migration was not required by the gate and is not implemented. |
| P5 third-party Provider / TCP / HTTP | **PASS (local gate); external network/CI and plugin-Worker resource binding open** | C10: `stdlib/net/http.mq.md` + fixture 71 parse HTTP/1.1 status/headers/chunked bodies and build requests; `http_over_tcp.rs` runs a loopback HTTP exchange wholly in `.mq.md` over `resource.open("tcp",...)`. ADR-0034 IP-literal blocking TCP endpoint (exact IP grant, project-only, 5s timeouts) and ADR-0033 in-process `ResourceProvider` v1 (registration, protocol/kind checks, name grant, mock read/write provider) ADR-0035 host-managed worker provider transport (framed JSON helper, ping/open/read/write/close, trusted-only) and ADR-0036 trusted C ABI provider (shared-library symbols, version/load checks) ADR-0037 local provider-package manifest installation and ADR-0038 C ABI provider worker binding (separate trusted helper process) are tested. External DNS/TLS/UDP, host HTTP, provider packages/signatures and the plugin-Worker resource binding remain out of scope. |
| P6 performance / native backends | **BASELINE ONLY** | reproducible profile harness `tools/perf/bench.py` and [`p6-profile`](evidence/2026-11-20-p6-profile.md) for 7 workloads; no bytecode/JIT/columnar numeric backend or equivalence/optimization gate yet |
@@ -300,6 +485,15 @@ user-written allocator is explicitly **not** claimed.
print, strict file-resource refusal, forged-marker rejection and raw log.
- [`evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md`](evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md)
— G0..G12 matrix and RC decision.
- [`evidence/2026-11-21-strict-gc-5fcc111.md`](evidence/2026-11-21-strict-gc-5fcc111.md)
— RFC-CI-0001 series G-C: local disposable VM, `dev` uid 1000, 7/7 plus the
strict JSON and inert-`@knowledge` contracts at `5fcc111`; the earlier `512b9f5`/`0d7e11b` runs are historical.
- [`evidence/2026-11-21-perf-woodpecker-agent-1.md`](evidence/2026-11-21-perf-woodpecker-agent-1.md)
— G4 `MEASURED` provisional baseline on the designated Woodpecker agent
(`1199c88`): 9/9 correctness-verified, 5 warmups + 20 samples, repeatability
deltas, and the shared-host/powersave caveat.
- [`evidence/2026-11-21-perf-smoke-0d7e11b.md`](evidence/2026-11-21-perf-smoke-0d7e11b.md)
— earlier `marqdo.perf.v2` smoke; historical, not the release baseline.
- [`security-gate.md`](security-gate.md) — mechanism, attack matrix and strict
L1/L2 limits.
@@ -323,8 +517,17 @@ python3 tests/consumers/analysis_consumer.py /tmp/a.json
python3 tests/consumers/trace_consumer.py /tmp/a.json /tmp/t.json
```
G-C (ordinary user, disposable VM only):
G-C (ordinary user, disposable VM only; bind the source archive and write the
evidence document):
```bash
MARQDO_REQUIRE_STRICT_SECURITY=1 ./tests/security/run-lane.sh
export MARQDO_TESTED_SHA="$(git rev-parse HEAD)"
export MARQDO_SOURCE_ARCHIVE="/path/to/marqdo-${MARQDO_TESTED_SHA}.tar"
export MARQDO_SOURCE_ARCHIVE_SHA256="sha256:<host-computed digest>"
export MARQDO_STRICT_EVIDENCE="/tmp/evidence-${MARQDO_TESTED_SHA}/strict-evidence.json"
export MARQDO_REQUIRE_STRICT_SECURITY=1
./tests/security/run-lane.sh
```
A root-only PASS, a container PASS or a missing/shorter archive is not
evidence. See [`security-vm.md`](security-vm.md).
@@ -1,6 +1,14 @@
# ADR-0002: Executable nodes execute only inside level-2 function bodies
Status: **Accepted**
> **Historical / superseded in part.** The execution-scope decision (payloads
> only inside an owning H2 function, never at module top level) remains, but
> the carrier/role rules in this ADR (`Strong` executes, `Emph` returns, lists
> and quotes are instructions) were replaced by
> [ADR-0052](0052-unified-markdown-semantics.md) and
> [`spec/mapping.md`](../../spec/mapping.md). The syntax examples below are the
> syntax at the time of the decision, not current syntax.
Status: **Accepted (historical mapping superseded by ADR-0052)**
Date: 2026-11-20
## Context
@@ -1,5 +1,14 @@
# ADR-0003: `ref` resolution order and `ref.ambiguous` (S-07)
> **Historical syntax note.** The AST shape below uses the pre-RFC-LANG-0002
> spelling, where a `strong` node carried an executable statement. Under the
> current contract
> ([RFC-LANG-0002](../architecture/RFC-LANG-0002-unified-semantics.md),
> [ADR-0052](0052-unified-markdown-semantics.md),
> [`spec/mapping.md`](../../spec/mapping.md)), `Strong`, `Emph` and inline
> `Code` are interchangeable one-statement payload carriers inside an `@exec`
> scope. The `ref` resolution decision itself remains current.
Status: **Accepted**
Date: 2026-11-20
+11
View File
@@ -1,5 +1,16 @@
# ADR-0005: AST normalization (Setext headings, task items, code-span ticks, strikethrough)
> **Historical mapping note.** The profile/AST normalization rules in this ADR
> remain current. Its role claims — that ordered/bullet list structure
> "determines branch/loop roles" and that `strong`/`emph` carry execute/return
> roles — were superseded by
> [RFC-LANG-0002](../architecture/RFC-LANG-0002-unified-semantics.md) /
> [ADR-0052](0052-unified-markdown-semantics.md) and
> [`spec/mapping.md`](../../spec/mapping.md): lists and block quotes are
> presentation containers, and `Strong`, `Emph` and inline `Code` are
> interchangeable one-statement payload carriers. Strikethrough and the other
> inert regions stay inert.
Status: **Accepted**
Date: 2026-11-20
+8
View File
@@ -55,6 +55,14 @@ Codes are stable contract. Diagnostic **codes** (for example
`markdown.invalid_construct`, `expr.parse`, `symbol.unknown`,
`permission.denied`, `ref.ambiguous`) are stable; message wording is not.
> **Historical diagnostic note.** `markdown.invalid_construct` (and the
> executable-shape `Strong`/`Emph`/list/quote errors it described) belongs to
> the pre-RFC-LANG-0002 mapping. Under the current contract
> ([RFC-LANG-0002](../architecture/RFC-LANG-0002-unified-semantics.md) /
> [ADR-0052](0052-unified-markdown-semantics.md),
> [`spec/mapping.md`](../../spec/mapping.md) §8) syntax-role failures use the
> `sem.*` codes. The rest of this CLI contract is unaffected.
### 3. Stream separation
* **Program output** (`打印`/`print`) goes to stdout in text mode. In JSON mode
@@ -1,5 +1,13 @@
# ADR-0007: Arbitrary-precision `Int` and the typed wire encoding (S-08)
> **Historical syntax note.** The numeric model and typed wire-encoding
> decisions remain current. The phrase "code spans/plain text of
> `strong`/`emph`/conditions" below describes the pre-RFC-LANG-0002
> payload/condition mapping; under
> [RFC-LANG-0002](../architecture/RFC-LANG-0002-unified-semantics.md) /
> [ADR-0052](0052-unified-markdown-semantics.md), payloads are carried by
> `Strong`, `Emph` or inline `Code` and control flow is keyword based.
Status: **Accepted**
Date: 2026-11-20
@@ -1,5 +1,14 @@
# ADR-0022: Addressable `Memory` and Memory-index place assignment (RFC-CM P1)
> **Historical syntax note.** The `Memory` index place-assignment decision
> remains current. The sentence below that "the surface syntax remains a
> `strong` node … no Markdown role, heading or list meaning changes" is the
> historical pre-RFC-LANG-0002 spelling: under
> [RFC-LANG-0002](../architecture/RFC-LANG-0002-unified-semantics.md) /
> [ADR-0052](0052-unified-markdown-semantics.md), `m[i] = b` is carried by any
> `Strong`/`Emph`/inline `Code` payload inside an `@exec` scope, and lists,
> quotes and headings carry no instruction role.
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_MACHINE_RFC_0001.md` (CM-A01/CM-A02/CM-A03),
+3 -3
View File
@@ -61,9 +61,9 @@ legacy `HostOp::Print` effect does not appear in Compat-B runs.
Strict L1 already permits the console channel. Compat-B needs stdout writes in
strict so the same print implementation works there. Therefore strict mode
permits **only** `stdio/stdout` write with the console grant; project-file,
network, process and device resource opens remain refused, and no new provider
is enabled. This is the existing console capability reached through the
permits **only** `stdio/stdout` write with the console grant; stdin/stderr,
project-file, network, process and device resource opens remain refused, and no new
provider is enabled (ADR-0040 keeps the later stdin/stderr slice separate). This is the existing console capability reached through the
resource ABI, not a new OS capability. File resources stay fail-closed.
### 5. Compatibility and fallback
@@ -0,0 +1,60 @@
# ADR-0040: Stdio stdin/stderr endpoints and stdlib stream APIs
Status: **Accepted**
Date: 2026-11-21
Relates to: ADR-0025, ADR-0026, ADR-0027, ADR-0030, ADR-0031
Specs: `spec/evaluation.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-21-stdio-streams.md`
## Context
ADR-0025 opened one `stdio/stdout` write endpoint. That left `stdin` and
`stderr` as hidden OS channels, and the bundled `io.写文本` ignored the
`resource.write` `{ok,count}` result. The audit RFC calls for a real resource
boundary, not a Rust `Input`/`Print` builtin.
## Decision
* Add `stdio/stdin` (rights `["read"]`) and `stdio/stderr` (rights
`["write"]`). `stdio/stdout` remains `["write"]`. Rights/kinds are exact;
mixing them is `resource.unsupported`.
* All stdin handles in one run share one cursor. `resource.read` consumes bytes and
reports `{count,eof}`; EOF is a successful zero-byte read, not an error.
Closing a handle does not reset or close the run cursor. There is no blocking
OS read inside the host table.
* stdout and stderr are distinct host buffers. Both charge the existing output
budget, but `output()`/`take_output()` never return stderr and
`error()`/`take_error()` never return stdout. A budget failure appends
nothing to either buffer.
* The CLI reads process stdin once at composition time only when stdin is not a
terminal. This keeps an interactive `marqdo run` from waiting for EOF. JSON run
output adds a separate `stderr` string; text mode writes captured program
stderr to process stderr. Diagnostics and traces remain CLI stderr and never mix
with the JSON stdout document.
* Strict mode remains fail-closed: only `stdio/stdout` with `["write"]` and the
console grant is allowed. stdin, stderr, file, TCP and provider streams are
`permission.denied`.
* The standard library keeps the product APIs in `.mq.md`:
`stdlib/io/stream.mq.md` owns `写全部`/`写文本` with short-write and close
checking; `stdlib/io/console.mq.md` owns `读取行`/`打印` over the resource
ABI; `stdlib/text/utf8.mq.md` owns a validating UTF-8 decoder.
`stdlib/io.mq.md` remains the compatibility facade and print target.
## Non-goals
No process, device, directory, DNS, TLS, UDP, seek, mmap, async or zero-copy
endpoint. The stdin source is the finite process input supplied by the
composition root, not a general interactive console driver. No new language
keyword, `Input` builtin, `HostOp::Print` or second Markdown parser.
## Compatibility
The resource ABI is additive. Existing stdout-only programs, `打印`, file
read/write and provider/tcp streams keep their prior behavior. A program that
used a closed handle still gets the recoverable `resource.closed` result.
## Consequences
CLI programs can now read piped stdin and write program stderr without leaking
either into JSON diagnostics. The remaining OS-shaped gaps are explicit and can be
added only through a separate resource operation decision.
+61
View File
@@ -0,0 +1,61 @@
# ADR-0041: Non-streaming Resource Operation for filesystem metadata and directories
Status: **Accepted**
Date: 2026-11-21
Relates to: ADR-0025, ADR-0026, ADR-0040
Specs: `spec/evaluation.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-21-fs-file-product.md`, `docs/evidence/2026-11-21-resource-operation.md`
## Context
`resource.open/read/write/close` is a byte-stream ABI. It cannot express
operations that do not return a byte stream: `exists`, `metadata`, directory
listing, `mkdir`, `remove` and `rename`. Adding one Rust builtin per operation,
or faking them by reading a directory as a file, would either grow the language
surface without a contract or overstate what a stream can do.
The audit RFC's P1 items require a separate, explicit capability class for
these non-streaming operations, not a silent `HostOp` for every filesystem
call.
## Decision
Add one builtin and one host trait method:
```text
resource.request(operation: Text, arguments: List) -> Result
Host::resource_request(operation: &str, arguments: Vec<Value>, ctx) -> Result<Value, RuntimeError>
```
* The first version registers only the `fs.*` operation group. An unknown
operation is fatal `resource.unsupported`; it is not a recoverable Result.
* Arguments must be `Text` paths. `Memory` and `Resource` handles are rejected
before dispatch, so they still cannot cross the host boundary.
* Every call is project-mode only. `pure` and `strict` return
`permission.denied`; this does not widen strict L1.
* Grants: `fs.exists`/`fs.metadata`/`fs.list` require `fs_read`; `fs.mkdir`/
`fs.remove` require `fs_write`; `fs.rename` requires `fs_write` on both
paths. Grants are checked before path resolution or any I/O.
* Paths use the same `ProjectFs::resolve` policy: relative, no `..`, no
absolute path, canonical target must stay inside the project root.
* Expected filesystem conditions are recoverable `{ok:false, code, message}`:
`fs.not_found`, `fs.not_directory`, `fs.exists`, `fs.not_empty`. Capability
denial, traversal, unsupported operations and I/O failures stay fatal
`RuntimeError`s.
* `remove` deletes a file or an **empty** directory; it never recurses.
`rename` refuses an existing target with `fs.exists` instead of overwriting.
* `mkdir` is single-level: a missing parent returns `fs.not_found`.
* Every call records one semantic `resource.request` and one effect event
with `op=fs.<name>`. No bytes move through the operation; file content still
uses `resource.read`/`resource.write`.
## Consequences
* The language surface grows by exactly one builtin (`resource.request`), not
one per operation. The operation strings are a versioned stdlib ABI, not new
Markdown syntax.
* A future directory walk, seek, stat or process operation can reuse the same
shape without inventing a new `HostOp`; it still needs its own ADR and
capability group.
* `std:fs/file` is a `.mq.md` facade over both ABIs: byte streams for content,
`resource.request` for metadata/directories. No Rust algorithm is added.
+40
View File
@@ -0,0 +1,40 @@
# ADR-0042: `clock.now` resource operation
Status: **Accepted**
Date: 2026-11-21
Relates to: ADR-0041
Specs: `spec/evaluation.md`
Evidence: `docs/evidence/2026-11-21-datetime-clock-product.md`
## Context
The bundled standard library needs a product clock API. Reading the OS clock
inside a Rust builtin would add a second side-effect path beside
`resource.request`, and making time available in every mode would break the
pure/strict fail-closed contract without an explicit capability decision.
## Decision
Extend `resource.request` with one non-filesystem operation:
```text
resource.request("clock.now", []) -> {ok:true, value:UnixMillis}
```
* Zero arguments; any argument is a fatal `call.arity` error.
* Project mode only. `pure` and `strict` return `permission.denied`, like
`fs.*`.
* The value is UTC Unix milliseconds, clamped to `i64`. A clock before the
Unix epoch is a fatal `host.error`.
* The call records the same `resource.request` semantic/effect events with
`op=clock.now`.
* `stdlib/clock.mq.md` is a thin `.mq.md` facade; `stdlib/datetime.mq.md` is a
pure UTC formatter/parser over Unix milliseconds and has no host dependency.
## Consequences
* The clock remains a capability-gated host operation, not a language keyword.
* No new CLI flag is introduced; project mode already implies the existing
host boundary. A future per-call clock grant can replace this default.
* Monotonic time, time zones and locale formatting are not part of this
decision.
+48
View File
@@ -0,0 +1,48 @@
# ADR-0043: Explicitly granted `env.get` resource operation
Status: **Accepted**
Date: 2026-11-21
Relates to: ADR-0041, ADR-0042
Specs: `spec/security.md`, `spec/evaluation.md`
Evidence: `docs/evidence/2026-11-21-env-product.md`
## Context
A product `std:env` module needs to read process environment variables. The
environment is an ambient OS side channel that can contain secrets and
credentials. Reading it through a wildcard path grant or by default would
silently widen the capability surface; `env.list` would expose every variable
without naming it.
## Decision
Add one operation to `resource.request`:
```text
resource.request("env.get", [name]) -> {ok:true, value:Text}
| {ok:false, code:"env.not_found"}
| {ok:false, code:"env.not_unicode"}
```
* Arguments: exactly one `Text` variable name. Non-Text or wrong arity is a
fatal type/arity error.
* Grants: a new `Capabilities::env: Vec<String>` with **exact name matching**
(no glob). `--allow-env NAME` adds one grant; un-granted names return
`permission.denied` before the OS is read.
* The check happens before `std::env::var`. Project mode only; `pure` and
`strict` refuse. No default grant.
* Missing variables and non-UTF-8 variables are recoverable
`env.not_found`/`env.not_unicode`, not fatal errors.
* There is no `env.list`, `env.set`, process spawning or `std::env::set_var`
in this decision. A future `env.list` would need a separate ADR because it
changes the disclosure model.
* `stdlib/env.mq.md` is a thin `.mq.md` facade: `读取` returns the Result and
`存在` maps `env.not_found` to `{ok:true, value:false}`.
## Consequences
* The environment is readable only by exact, operator-supplied names in
project mode.
* The CLI gains `--allow-env NAME`; strict mode rejects it with exit 2.
* `std:process` remains unsupported in this decision; process execution needs
its own capability and OS-sandbox story.
@@ -0,0 +1,52 @@
# ADR-0044: Explicitly granted `process.run` resource operation
Status: **Accepted**
Date: 2026-11-21
Relates to: ADR-0041, ADR-0042, ADR-0043
Specs: `spec/security.md`, `spec/evaluation.md`
Evidence: `docs/evidence/2026-11-21-process-product.md`
## Context
The audit RFC lists a `std:process` product module. Spawning a process is the
most privileged OS operation in the runtime and is **not** contained by the
in-process host checks. Without an explicit contract it would silently turn
the CLI into an unrestricted command runner.
## Decision
Add a project-mode-only `resource.request` operation:
```text
resource.request("process.run", [program, args]) -> {ok:true, value:{status, stdout, stderr}}
| {ok:false, code:"process.not_found"}
| {ok:false, code:"process.timeout"}
| {ok:false, code:"process.output_too_large"}
```
* `program` is an exact Text path; `args` is a List of Text values. Wrong
arity/types are fatal. There is no shell expansion, no glob and no implicit
`PATH` lookup.
* A new exact `Capabilities::process_programs: Vec<String>` grant is required
(`--allow-process PROGRAM`). Un-granted programs return
`permission.denied` before spawning.
* The child runs with `env_clear`, stdin `null`, the project root as its
working directory, and a 5 second wall-clock limit. On timeout it is killed
and the call returns recoverable `process.timeout`.
* `stdout`/`stderr` are captured separately, require valid UTF-8 (otherwise
fatal type error), and are capped at 1 MiB each; exceeding the cap is
recoverable `process.output_too_large`.
* `pure`/`strict` refuse the operation. This does **not** make the child an OS
sandbox: the in-process host check is cooperative and the child inherits the
CLI user's OS privileges.
* `stdlib/process.mq.md` is a thin `.mq.md` facade; no Rust builtin or
`HostOp` is added.
## Consequences
* Process execution is opt-in per exact program path and remains project-mode
only.
* The operation is not a sandbox. A future strict-mode process provider needs
OS isolation evidence and a separate ADR.
* `--allow-process` is rejected by strict preflight, like the other external
grants.
+46
View File
@@ -0,0 +1,46 @@
# ADR-0045: `fs.append`, `fs.read_at` and `fs.walk`
Status: **Accepted**
Date: 2026-11-21
Relates to: ADR-0041
Specs: `spec/evaluation.md`
Evidence: `docs/evidence/2026-11-21-fs-extended.md`
## Context
ADR-0041 added metadata/directory operations, but file content still required
a whole-file `resource.open` read and a whole-buffer write. That leaves append,
offset reads and recursive directory discovery unavailable to `.mq.md` without
adding Rust algorithms to the standard library.
## Decision
Extend the existing `fs.*` group of `resource.request`:
```text
resource.request("fs.append", [path, text]) -> {ok:true, value:count}
resource.request("fs.read_at", [path, offset, len])-> {ok:true, value:Text}
resource.request("fs.walk", [dir]) -> {ok:true, value:List<Text>}
```
* `fs.append` writes UTF-8 text to the end of a project file, creating it when
missing. It requires `fs_write`, is bounded by the existing 8 MiB file limit,
and never bypasses the path policy.
* `fs.read_at` reads at most `len` bytes from `offset` and returns Text. Both
values are non-negative integers; `len` is capped at 8 MiB. Invalid UTF-8 is
a fatal type error, not silent loss.
* `fs.walk` recursively lists regular files under a directory, relative to the
project root. It requires `fs_read`, skips symlinks, has a depth limit of 64
and a 4096-entry limit. The list is sorted by name at each level.
* All three are project-mode only, use the same path/grant checks as the other
`fs.*` operations, and record one `resource.request` effect event.
* No new builtin or host trait method is added; the operation strings extend
the existing ABI.
## Consequences
* `std:fs/file` exposes `追加文本`, `读取片段` and `遍历` in `.mq.md`.
* Binary offset reads, append handles and filesystem watch remain open; a
watch API needs a separate event model and ADR.
* The operations allocate at most the existing 8 MiB read limit; `walk` is
metadata-only and returns names, not contents.
Loaded 100 of 548 files, more files were not shown because too many files have changed in this diff. Show more