- Add ResourceProvider v1 trait with name/protocol/kinds/open/read/write/close; host still issues the opaque language Resource handle, provider tokens stay private. - Add StdHost::register_resource_provider with protocol/version, reserved-name and duplicate-kind checks; add Capabilities.allow_resource_provider grants. - Provider kinds are project-mode only; pure/strict refuse them; in-process providers are trusted and not a sandbox (ADR-0033). - Reuse resource.* and Result shapes; no parser/Value/runtime/HostOp changes. - Add mock read/write provider and rejection tests, docs/spec/status/evidence. Gate: fmt, clippy -D warnings, 423 tests passed.
13 KiB
Changelog
All notable changes to the Marqdo CLI workspace. The workspace version is
currently 0.1.0; no release tag has been cut. Dates use ISO 8601.
Unreleased (stabilization candidate)
Added
-
.gitattributesbyte-identity policy: canonical LF for text files and one explicit CRLF byte fixture for Markdown line-ending semantics. -
Byte-level AST golden diagnostics: first differing byte, byte windows and an explicit "only line endings differ" classification.
-
Cross-platform Native ABI test fixtures generated for the host target, plus
no_matching_artifact, artifact-tamper and ABI-major-mismatch negative tests. -
Host-specific example plugin builds under
examples/plugin-*/build/; the tracked manifests are never rewritten by tests or CI. -
Foreign pinned-environment binding (ADR-0020): an external provider's
entrypointmust resolve insideenvironment_dir, built-in providers cannot be bound to one, the child gets only the environment executable directory onPATH, andinterpreter_versionis verified before execution. -
Linux strict isolation (ADR-0021): a Bubblewrap supervisor with user/mount/ PID/net/UTS/IPC namespaces, no-new-privs, capability dropping, a read-only
/project, rlimits, output/wall-clock supervision and child self-verification. Probe-gated; L1 supports pure Marqdo + Math only. -
Adversarial G-C lane (
tests/security/escape_helper.c,crates/marqdo-sandbox/tests/security_linux.rs,tests/security/run-lane.sh) covering filesystem,/proc, network, process, privilege and resource escapes. 7/7 PASS in a disposable Ubuntu 26.04 VM. -
Release documentation (
COMPATIBILITY.md,docs/releasing.md) and the Linux security-gate evidence report (docs/security-gate.md). -
check_never_spawns_a_foreign_kernel: proves staticcheckdoes not spawn a declared Foreign provider, whilerunon the same document does. -
Power/control-flow corpus: 10 new AST goldens, 8 semantic goldens, 4 compile-negative fixtures and 2 runtime-negative fixtures covering right-associative
**, unary-minus precedence, parentheses, float negative exponents, Strong+CodeSpan payloads, branch/loop conditions, inert prose and code fences, and source spans inside one/two-backtick code spans. -
Independent Tool Protocol v1 consumers
sample-view,sample-knowledgeandsample-proof(analysis/trace readers) with install, dispatch and remove tests; none is a hard-coded CLI command. -
Locked path-dependency content-tamper test: a changed dependency is rejected without rewriting
marqdo.lockor the materialized.marqdo/env. -
G-C lane honours
CARGO_TARGET_DIR, so a run from a cleangit archivewith an external target directory uses the binary it just built. -
Addressable
Memory(RFC-CM-0001 P1, ADR-0022): an opaque run-scoped region value with 64 KiB zero-filled pages, grow-only capacity, bounded byte read/write,memory.new/size/grow, andm[offset] = byte. Copies share the region identity; different regions never alias. Default budgets are 64 MiB committed per run and 16 MiB per region;--max-memoryand--max-region-memoryoverride the budgets onrun/call. -
Memory-index place assignment in the expression/IR model (
AssignTarget::Index/PlaceTarget::Index) with the frozen evaluation order base, index, RHS and no change toList/Map/Tablevalue semantics. -
Stable
runtime.memoryerror category for invalid/stale handles and page counts;runtime.indexfor offsets,runtime.typefor operand/byte-range errors,resource.exhaustedfor budgets. -
Analysis v2 projects an index assignment target as an additive
{"kind":"index",...}object while local targets keep the v1 string. -
examples/memory-buffer.mq.md, semantic goldens 45–47 and runtime negatives 13–18 cover sharing, isolation, growth, zero-fill, offsets, byte range, non-Memory index assignment, declared-max growth and negative page counts. -
Partial RFC-CM P2 closure evidence (
.mq.mdonly, no Rust algorithm additions): fixtures 48–50 implement a big-endian u32 codec, a bump allocator, node allocation and a linked-list traversal/sum, and prove that a failed allocation leaves the allocator header unchanged. This is not the full P2 gate: a separate stdlib package and a frozen-CLI closure lane are still missing. -
Memoryhandles are never transferable:to_wireemits a bare rejected tag,from_wirefails, and every host/plugin/foreign call rejects arguments that contain a handle. -
Minimal structure observation/construction (RFC-CM CM-A04, ADR-0023):
list.append/concat,map.keys/has/get/set/remove,text.codepoints/from_codepoint, andtable.columns/row/cell. All are pure, value-returning builtins; inputs are never mutated and no new syntax is added. Semantic fixtures 51–53 and runtime negatives 19–21 cover value semantics, Unicode scalars, table projection, wrong types and bounds. -
.mq.mdtable filter (过滤) built ontable.columns/row/cell,list.appendand表, with a semantic golden showing the input table is unchanged (fixture 54). Notable.filter/HostOpis added. -
Recoverable
Resultconvention (RFC-CM CM-A05, ADR-0024): an ordinary Map{ok:true,value}/{ok:false,code,message[,details]}, constructed by.mq.mdfunctions, with expected failures recoverable and language invariant violations still fatal. NoResulttype, exception ortry/catchsyntax. -
.mq.mdfirst-fit free-list allocator (fixture 55): 8-byte block headers, free-block reuse, block splitting, address-adjacent coalescing (tested in a second region whose bump space is exhausted, so only a merged block can satisfy the request), non-overlapping reused payloads andalloc.no_spacereturned as aResultwithout aborting the run. No allocator Rust builtin orHostOpwas added. -
.mq.mdP2 closures beyond the first slice: binary tree insert/in-order traversal (56), CSV row splitter (57), open-addressing hash table (58), UTF-8 encoder/validating decoder (59–60), graph adjacency list (61), recursive JSON parser for arrays/objects/strings (62), stack bytecode VM (63), 16/32/64-bit + LEB128 codecs (64), table sort (65), table equality join (67) and quoted/escaped CSV (68). No Rust algorithm, container or parser was added for these; algorithms remain.mq.mdcode. JSON float parsing and a compact formatter are included. -
Resource/stream first slice (RFC-CM CM-A06, ADR-0025): opaque host-issued
Resourcehandles,resource.open/read/write/close, bytes transferred throughMemory, stdout write in pure/project under console/output budgets, granted project-file chunked read in project, EOF as success, closed-handle Result errors, forged-handle fatal errors, wire/host-boundary rejection, and strict refusing every resource open. Semantic fixture 66 and host/CLI tests. -
Buffered project-file write streams (ADR-0026):
file+["write"]requires anfs_writegrant, buffers in the host, flushes only onclose, never flushes on drop, has an 8 MiB ceiling, and is refused in pure/strict. A.mq.mdchunked copy loop reproduces a file without a Rustcopy_file. -
Compat-C print completion (ADR-0031): remove
HostOp::PrintandStdHost::op_print; the print builtin only routes to bundledio.写值or returnskernel.unavailable; print traces contain only resource effects. -
Compat-B
打印/printrouting (ADR-0030): aProgram.print_overridesends print calls to bundledio.写值when the program uses them; stdlib modules are injected lazily, resource operations appear as trace effects and the legacyHostOp::Printpath does not run. Strict permits only stdio/stdout (existing console), not file/network resources. No Compat-C removal yet. -
Full-value stdlib formatting (ADR-0029): FunctionRef
module/nameand PluginHandleplugin/exportread-only members (host id private);stdlib/format.mq.mdnow reproduces builtin打印/reprfor all 15 value types, including Formula/Code and UTF-8 byte length for Code. -
CM-A10 bundled stdlib (ADR-0028): a fixed
stdlib/tree withformat/text/io, explicitstd:<path>imports, SHA-256 digest verification before module loading, astd:marqdo-stdlib@<version>#<digest>package instance id,project.stdlibin Analysis v2, and test coverage forstd:resolution, tamper rejection and frozen-binary hash equality. -
CM-A09 in-process ResourceProvider v1 (ADR-0033):
ResourceProvidertrait,StdHost::register_resource_provider, per-kind registration with protocol version/conflict checks andCapabilities.allow_resource_provider; provider streams reuseresource.*and are project-mode only. A mock read/write provider test runs without parser/runtime/HostOpchanges. C/Worker provider transport remains open. -
CM-A08 table logical/stdlib slice (ADR-0032):
stdlib/data/table.mq.mdadds select/filter/sort/append/set-cell/join/group as value-returning.mq.mdfunctions;Value::Tablesemantics and GFM AST laziness unchanged; no physical storage migration or new table builtin. -
CM-A07 Compat-A prototype (ADR-0027):
stdlib/providesformat.显示, UTF-8 encoding andio.写值(resource.write(stdout)), and its output is comparison-tested byte-for-byte against builtin打印for Unit/None/Bool/Int/Float/Text/List/Map/Table. Builtin打印remains the default; no alias routing, bundled stdlib orHostOp::Printremoval; the other value types lack public observation.
Changed
- Assignment targets are no longer restricted to bare identifiers: a single
index place is accepted at statement level. Member assignment stays a parse
error; chained assignment stays a parse error.
spec/expression.ebnf,spec/evaluation.md,spec/value-wire.md,spec/analysis-v2.mdandspec/security.mddocument the new semantics. docs/STABLE_RELEASE_IMPLEMENTATION.mdis the single next-phase implementation plan. Older plan documents are historical input, not current status.- AST conformance tests fail early when a golden contains CR bytes instead of reporting an opaque mismatch.
- Worker plugin fixtures use the platform dynamic-library name and compiler flags.
- List-item condition paragraphs now accept only inert description; an
executable
Strong/Emphthere is a hardmarkdown.invalid_constructerror instead of being silently swallowed. Condition expression spans are mapped from the literal bytes inside the code-span backticks. - Linux strict
inspect --platformremains a snapshot, not a promise. A run that loses namespace/resources between probe and launch must fail closed.
Fixed
- Windows CI AST golden failure caused by CRLF checkout of canonical JSON
goldens (reproduced with a fresh
core.autocrlf=trueclone). - macOS CI plugin failure caused by assuming a Linux
.soand Linux-only manifest target. - macOS/Windows
E0433: platform detection now uses#[cfg]functions instead ofcfg!aroundlinux::probe(), so the Linux module is not referenced on other targets. - Linux strict probe→launch race: the CLI contract and the sandbox seam now require a non-zero, diagnosed, fail-closed refusal when the real supervisor launch fails after a successful probe.
- Parallel security tests shared process-id scratch paths and deleted each other's project; every case now has a unique path and the helper is compiled once.
- Condition parse diagnostics pointed at the backtick instead of the missing operand; the payload map now excludes delimiters.
Security
- Linux
strictis implemented and probe-gated. It isAvailableonly when a real Bubblewrap smoke test succeeds; otherwise it staysUnavailableand refuses. There is no silent downgrade toproject. - Strict L1 covers pure Marqdo and native Math; Foreign, native/worker plugins, kernels, external grants and trace files are rejected before the sandbox starts. Strict L2 inheritance is future work.
- Capability brokers and worker processes remain cooperative authorization/crash isolation, not an OS sandbox.
- The G-C lane was re-run at
8af7262from a cleangit archivein a disposable Ubuntu 26.04 VM as ordinary userdev: 7/7 adversarial tests, positive CLI contract and forged-marker rejection PASS. This is still a local VM result, not a self-hosted GitHub security-lane URL.
Baseline 0.1.0 (before this stabilization slice)
- Markdown profile
marqdo.md.v1with 150 AST goldens. - Native Rust interpreter, analysis and execution conformance corpus.
- lock v3 package-instance graph with full transitive closure, verified Git
cache and atomic
.marqdo/envmaterialization (ADR-0016). - Analysis v2 and Trace v2 with independent Python consumers and JSON Schemas.
- Tool Protocol v1 with SemVer/protocol negotiation and whole-bundle digests.
- Plugin ABI v1 native and worker transports with a brokered
fs.readcapability path (ADR-0018). - Pinned Foreign provider/environment digests and one-shot Python/Node kernels.