Commit Graph
39 Commits
Author SHA1 Message Date
Marqdo Agent efbb8007e1 feat(compress): add bounded gzip product codec 2026-11-21 10:48:27 +08:00
Marqdo Agent ff3f19a2ea feat(net): add DNS, UDP, TCP listener and TLS endpoints 2026-11-21 10:43:08 +08:00
Marqdo Agent ee0c6cc80a feat(random): add explicitly granted OS entropy API 2026-11-21 10:25:11 +08:00
Marqdo Agent 6338fa10e9 feat(fs): add append, read_at and recursive walk 2026-11-21 10:20:07 +08:00
Marqdo Agent 3ef18fb496 feat(process): add explicitly granted process API 2026-11-21 08:46:02 +08:00
Marqdo Agent 9922bfae98 feat(stdlib): add explicitly granted env API 2026-11-21 08:40:54 +08:00
Marqdo Agent ae69391bc0 feat(stdlib): add datetime and project-mode clock APIs 2026-11-21 08:36:40 +08:00
Marqdo Agent fbe36bcc43 feat(resource): add grant-checked fs resource operations 2026-11-21 08:20:31 +08:00
cflmy 23e68cba5f docs(resource): record stdio stream endpoints and stdlib APIs 2026-11-21 00:58:34 +08:00
cflmy 328f1a8db3 docs: defer stdlib extraction; add local credential and Woodpecker setup
ci/woodpecker/push/woodpecker Pipeline was canceled
CI / fmt + clippy + test (Linux x86_64) (push) Canceled after 0s
CI / schema instance validation + consumers (push) Canceled after 0s
CI / runtime contract (macos-latest) (push) Canceled after 0s
CI / runtime contract (windows-latest) (push) Canceled after 0s
CI / release candidate artifact (macos-latest) (push) Canceled after 0s
CI / release candidate artifact (ubuntu-latest) (push) Canceled after 0s
- Add ADR-0039: keep stdlib in-repo while architecture is small; extraction is
  a future trigger-driven option, not a pending gate. Update ADR-0028,
  STATUS, architecture index, package/P2/completion evidence.
- Add .woodpecker.yml (fmt/clippy/test + release build) for ci.cflmy.cn.
- Add docs/development-credentials.md and ignore .secrets/ plus .env.local;
  real credentials stay in the ignored local file and are never committed.

Gate: fmt, clippy -D warnings, 440 tests passed.
2026-11-20 23:40:38 +08:00
cflmy 757c5ed15e feat(provider): add C ABI provider worker binding v1
- Add the separate marqdo-provider-worker binary: loads one trusted c-abi-1
  shared library and serves the ADR-0035 framed-JSON resource protocol, with
  fail-closed startup and no in-process fallback (ADR-0038).
- Reuse StdHost::register_worker_resource_provider; language still sees only
  resource.* and project-mode/name grants.
- Compile a C test provider and run open/read/write/close through the helper;
  cover missing-library fail-closed. Trusted process boundary, not a sandbox;
  existing plugin-Worker ABI binding remains open.
- Add ADR/spec/status/README/changelog/evidence; workspace member and tests.

Gate: fmt, clippy -D warnings, 440 tests passed.
2026-11-20 23:30:22 +08:00
cflmy 974102dfc3 feat(host): add provider package manifest v1
- Add a marqdo.provider schema v1 loader (schema/version/ABI/platform library)
  and StdHost::register_provider_package, reusing the C ABI adapter and shared
  kind/name checks (ADR-0037).
- Add a compiled-provider + provider.json installation test; language use still
  requires project mode and allow_resource_provider.
- Update ADR/spec/status/README/changelog and evidence; no registry, signature
  or CLI install is claimed.

Gate: fmt, clippy -D warnings, 437 tests passed.
2026-11-20 23:28:30 +08:00
cflmy b22a89e7b4 feat(host): add trusted C ABI ResourceProvider v1
- Add the frozen marqdo_resource_provider.h ABI (version + open/read/write/
  close/last_error) and a libloading-based host adapter with version/symbol
  checks and callsite-attached host errors (ADR-0036).
- Add StdHost::register_c_resource_provider using the same kind/name validation
  and project-mode name grant as the in-process/worker providers.
- Compile a C shared-library test provider; cover round-trip and fail-closed
  missing-library behavior. Trusted native code, not a sandbox; plugin-Worker
  binding remains open.
- Update ADRs/spec/status/README/changelog and evidence.

Gate: fmt, clippy -D warnings, 435 tests passed.
2026-11-20 23:26:25 +08:00
cflmy 01f6fe46a7 feat(host): add worker resource-provider transport v1
- Add a host-managed helper process speaking framed JSON (4-byte BE length)
  with ping/open/read/write/close/shutdown, max 1 MiB frames and structured
  error mapping (ADR-0035).
- Add StdHost::register_worker_resource_provider using the same protocol,
  kind-conflict and name-grant checks as the in-process provider; pure/strict
  refuse provider kinds.
- Tests spawn a Python helper and cover round-trip, grant/mode denial and
  missing-program failure; no parser/Value/runtime/HostOp change.
- Update ADR/spec/status/README/changelog and evidence; C ABI binding remains
  open.

Gate: fmt, clippy -D warnings, 433 tests passed.
2026-11-20 23:25:09 +08:00
cflmy b241d1d978 feat(stdlib): add deterministic offline stdlib packaging
- tools/pack-stdlib.py builds a byte-deterministic
  marqdo-stdlib-0.1.0.tar.gz (fixed metadata, gzip mtime 0) plus a schema-tagged
  descriptor with tree digest, archive SHA-256 and file list.
- Verified repeated builds are identical and the extracted tree digest matches
  stdlib.json; archive is for the future external repo, not the CLI loader.
- Update ADR-0028, status/changelog/tests README and evidence.

Gate: fmt, clippy -D warnings, 430 tests passed.
2026-11-20 23:17:09 +08:00
cflmy b020904199 test(p4): prove row-store/columnar table equivalence
- Add semantic fixture 72: a .mq.md columnar List/Map table store, converted,
  sorted and filtered; compare headers/cells with the T0 row Table via a
  cell-wise helper (Table == is not structural in this language version).
- Teach the direct-runtime semantic harness to resolve entry std: imports.
- Update ADR-0032, P4 status, docs/changelog/evidence; semantic goldens 72.

Gate: fmt, clippy -D warnings, 430 tests passed.
2026-11-20 23:14:38 +08:00
cflmy e981168d92 feat(stdlib): add HTTP framing over TCP and close P5 C10 locally
- Add stdlib/net/http.mq.md: status/header/chunked response parsing and request
  construction in .mq.md; no host HTTP/TLS implementation.
- Add stdlib/tests/http.mq.md and an end-to-end loopback test that sends a GET
  over resource.open("tcp"), reads bytes, parses the response and returns
  status/content-type/body.
- Update ADR-0034, status/spec/README/changelog and evidence; refresh the
  bundle digest.

Gate: fmt, clippy -D warnings, 430 tests passed.
2026-11-20 23:13:06 +08:00
cflmy 69a28ed1e1 feat(stdlib): add table GFM rendering and an HTTP/1.1 parser fixture
- stdlib/data/table.mq.md adds 连接片段/连接行/转义单元格/转GFM; logical Table
  renders as a GFM pipe table with pipe/newline escaping (CM-A08 P4 gate).
- Add semantic fixture 71: HTTP/1.1 status/header/chunked response parsing and
  request construction in .mq.md; no host HTTP/TLS implementation (C10).
- Update table CLI test, golden count 71, specs/status/README/changelog and
  evidence.

Gate: fmt, clippy -D warnings, 426 tests passed; bundle digest refreshed.
2026-11-20 23:10:19 +08:00
cflmy 00bdcad1ff feat(host): add IP-literal blocking TCP resource endpoint (P5)
- resource.open("tcp", ip:port, ["read","write"]) with exact IP grants,
  project mode only, 5s connect/read/write timeouts; no DNS/TLS/UDP/server
  sockets (ADR-0034).
- Add TCP stream storage/dispatch in the resource table; CLI --allow-net IP;
  pure/strict preflight rejects the grant and the language endpoint.
- Host loopback echo/grant/DNS/strict tests and a .mq.md CLI echo test.
- Update specs, status, architecture, README, changelog and evidence.

Gate: fmt, clippy -D warnings, 426 tests passed.
2026-11-20 23:05:06 +08:00
cflmy 1d87f98539 fix(strict): bind the bundled stdlib and verify strict resources
- Strict child now gets the verified stdlib tree read-only at
  /run/marqdo-stdlib with MARQDO_STDLIB_DIR set, so Compat-C 打印 works under
  strict without exposing a host path (ADR-0021 update).
- Add a strict-lane negative: resource.open(file) is refused with
  permission.denied; hello/factorial/multi-module and --json still pass.
- StrictHostCommand gains stdlib_dir; CLI discovers the bundle and sandbox
  binds it; sandbox tests constructors updated.

Evidence: G-C lane PASS on the disposable Ubuntu 26.04 VM (7/7 adversarial
matrix, positive CLI, strict language-resource refusal, forged marker), plus
fmt/clippy/423 local tests.
2026-11-20 22:57:49 +08:00
cflmy 1be4662049 feat(host): add in-process ResourceProvider v1 (CM-A09 slice)
- Add ResourceProvider v1 trait with name/protocol/kinds/open/read/write/close;
  host still issues the opaque language Resource handle, provider tokens stay
  private.
- Add StdHost::register_resource_provider with protocol/version, reserved-name
  and duplicate-kind checks; add Capabilities.allow_resource_provider grants.
- Provider kinds are project-mode only; pure/strict refuse them; in-process
  providers are trusted and not a sandbox (ADR-0033).
- Reuse resource.* and Result shapes; no parser/Value/runtime/HostOp changes.
- Add mock read/write provider and rejection tests, docs/spec/status/evidence.

Gate: fmt, clippy -D warnings, 423 tests passed.
2026-11-20 22:38:16 +08:00
cflmy a8e62a5ba9 feat(stdlib): add table logical-value algorithms (CM-A08)
- Add stdlib/data/table.mq.md with select/filter/sort/append/set-cell/join/
  group as value-returning .mq.md functions over table.columns/row/cell.
- Keep Value::Table logical semantics, GFM AST laziness and T0 physical
  storage unchanged; no new table builtin or HostOp (ADR-0032).
- Add stdlib/tests/table.mq.md and a CLI test asserting shapes/order and
  input immutability; refresh the bundle digest.
- Update architecture/status/spec/README/changelog and evidence.

Gate: fmt, clippy -D warnings, 420 tests passed.
2026-11-20 22:28:52 +08:00
cflmy 49910ece43 refactor(print): remove HostOp::Print and legacy formatting (Compat-C)
- Delete HostOp::Print and StdHost::op_print; the print builtin only routes to
  bundled io.写值 when Program.print_override is set, otherwise returns a
  structured kernel.unavailable error (ADR-0031).
- Keep the 打印/print builtin name for arity/static validation; Value::display
  remains only for CLI result rendering, not print execution.
- Teach the direct-runtime semantic harness to mirror loader stdlib injection
  and import edges; replace runtime tests that used print with non-print
  effects.
- Update spec/security/status/changelog/README, add ADR-0031 and evidence.

Gate: fmt, clippy -D warnings, 419 tests passed.
2026-11-20 22:22:51 +08:00
cflmy e26971f489 feat(cli): route print through the bundled stdlib (Compat-B)
- Add Program.print_override and lazy stdlib injection: only programs that
  call 打印/print load std:io and route to io.写值 (ADR-0030).
- Record resource.open/read/write/close as trace effect events so the stdlib
  print path is auditable; HostOp::Print no longer runs on that path.
- Permit only stdio/stdout (existing console) in strict; file/network
  resource opens remain permission.denied.
- Exclude std: modules from the human-readable checked-module count; update
  the Python analysis consumer to accept std: instance ids.
- Add print-routing and strict-console tests, ADR-0030 and evidence.

Gate: fmt, clippy -D warnings, 418 tests passed.
2026-11-20 22:10:59 +08:00
cflmy e119727c5d feat(stdlib): format all value types and expose handle identity members
- Add read-only FunctionRef.module/name and PluginHandle.plugin/export
  members; PluginHandle.id stays private (ADR-0029).
- Extend stdlib/format.mq.md to Formula/Code/FunctionRef/PluginHandle/
  Memory/Resource/Unit, with UTF-8 byte length for Code.
- Fix two-digit ordered-list body indentation so arms 10+ stay inside their
  branch (they were previously top-level statements).
- Extend stdlib demo to Formula/Code/FunctionRef and assert all three
  io.写值/打印 pairs match byte-for-byte.
- Add runtime member observation test, ADR-0029, spec/status/changelog/evidence.

Gate: fmt, clippy -D warnings, 416 tests passed.
2026-11-20 21:57:04 +08:00
cflmy 1377af1601 feat(stdlib): ship a verified bundled stdlib with std: imports
- Move the Compat-A print prototype into stdlib/ as the official offline
  bundle with stdlib.json (version 0.1.0 + sha256 content digest).
- Add std:<relative> import resolution in the loader with bundle path
  confinement, extension inference and fail-closed digest verification.
- Record the verified version/digest/instance_id in Analysis v2
  project.stdlib; stdlib modules get a std:name@version#digest instance id.
- Add CLI tests: std:io execution, digest-tamper rejection, missing manifest,
  Analysis identity, no foreign/plugin calls, and frozen binary hash equality.
- Update ADR-0028, specs, status, changelog and evidence.

Gate: fmt, clippy -D warnings, 415 tests passed.
2026-11-20 21:43:14 +08:00
cflmy 92bfbf2568 feat(rfc-cm): baseline P0-P3 compute machine, stdlib and resource slices
Implements the first vertical slices of RFC-CM-0001/0002 on top of a113d89:

- P1: opaque Memory regions (64 KiB pages, budgets, grow-only), byte
  read/write, Memory-index place assignment, wire/host-boundary rejection.
- CM-A04/A05: value-semantics List/Map/Text/Table primitives and the
  recoverable Result map convention.
- CM-A06: opaque Resource handles, resource.open/read/write/close, stdout
  write, granted project-file read and buffered write, strict fail-closed.
- CM-A07 Compat-A: explicit .mq.md format/utf8/io chain matching builtin
  print for the covered value types; builtin remains default.
- P2 fixtures: allocators, list/tree/graph/hash, integer codecs, UTF-8
  encode/decode, CSV/JSON subsets, table filter/sort/join, bytecode VM.
- ADRs 0022-0027, spec updates, evidence reports and 68 semantic goldens.

Gate: cargo fmt --check, clippy -D warnings, 407 tests passed.
2026-11-20 21:16:54 +08:00
cflmy b6fc18233d feat(sandbox): implement probe-gated Linux strict isolation (ADR-0021)
Strict mode now re-executes the CLI inside a Bubblewrap supervisor with
user/mount/PID/net/UTS/IPC namespaces, no-new-privs, all capabilities dropped,
a read-only /project, private /proc and /dev, rlimits, output capture and
wall-clock process-group cleanup. The child verifies PID 1, NoNewPrivs, empty
CapEff, an unshared uid map and loopback-only networking before project code.

- PlatformCapabilities::detect runs a real bwrap probe; no probe, no strict
- strict L1 supports pure Marqdo + Math; Foreign/ABI/worker/kernel/grants/trace
  are rejected with exit 2 before sandboxing
- forged MARQDO_STRICT_ACTIVE is rejected; missing bwrap fails closed
- add adversarial helper, 7-test escape/resource matrix, seeded security lane
  script and self-hosted security workflow
- executed G-C lane as uid 1000 in a disposable Ubuntu 26.04 VM: 7/7 PASS
  plus positive CLI and refusal matrix; evidence in docs/evidence
- ADR-0021, spec/security.md, STATUS/COMPATIBILITY/README/CHANGELOG updated;
  ADR-0019 remains for non-Linux/fail-closed rules
2026-11-20 15:01:49 +08:00
cflmy faa6c95cfe fix(foreign): bind pinned environments to the executed provider
A verified environment_dir was only prepended to PATH, so built-in providers
still ran the host interpreter and external entrypoints could live outside the
verified tree.

- external entrypoints must resolve inside environment_dir
- built-in providers cannot be bound to an environment (fail closed)
- bound children get exactly the environment bin on PATH, no host fallback
- declared interpreter_version is verified against the resolved program
- built-in descriptors now also run environment validation
- ADR-0020 + spec/foreign.md document the binding and limitations
- positive/negative tests for binding, outside-entrypoint, builtin binding and
  version drift
2026-11-20 13:56:08 +08:00
cflmy e966b7661f docs(security): gate strict-mode availability behind a security test lane (H3-B)
Add ADR-0019 freezing the strict availability contract: strict remains
unavailable with an actionable reason on every platform until runtime probes
and filesystem/network/process/resource escape tests pass in a dedicated
security lane. Add a CLI test asserting inspect --platform reports unavailable
and worker processes are not presented as isolation. No G-C claim is made.
2026-11-20 13:13:42 +08:00
cflmy f69c563afd fix(plugin): bound worker capability calls per invocation (H3-A)
- enforce at most 64 capability requests and 64 KiB per serialized request
  in both the worker handler and the host bridge
- return structured capability.budget_exceeded / request_too_large errors and
  kill the worker on a host-side violation
- add a real C loop fixture and CLI test proving a runaway plugin is stopped
  without producing file contents
- update ADR-0018, plugin-abi and STATUS
2026-11-20 13:11:24 +08:00
cflmy 0f01a69048 feat(plugin): bridge worker capability requests to the host broker (H3-A)
- add request-scoped capability_request/capability_reply/capability_error
  frames to the worker transport (ADR-0018); no C ABI change
- worker routes its capability handler through stdin/stdout, the host serves
  requests inside the existing invoke deadline and binds identity from the
  spawned worker, never from frame text
- both native and worker plugins now execute a granted fs.read through
  ProjectFs with audit; unsupported capabilities return capability.unavailable
- add a real C capability plugin and CLI test for granted read and denial
- document the bridge and that strict OS isolation (H3-B) is still refused
2026-11-20 13:08:56 +08:00
cflmy 509bdb1bf6 fix(runtime): resolve runtime errors against their logical module (B-08)
RuntimeError and CallFrame now carry a logical ModuleId and an origin
(source/callsite). Errors are attributed at function/body boundaries so the
innermost callee module wins; argument-binding errors and host/plugin/foreign
boundaries are attributed to the caller callsite without fabricating a foreign
stack.

The CLI resolves (module_id, span) through LoadedProject::locate instead of
indexing every span against the entry file. Unknown modules and out-of-range
spans keep the module id and byte span but emit null line/file. Human output and
frames now point at the correct file.

Record the decision in ADR-0017 and supersede the ADR-0006 limitation. Add an
end-to-end imported-module division-by-zero test.
2026-11-20 12:57:44 +08:00
cflmy cd90f38ce1 fix(package): unify package file-set hashing and materialization (B-05)
Add content_profile_version 1 with one selection shared by hash_tree and
materialization: exclude .git/.marqdo, include target, reject symlinks and
special files, propagate walk errors, enforce file/size/depth budgets, and use
a domain-separated length-prefixed encoding (fixes the old NUL-delimiter
ambiguity). Materialization hashes exactly the bytes it writes and verifies the
expected digest before publishing.

Record the decision in ADR-0016 and update the package-format hash contract.
2026-11-20 12:51:54 +08:00
cflmy b5a28a9825 feat(m0.6-r4): public Analysis v1 and Trace v1
- add check --emit with a deterministic, read-only semantic projection:
  package-aware modules, signatures, normalized bodies, refs, calls,
  diagnostics and source digests
- remove stale analysis artifacts on compile errors; atomic writes
- add run/call --trace-file with interpreter call/branch/loop/return events
  and host effect events under one sequence; redact values and record error
  status
- add Analysis/Trace v1 specs, ADR-0015 and an independent Python consumer
- add contract tests for determinism, stale artifacts, trace ordering/status
  and secret redaction
2026-11-20 11:46:37 +08:00
cflmy 25a1faa6ef feat(m0.6-r3): generic external CLI Tool Protocol v1
- add tool install/list/remove and unknown-command dispatch without hardcoded
  tool names or clap variants
- validate and copy local tool packages atomically, reject entrypoint escapes
  and symlinks, record and re-verify SHA-256
- forward OsString args, stdio, cwd and exit status without a shell; never
  execute PATH programs implicitly
- add black-box tests for arbitrary names, Unicode args, built-in conflicts,
  traversal, version mismatch, tampering and PATH hijacking
- add ADR-0013 and mark R3 complete
2026-11-20 11:44:20 +08:00
cflmy 7797d78d86 feat(m0.6-r2): host-owned plugin handle authority
- add PluginHandleTable with owner, run id and generation scoping
- reject forged, cross-owner, cross-run and post-unload handles
- reject any plugin_handle returned by plugin or foreign-kernel output,
  including nested values
- add ADR-0014 and update value-wire/security specs
2026-11-20 11:43:21 +08:00
cflmy 66476fbbfb docs(m0.6): integrate plan and freeze R0 baseline decisions
- move the revision package into docs/ and spec/, preserving all content
- record the M0.6 plan/audit cross-references and Proposed status
- correct stale STATUS claims about CI, console capability and check --refs
- add Accepted ADRs 0008-0012 for package identity, callable resolution,
  plugin export routing, capability broker and kernel providers
- record the passing R0 baseline gate (fmt, clippy, workspace tests)
2026-11-20 11:34:45 +08:00
cflmy 0116160399 docs: formal specs, ADRs, STATUS matrix, contributor guide and honest README
Complete the documentation set for the M0.5 toolchain, grounded in the code:

- spec/references.md: import/reference/call edges, identities, loader rules,
  anchors, `ref` resolution/ambiguity, `check --refs` validation.
- spec/value-wire.md: exact wire_version 1 typed encoding, integer-as-decimal
  string rule, special floats, limits and exact float round-trip.
- spec/security.md: modes (strict refuses without OS isolation), capabilities,
  path rules, budgets, tracing, foreign process caveats, plugin trust.
- spec/package-format.md: marqdo.yaml/lock formats, validation, lock/sync,
  unsupported registry/git/environment work.
- spec/plugin-abi.md: ABI v1 layout/ownership/negotiation, manifest/artifact
  rules, wire envelopes, loader lifecycle, trust boundary, SDK/C demo, and the
  implemented CLI plugin-invocation flow with its limitations.
- spec/math.md and spec/foreign.md: the implemented math subset and the
  foreign request/response/timeout/capability contract.
- docs/adr/0001..0007: accepted decisions for the span foundation, inert
  executables outside function bodies, ref ambiguity (S-07), strict refusal,
  AST normalization, CLI contract and BigInt/typed wire (S-08).
- docs/contributor-guide.md: repo map, workflow, fixture/golden updates,
  quality gate, ADR/commit policy.
- docs/STATUS.md: P0-P7 matrix with evidence, open items and the blueprint §18
  Alpha checklist (conclusion: not yet Alpha-complete), including known
  divergences rather than overclaims.
- README.md rewritten from "design-only starter" to the working implementation
  with quickstart, implemented phases, deliberate limits and doc index.
- AGENTS.md: updated pointers/workflow only; architectural rules unchanged.
- Consistency fixes in spec/mapping.md/evaluation.md (inert emph outside
  functions, unsupported math.simplify/differentiate).
2026-11-20 10:56:26 +08:00