# Marqdo Project Package Format v3 Status: **lock_version 3 package-instance graph as implemented by S1 (ADR-0016).** Path and pinned Git sources, transitive closure locking, fail-closed locked resolution, strict `sync`, and content-profile-1 file-set hashing/materialization are implemented. Registries and SemVer solving remain out of scope; see §8. Implementation: `crates/marqdo-package/src/lib.rs` (manifest/lock types and validation), `crates/cli/src/commands.rs` (`init`, `add`, `remove`, `lock`, `sync`). ## 1. Project root The project root is the nearest ancestor directory of the entry document that contains `marqdo.yaml` (`marqdo_package::discover_root`). If no manifest exists anywhere above the entry, the entry file's parent directory is used as the root (a document can be checked/run without a project). `ModuleId`s are package-instance + package-relative paths (`spec/references.md` §2.1), so the project root selects the entry package instance; physical paths do not form module identity. `add`/`remove`/`lock`/`sync` discover the root from the **current working directory**, not from a document path. There is no `--project` flag in this build. ## 2. `marqdo.yaml` ```yaml project: name: example version: 0.1.0 marqdo: ">=0.1,<1" # optional; informational in this build dependencies: geometry: path: ./packages/geometry # exactly one of path / git+rev quantum: git: https://github.com/example/quantum-mq rev: 0123456789abcdef0123456789abcdef01234567 plugins: solver: version: "^1.0" # requirement only; not installed or authorized capabilities: # requests only; not grants fs: read: ["./data/**"] write: ["./output/**"] net: hosts: [] process: false ``` ### 2.1 Fields | Field | Type | Required | Semantics | |---|---|---|---| | `project.name` | string | yes | non-empty package name | | `project.version` | string | yes | non-empty version string (not semver-validated in the manifest) | | `project.marqdo` | string | no | required Marqdo runtime range; stored, not enforced by this build | | `dependencies..path` | path string | one of path/git | project-relative path to a dependency directory or `.mq.md` file | | `dependencies..git` | URL string | one of path/git | git source (must be pinned with `rev`) | | `dependencies..rev` | string | with `git` | exact revision; a bare `git` without `rev` is rejected | | `plugins..version` | string | per entry | non-empty plugin version requirement; declarative only | | `capabilities.fs.read` / `.write` | list of strings | no | requested project-relative globs | | `capabilities.net.hosts` | list of strings | no | requested hosts | | `capabilities.process` | bool | no | requested process execution | Unknown fields are ignored (the types do not `deny_unknown_fields`). Duplicate YAML keys follow `serde_yaml` behavior (last one wins); there is no duplicate-key diagnostic. ### 2.2 Validation (`Manifest::validate`) `marqdo.yaml` is loaded with `Manifest::from_yaml`/`Manifest::load`, which validates: * `project.name` and `project.version` are non-empty after trimming; * for every dependency, exactly one source is present: * both `path` and `git` → error "declares both path and git"; * neither → error "must declare a path or a pinned git source"; * `git` without `rev` → error requiring an exact revision; * an absolute `path` → error "path must be project-relative"; * for every plugin requirement, `version` is non-empty. Note: a *git* dependency passes manifest validation when pinned, but the dependency is not resolvable or lockable in this build (§5). ### 2.3 `capabilities` are requests The manifest is the project's **request** for permissions. It is never read when constructing the host capability set; only CLI flags/grant APIs create grants (`spec/security.md` §2). `net` has no implementing host op, and `process`/foreign execution still needs `--allow-foreign`. ## 3. `marqdo.lock` `lock_version: 3` records the complete package-instance closure reachable from the root manifest. Packages are sorted by `instance_id`: ```yaml lock_version: 3 content_profile_version: 1 root: package: root:quantum@0.1.0 manifest_digest: 4f5c... packages: - instance_id: dep:math@1.0.0#9e2... name: math version: 1.0.0 source: kind: path path: deps/math content_digest: 9e2... manifest_digest: 7a1... dependencies: helper: dep:helper@2.0.0#1b3... - instance_id: dep:helper@2.0.0#1b3... name: helper version: 2.0.0 source: kind: git url: https://example.invalid/helper.git rev: 0123456789abcdef0123456789abcdef01234567 content_digest: 1b3... manifest_digest: 3c9... dependencies: {} root_dependencies: math: dep:math@1.0.0#9e2... ``` | Field | Semantics | |---|---| | `lock_version` | writers emit `3`; readers accept only `3` and tell v1/v2 users to run `marqdo lock` | | `content_profile_version` | file-set/digest profile; only `1` is accepted | | `root.package` | logical root instance `root:@` | | `root.manifest_digest` | lowercase hex SHA-256 of the canonical `marqdo.yaml` | | `packages[].instance_id` | `dep:@#` | | `packages[].source` | `{kind: path, path}` (project-relative) or `{kind: git, url, rev}` (full commit hash) | | `packages[].content_digest` | 64-character lowercase hex SHA-256 of the package file set | | `packages[].manifest_digest` | optional lowercase hex digest of the dependency's own manifest | | `packages[].dependencies` | this instance's alias → target instance edges | | `root_dependencies` | root manifest alias → target instance edges | The graph is the unit of reproducibility: a transitive dependency that can be imported is also locked. Edges are keyed by `(parent instance, alias)`, so the same alias in two packages may resolve to different instances. ### 3.1 Serialization and validation `lock` writes atomically (temporary file, `sync_all`, rename); a failed resolution leaves the previous lock untouched. `LockFile::to_yaml` sorts packages by `instance_id`. The format contains no timestamps, absolute paths or machine-specific values, so repeated `lock` runs over unchanged inputs are byte-identical. `LockFile::from_yaml` validates before returning: * `lock_version == 3`, `content_profile_version == 1`; * unique ids, each consistent with `dep:@#`; * every digest is 64 lowercase hex characters; * every source is normalized (path relative to the project root without `..`, Git revision a full lowercase 40/64-character hash); * every edge target exists and every package is reachable from `root_dependencies`; empty aliases and self-edges are errors. v1/v2 locks are rejected, not silently upgraded: a flat lock cannot represent the closure and treating it as complete would re-open fail-open loading. ### 3.2 Tree hash For a path or Git dependency, `lock` hashes the resolved package tree with `content_profile_version: 1` (`crates/marqdo-package/src/fileset.rs`, ADR-0016): 1. walk the dependency directory with `walkdir` (`sort_by_file_name`, `follow_links(false)`), propagating every walk error; 2. exclude any path component `.git` or `.marqdo`; include `target/`; 3. accept regular files only. Symlinks and special files are hard errors, not skipped or followed; paths must be UTF-8, `/`-separated, relative and duplicate-free; 4. sort files by root-relative path (UTF-8 bytes); enforce budgets of 10 000 files, 64 MiB per file, 512 MiB total and depth 64; 5. hash with domain separation and length prefixes: `b"marqdo.package.file-set\0"`, `profile_version: u32le`, `entry_count: u64le`, then for each file `rel_len: u64le`, `rel_bytes`, `content_len: u64le`, `content_bytes`; 6. hex-encode the digest. Materialization uses the same selection and reads each file once, hashing the exact bytes it writes; the published environment is accepted only when the final digest equals the lock. This is a deterministic content fingerprint, not a Merkle tree and not a commitment to file modes or empty directories. Modes are normalized on materialization (`0644` files, `0755` directories on Unix). ### 3.3 Lock generation rules `marqdo lock` (run in a project): 1. discovers the project root from the cwd; 2. loads and validates `marqdo.yaml`; 3. traverses the full dependency closure in sorted manifest order: * a `path` source must resolve inside the project root; its module root is hashed and recorded as a project-relative path; * a `git` source requires a full lowercase commit hash. `lock` clones into `.marqdo/cache/git/-`, checks out the revision, verifies `git rev-parse HEAD` equals it, strips `.git`, hashes the tree, and writes a sibling `.meta.json` marker (`url`, `rev`, profile, digest). Install hooks and submodules are not run; * package cycles, duplicate ids with conflicting content, depth >128 and more than 4096 instances are errors; 4. records every package instance with its alias→instance edges, plus the root manifest digest and root edges; 5. validates the graph and atomically writes `marqdo.lock` (exit 0, `locked N package(s)`). What it does **not** do: resolve SemVer ranges, download a registry, verify publisher signatures, install packages or run dependency scripts. A dependency's own `marqdo.yaml` supplies its dependency edges. ## 4. `sync` `marqdo sync [--locked]` (run in a project): 1. discovers the root; loads and validates the existing `marqdo.lock` (missing or v1/v2 lock → exit 1 with a relock instruction); 2. verifies the lock belongs to the current root manifest (`root.package` and `root.manifest_digest`); drift fails closed; 3. materializes **every** package in the locked closure into a project-private `.marqdo/env/pkg-/` tree using the profile-1 file set and the lock's content digest; 4. writes `.marqdo/env/env.json` schema 2 with the content profile, canonical lock digest, root edges and one record per package; publishes via staging + rename, restoring the previous environment on failure; 5. never rewrites the lock. Plain `sync` may fetch a pinned Git revision named by the lock; `--locked` forbids fetching. Both modes verify contents and fail with "lockfile is out of date or dependency contents changed; run `marqdo lock`" on drift. `run`, `call`, `check` and `inspect` never fetch and never write the lock. With declared dependencies they require a lock. They resolve each `(parent instance, alias)` edge in the lock; missing entries, digest/source drift and root-manifest drift are errors. A materialized env or Git cache directory is trusted only after its file-set digest and cache marker verify. ## 5. `init` `marqdo init [dir]`: * refuses to overwrite an existing `marqdo.yaml` (exit 1); * creates the directory if needed; * derives the project name from the directory's final path component (falling back to `example`); * writes `marqdo.yaml` from `Manifest::template`: ```yaml project: name: version: 0.1.0 marqdo: '>=0.1,<1' ``` (empty dependencies/plugins/capabilities are omitted); * writes `index.mq.md` (the target syntax; the template in a pre-RFC binary still emits the old spelling and is part of the pending migration): ```markdown ## @exec main `问候 = "你好,Marqdo"` `打印(问候)` `return 问候` ``` * writes `.gitignore` containing `.marqdo/`; * prints the next-step hint `marqdo check index.mq.md && marqdo run index.mq.md`. `init` does not create `marqdo.lock`, `.marqdo/`, a plugin directory or a git repository. ## 6. `add` / `remove` Both commands edit **only** `marqdo.yaml` (and, for `remove`, prune a matching stale lock entry). They do not fetch, install, resolve or authorize anything. * `marqdo add --path ` or `marqdo add --git --rev `: validate the dependency shape (same rules as §2.2), refuse an existing name (exit 1), insert it, write the manifest, and tell the user to run `marqdo lock`. A pinned Git dependency is lockable/resolvable in R5. * `marqdo remove `: remove the dependency from the manifest (exit 1 if absent), and drop a same-named entry from an existing lock file. It does not re-hash or re-lock the rest of the project. This differs from the original task brief that listed `add`/`remove` as "not supported yet": the CLI does implement them as manifest editors. They are intentionally shallow and should not be confused with a dependency resolver. ## 7. JSON output `check`, `inspect`, `run`, `call`, `plugin inspect`, `add`, `remove`, `lock` and `sync` accept `--json` and print a single JSON object on stdout. Relevant package shapes: * `lock`: `{"command":"lock","success":true,"packages":[...]}` * `sync`: `{"command":"sync","success":true,"locked":,"packages":[...]}` * `add`: `{"command":"add","success":true,"dependency":"","manifest":""}` * `remove`: `{"command":"remove","success":true,"dependency":""}` Errors in these commands are printed as plain `error: ...` on stderr even when `--json` is passed (the JSON success envelope is not emitted; this is an inconsistency to fix). ## 8. Not supported in this build (explicit) | Capability | Status | |---|---| | Git dependency resolution (pinned rev, clone/fetch/checkout) | Implemented for exact revisions; no floating branches, submodules or dependency scripts. | | Registry / package publishing / SemVer resolution | Not implemented. | | Transitive dependency graph and conflict resolution | The lock records the full transitive closure with per-instance edges; two versions of one package can coexist. A general SemVer conflict solver is not implemented. | | `tree`, `update`, `vendor`, `package` commands | Not implemented. | | `--offline`, `--locked` on `lock` | Not implemented; missing cache fails instead of fetching during check/run. | | Environment materialization (`.marqdo/env`, `cache`) | Implemented: `sync` materializes the full locked closure into `.marqdo/env` atomically; `lock`/`sync` populate a verified Git cache. Kernels/sessions are not materialized. | | Dependency signatures / trusted publishers / supply-chain scanning | Not implemented. | | Multiple/nested project boundary diagnostics | Not implemented; nearest `marqdo.yaml` wins. | | Lock file migration between `lock_version`s | Only v3 is read; v1/v2 are rejected with a relock instruction. `lock` regenerates the graph. | | Plugin installation / plugin dependency management | Not implemented; `plugins:` in the manifest is declarative only. | The lock/package format is deliberately small and deterministic so these can be added later without changing the identity model. Changing `lock_version`, `source` syntax or the tree-hash algorithm requires an ADR and a migration plan.