- replace path-only ModuleId with {instance, relative_path} and canonical
<instance>::<path> encoding
- add PackageResolver/ResolvedPackage in marqdo-package for path sources;
hash package trees deterministically for content-addressed instance ids
- move import resolution out of the CLI loader; reject package escapes
- emit lock_version 2 with instance ids and migrate v1 lockfiles
- update references/package-format specs and the function-ref golden
- add package resolver tests for stable identity across path moves
11 KiB
Marqdo Project Package Format v2
Status: lock_version 2 as implemented by M0.6 R1. Path sources and package-instance identity are implemented; pinned Git sources, registries, transitive dependency solving and environment materialization are added in R5 or later; see §8.
Implementation: crates/marqdo-package/src/lib.rs (manifest/lock types and
validation), crates/cli/src/commands.rs (init, add, remove, lock,
sync).
1. Project root
The project root is the nearest ancestor directory of the entry document that
contains marqdo.yaml (marqdo_package::discover_root). If no manifest exists
anywhere above the entry, the entry file's parent directory is used as the
root (a document can be checked/run without a project).
ModuleIds are package-instance + package-relative paths (spec/references.md
§2.1), so the project root selects the entry package instance; physical paths do
not form module identity. add/remove/lock/sync discover the
root from the current working directory, not from a document path. There is
no --project flag in this build.
2. marqdo.yaml
project:
name: example
version: 0.1.0
marqdo: ">=0.1,<1" # optional; informational in this build
dependencies:
geometry:
path: ./packages/geometry # exactly one of path / git+rev
quantum:
git: https://github.com/example/quantum-mq
rev: 0123456789abcdef0123456789abcdef01234567
plugins:
solver:
version: "^1.0" # requirement only; not installed or authorized
capabilities: # requests only; not grants
fs:
read: ["./data/**"]
write: ["./output/**"]
net:
hosts: []
process: false
2.1 Fields
| Field | Type | Required | Semantics |
|---|---|---|---|
project.name |
string | yes | non-empty package name |
project.version |
string | yes | non-empty version string (not semver-validated in the manifest) |
project.marqdo |
string | no | required Marqdo runtime range; stored, not enforced by this build |
dependencies.<name>.path |
path string | one of path/git | project-relative path to a dependency directory or .mq.md file |
dependencies.<name>.git |
URL string | one of path/git | git source (must be pinned with rev) |
dependencies.<name>.rev |
string | with git |
exact revision; a bare git without rev is rejected |
plugins.<name>.version |
string | per entry | non-empty plugin version requirement; declarative only |
capabilities.fs.read / .write |
list of strings | no | requested project-relative globs |
capabilities.net.hosts |
list of strings | no | requested hosts |
capabilities.process |
bool | no | requested process execution |
Unknown fields are ignored (the types do not deny_unknown_fields). Duplicate
YAML keys follow serde_yaml behavior (last one wins); there is no duplicate-key
diagnostic.
2.2 Validation (Manifest::validate)
marqdo.yaml is loaded with Manifest::from_yaml/Manifest::load, which
validates:
project.nameandproject.versionare non-empty after trimming;- for every dependency, exactly one source is present:
- both
pathandgit→ error "declares both path and git"; - neither → error "must declare a path or a pinned git source";
gitwithoutrev→ error requiring an exact revision;- an absolute
path→ error "path must be project-relative";
- both
- for every plugin requirement,
versionis non-empty.
Note: a git dependency passes manifest validation when pinned, but the dependency is not resolvable or lockable in this build (§5).
2.3 capabilities are requests
The manifest is the project's request for permissions. It is never read
when constructing the host capability set; only CLI flags/grant APIs create
grants (spec/security.md §2). net has no implementing host op, and
process/foreign execution still needs --allow-foreign.
3. marqdo.lock
lock_version: 2, packages sorted by name:
lock_version: 2
packages:
- name: geo
instance_id: dep:geo@0.1.0#101a6f6db4cc
source: path:./packages/geo
sha256: 101a6f6db4cc48d8c138b4716028da9bc8934b5f02b2235e020fbcabba15b757
| Field | Semantics |
|---|---|
lock_version |
writers emit 2; readers accept 1 and 2 and migrate v1 entries by deriving an instance id |
packages[].name |
dependency name from the manifest |
packages[].instance_id |
logical dep:<name>@<version>#<digest-prefix> identity (ADR-0008) |
packages[].source |
path:<manifest path as written>; a Git source is git:<url>@<rev> |
packages[].sha256 |
optional; lowercase hex SHA-256 of the dependency tree |
3.1 Deterministic serialization
LockFile::to_yaml sorts packages by name before serializing. The format
contains no timestamps, no absolute paths and no machine-specific values, so
repeated marqdo lock runs over unchanged dependencies are byte-identical.
LockFile::from_yaml also sorts packages on load, so equality comparison is
order-insensitive.
3.2 Tree hash
For a path dependency, lock hashes the dependency tree
(crates/cli/src/commands.rs::hash_tree):
- walk the dependency directory with
walkdir(sort_by_file_name), keeping regular files only (symlinks are not followed and are skipped); - sort files by their root-relative path (normalized to
/); - feed into SHA-256, for each file in order:
relative_path bytes, a0x00byte,file bytes, a0x00byte; - hex-encode the digest.
This is a deterministic content fingerprint, not a Merkle tree and not a cryptographic commitment to file modes, symlinks or empty directories. It detects content changes and additions/removals of regular files.
3.3 Lock generation rules
marqdo lock (run in a project):
- discovers the project root from the cwd;
- loads and validates
marqdo.yaml; - for each manifest dependency, in map (sorted) order:
- a git source → error "git dependencies are not supported by this build (path dependencies only)"; no lock is written;
- no path source → error;
- the dependency path must exist; the tree is hashed;
- writes
marqdo.lock(exit 0,locked N package(s)).
What it does not do: resolve semver ranges, traverse a dependency's own
marqdo.yaml/transitive dependencies, clone git repositories, download
registries, verify package signatures, or install anything.
4. sync
marqdo sync [--locked] (run in a project):
- discovers the root; loads the manifest and the existing
marqdo.lock(missing lock → error "runmarqdo lockfirst", exit 1); - recomputes the expected lock file exactly as
lockwould (including tree hashes); - with
--locked, compares the expected lock with the on-disk lock; a difference (manifest change or dependency content change) prints "lockfile is out of date or dependency contents changed; runmarqdo lock" and exits 1; - without
--locked, printssynchronized N package(s)and exits 0.
Current behavior: sync never rewrites the lock file and never materializes
a .marqdo/env directory. It is a verification command only. There is no
--offline flag and no lock --locked flag (the blueprint example
marqdo lock --locked is not implemented; use sync --locked).
5. init
marqdo init [dir]:
-
refuses to overwrite an existing
marqdo.yaml(exit 1); -
creates the directory if needed;
-
derives the project name from the directory's final path component (falling back to
example); -
writes
marqdo.yamlfromManifest::template:project: name: <name> version: 0.1.0 marqdo: '>=0.1,<1'(empty dependencies/plugins/capabilities are omitted);
-
writes
index.mq.md:## main **问候 = "你好,Marqdo"** > 打印(问候) *问候* -
writes
.gitignorecontaining.marqdo/; -
prints the next-step hint
marqdo check index.mq.md && marqdo run index.mq.md.
init does not create marqdo.lock, .marqdo/, a plugin directory or a git
repository.
6. add / remove
Both commands edit only marqdo.yaml (and, for remove, prune a matching
stale lock entry). They do not fetch, install, resolve or authorize anything.
marqdo add <name> --path <dir>ormarqdo add <name> --git <url> --rev <sha>: validate the dependency shape (same rules as §2.2), refuse an existing name (exit 1), insert it, write the manifest, and tell the user to runmarqdo lock. Adding a git dependency is allowed at the manifest level even thoughlockwill refuse it later.marqdo remove <name>: remove the dependency from the manifest (exit 1 if absent), and drop a same-named entry from an existing lock file. It does not re-hash or re-lock the rest of the project.
This differs from the original task brief that listed add/remove as
"not supported yet": the CLI does implement them as manifest editors. They
are intentionally shallow and should not be confused with a dependency
resolver.
7. JSON output
check, inspect, run, call, plugin inspect, add, remove, lock
and sync accept --json and print a single JSON object on stdout. Relevant
package shapes:
lock:{"command":"lock","success":true,"packages":[...]}sync:{"command":"sync","success":true,"locked":<bool>,"packages":[...]}add:{"command":"add","success":true,"dependency":"<name>","manifest":"<path>"}remove:{"command":"remove","success":true,"dependency":"<name>"}
Errors in these commands are printed as plain error: ... on stderr even when
--json is passed (the JSON success envelope is not emitted; this is an
inconsistency to fix).
8. Not supported in this build (explicit)
| Capability | Status |
|---|---|
| Git dependency resolution (clone/fetch/checkout) | Not implemented. Manifest accepts git+rev; lock refuses it. |
| Registry / package publishing / SemVer resolution | Not implemented. |
| Transitive dependency graph and conflict resolution | Not implemented; only direct manifest dependencies are hashed. |
tree, update, vendor, package commands |
Not implemented. |
--offline, --locked on lock |
Not implemented. |
Environment materialization (.marqdo/env, cache, kernels, sessions) |
Not implemented; init only writes .marqdo/ to .gitignore. |
| Dependency signatures / trusted publishers / supply-chain scanning | Not implemented. |
| Multiple/nested project boundary diagnostics | Not implemented; nearest marqdo.yaml wins. |
Lock file migration between lock_versions |
v1 is read and migrated; writers emit v2. |
| Plugin installation / plugin dependency management | Not implemented; plugins: in the manifest is declarative only. |
The lock/package format is deliberately small and deterministic so these can
be added later without changing the identity model. Changing lock_version,
source syntax or the tree-hash algorithm requires an ADR and a migration
plan.