Files
marqdo-cli/spec/package-format.md
T
cflmy 5877d9aa5e feat(m0.6-r1): package-aware module identity and resolver
- replace path-only ModuleId with {instance, relative_path} and canonical
  <instance>::<path> encoding
- add PackageResolver/ResolvedPackage in marqdo-package for path sources;
  hash package trees deterministically for content-addressed instance ids
- move import resolution out of the CLI loader; reject package escapes
- emit lock_version 2 with instance ids and migrate v1 lockfiles
- update references/package-format specs and the function-ref golden
- add package resolver tests for stable identity across path moves
2026-11-20 11:37:16 +08:00

263 lines
11 KiB
Markdown

# Marqdo Project Package Format v2
Status: **lock_version 2 as implemented by M0.6 R1.** Path sources and
package-instance identity are implemented; pinned Git sources, registries,
transitive dependency solving and environment materialization are added in R5
or later; see §8.
Implementation: `crates/marqdo-package/src/lib.rs` (manifest/lock types and
validation), `crates/cli/src/commands.rs` (`init`, `add`, `remove`, `lock`,
`sync`).
## 1. Project root
The project root is the nearest ancestor directory of the entry document that
contains `marqdo.yaml` (`marqdo_package::discover_root`). If no manifest exists
anywhere above the entry, the entry file's parent directory is used as the
root (a document can be checked/run without a project).
`ModuleId`s are package-instance + package-relative paths (`spec/references.md`
§2.1), so the project root selects the entry package instance; physical paths do
not form module identity. `add`/`remove`/`lock`/`sync` discover the
root from the **current working directory**, not from a document path. There is
no `--project` flag in this build.
## 2. `marqdo.yaml`
```yaml
project:
name: example
version: 0.1.0
marqdo: ">=0.1,<1" # optional; informational in this build
dependencies:
geometry:
path: ./packages/geometry # exactly one of path / git+rev
quantum:
git: https://github.com/example/quantum-mq
rev: 0123456789abcdef0123456789abcdef01234567
plugins:
solver:
version: "^1.0" # requirement only; not installed or authorized
capabilities: # requests only; not grants
fs:
read: ["./data/**"]
write: ["./output/**"]
net:
hosts: []
process: false
```
### 2.1 Fields
| Field | Type | Required | Semantics |
|---|---|---|---|
| `project.name` | string | yes | non-empty package name |
| `project.version` | string | yes | non-empty version string (not semver-validated in the manifest) |
| `project.marqdo` | string | no | required Marqdo runtime range; stored, not enforced by this build |
| `dependencies.<name>.path` | path string | one of path/git | project-relative path to a dependency directory or `.mq.md` file |
| `dependencies.<name>.git` | URL string | one of path/git | git source (must be pinned with `rev`) |
| `dependencies.<name>.rev` | string | with `git` | exact revision; a bare `git` without `rev` is rejected |
| `plugins.<name>.version` | string | per entry | non-empty plugin version requirement; declarative only |
| `capabilities.fs.read` / `.write` | list of strings | no | requested project-relative globs |
| `capabilities.net.hosts` | list of strings | no | requested hosts |
| `capabilities.process` | bool | no | requested process execution |
Unknown fields are ignored (the types do not `deny_unknown_fields`). Duplicate
YAML keys follow `serde_yaml` behavior (last one wins); there is no duplicate-key
diagnostic.
### 2.2 Validation (`Manifest::validate`)
`marqdo.yaml` is loaded with `Manifest::from_yaml`/`Manifest::load`, which
validates:
* `project.name` and `project.version` are non-empty after trimming;
* for every dependency, exactly one source is present:
* both `path` and `git` → error "declares both path and git";
* neither → error "must declare a path or a pinned git source";
* `git` without `rev` → error requiring an exact revision;
* an absolute `path` → error "path must be project-relative";
* for every plugin requirement, `version` is non-empty.
Note: a *git* dependency passes manifest validation when pinned, but the
dependency is not resolvable or lockable in this build (§5).
### 2.3 `capabilities` are requests
The manifest is the project's **request** for permissions. It is never read
when constructing the host capability set; only CLI flags/grant APIs create
grants (`spec/security.md` §2). `net` has no implementing host op, and
`process`/foreign execution still needs `--allow-foreign`.
## 3. `marqdo.lock`
`lock_version: 2`, packages sorted by name:
```yaml
lock_version: 2
packages:
- name: geo
instance_id: dep:geo@0.1.0#101a6f6db4cc
source: path:./packages/geo
sha256: 101a6f6db4cc48d8c138b4716028da9bc8934b5f02b2235e020fbcabba15b757
```
| Field | Semantics |
|---|---|
| `lock_version` | writers emit `2`; readers accept `1` and `2` and migrate v1 entries by deriving an instance id |
| `packages[].name` | dependency name from the manifest |
| `packages[].instance_id` | logical `dep:<name>@<version>#<digest-prefix>` identity (ADR-0008) |
| `packages[].source` | `path:<manifest path as written>`; a Git source is `git:<url>@<rev>` |
| `packages[].sha256` | optional; lowercase hex SHA-256 of the dependency tree |
### 3.1 Deterministic serialization
`LockFile::to_yaml` sorts packages by name before serializing. The format
contains no timestamps, no absolute paths and no machine-specific values, so
repeated `marqdo lock` runs over unchanged dependencies are byte-identical.
`LockFile::from_yaml` also sorts packages on load, so equality comparison is
order-insensitive.
### 3.2 Tree hash
For a path dependency, `lock` hashes the dependency tree
(`crates/cli/src/commands.rs::hash_tree`):
1. walk the dependency directory with `walkdir` (`sort_by_file_name`), keeping
regular files only (symlinks are not followed and are skipped);
2. sort files by their root-relative path (normalized to `/`);
3. feed into SHA-256, for each file in order:
`relative_path bytes`, a `0x00` byte, `file bytes`, a `0x00` byte;
4. hex-encode the digest.
This is a deterministic content fingerprint, not a Merkle tree and not a
cryptographic commitment to file modes, symlinks or empty directories. It
detects content changes and additions/removals of regular files.
### 3.3 Lock generation rules
`marqdo lock` (run in a project):
1. discovers the project root from the cwd;
2. loads and validates `marqdo.yaml`;
3. for each manifest dependency, in map (sorted) order:
* a git source → error "git dependencies are not supported by this build
(path dependencies only)"; no lock is written;
* no path source → error;
* the dependency path must exist; the tree is hashed;
4. writes `marqdo.lock` (exit 0, `locked N package(s)`).
What it does **not** do: resolve semver ranges, traverse a dependency's own
`marqdo.yaml`/transitive dependencies, clone git repositories, download
registries, verify package signatures, or install anything.
## 4. `sync`
`marqdo sync [--locked]` (run in a project):
1. discovers the root; loads the manifest and the existing `marqdo.lock`
(missing lock → error "run `marqdo lock` first", exit 1);
2. recomputes the expected lock file exactly as `lock` would (including tree
hashes);
3. with `--locked`, compares the expected lock with the on-disk lock; a
difference (manifest change or dependency content change) prints
"lockfile is out of date or dependency contents changed; run `marqdo lock`"
and exits 1;
4. without `--locked`, prints `synchronized N package(s)` and exits 0.
Current behavior: `sync` **never rewrites** the lock file and never materializes
a `.marqdo/env` directory. It is a verification command only. There is no
`--offline` flag and no `lock --locked` flag (the blueprint example
`marqdo lock --locked` is not implemented; use `sync --locked`).
## 5. `init`
`marqdo init [dir]`:
* refuses to overwrite an existing `marqdo.yaml` (exit 1);
* creates the directory if needed;
* derives the project name from the directory's final path component (falling
back to `example`);
* writes `marqdo.yaml` from `Manifest::template`:
```yaml
project:
name: <name>
version: 0.1.0
marqdo: '>=0.1,<1'
```
(empty dependencies/plugins/capabilities are omitted);
* writes `index.mq.md`:
```markdown
## main
**问候 = "你好,Marqdo"**
> 打印(问候)
*问候*
```
* writes `.gitignore` containing `.marqdo/`;
* prints the next-step hint `marqdo check index.mq.md && marqdo run index.mq.md`.
`init` does not create `marqdo.lock`, `.marqdo/`, a plugin directory or a git
repository.
## 6. `add` / `remove`
Both commands edit **only** `marqdo.yaml` (and, for `remove`, prune a matching
stale lock entry). They do not fetch, install, resolve or authorize anything.
* `marqdo add <name> --path <dir>` or
`marqdo add <name> --git <url> --rev <sha>`:
validate the dependency shape (same rules as §2.2), refuse an existing name
(exit 1), insert it, write the manifest, and tell the user to run
`marqdo lock`. Adding a git dependency is allowed at the manifest level even
though `lock` will refuse it later.
* `marqdo remove <name>`: remove the dependency from the manifest (exit 1 if
absent), and drop a same-named entry from an existing lock file. It does not
re-hash or re-lock the rest of the project.
This differs from the original task brief that listed `add`/`remove` as
"not supported yet": the CLI does implement them as manifest editors. They
are intentionally shallow and should not be confused with a dependency
resolver.
## 7. JSON output
`check`, `inspect`, `run`, `call`, `plugin inspect`, `add`, `remove`, `lock`
and `sync` accept `--json` and print a single JSON object on stdout. Relevant
package shapes:
* `lock`: `{"command":"lock","success":true,"packages":[...]}`
* `sync`: `{"command":"sync","success":true,"locked":<bool>,"packages":[...]}`
* `add`: `{"command":"add","success":true,"dependency":"<name>","manifest":"<path>"}`
* `remove`: `{"command":"remove","success":true,"dependency":"<name>"}`
Errors in these commands are printed as plain `error: ...` on stderr even when
`--json` is passed (the JSON success envelope is not emitted; this is an
inconsistency to fix).
## 8. Not supported in this build (explicit)
| Capability | Status |
|---|---|
| Git dependency resolution (clone/fetch/checkout) | Not implemented. Manifest accepts `git`+`rev`; `lock` refuses it. |
| Registry / package publishing / SemVer resolution | Not implemented. |
| Transitive dependency graph and conflict resolution | Not implemented; only direct manifest dependencies are hashed. |
| `tree`, `update`, `vendor`, `package` commands | Not implemented. |
| `--offline`, `--locked` on `lock` | Not implemented. |
| Environment materialization (`.marqdo/env`, `cache`, `kernels`, `sessions`) | Not implemented; `init` only writes `.marqdo/` to `.gitignore`. |
| Dependency signatures / trusted publishers / supply-chain scanning | Not implemented. |
| Multiple/nested project boundary diagnostics | Not implemented; nearest `marqdo.yaml` wins. |
| Lock file migration between `lock_version`s | v1 is read and migrated; writers emit v2. |
| Plugin installation / plugin dependency management | Not implemented; `plugins:` in the manifest is declarative only. |
The lock/package format is deliberately small and deterministic so these can
be added later without changing the identity model. Changing `lock_version`,
`source` syntax or the tree-hash algorithm requires an ADR and a migration
plan.