Compare commits

...
13 Commits
Author SHA1 Message Date
cflmyandCursor 27991a7822 docs(release): fold unbound-table fix into v1.0.3 notes
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-17 10:31:41 +08:00
cflmyandCursor d1e2f0ced0 fix(parse): allow unbound GFM tables as discarded values
Bare tables in a function body evaluate as collections without requiring
a name bind; restores welcome-page install tables.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-17 10:29:13 +08:00
cflmyandCursor e848306187 fix(public): drop bare GFM tables from welcome # main
Tables are collections in Marqdo; bare rows aborted pages CI.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-17 10:26:12 +08:00
cflmyandCursor 2ee9d7e66d release: v1.0.3 — PPA, ext CDN, synced CLI+ext packs
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-17 10:20:37 +08:00
cflmyandCursor e31585842d fix(ppa): bundle Rust toolchain for noble Cargo.lock v4 builds
Document install channels (PPA/GitHub/CDN); release skill supports
full/cli/ext modes with R2+GitHub ext uploads and manual PPA commands.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-17 10:18:50 +08:00
cflmyandCursor f068b4ab75 fix(packaging): skip cp when orig tarball already in build-area
Same-inode copy aborted ppa-build under set -e before dput.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-17 10:05:23 +08:00
cflmyandCursor 26e29409a9 docs(changelog): mention ppa-ship one-shot script
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-17 10:01:33 +08:00
cflmyandCursor 4923129d10 feat(packaging): add ppa-ship.sh one-shot build and dput
Installs apt deps with sudo, then builds/signs/uploads as the real user.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-17 10:01:22 +08:00
cflmyandCursor 8ab55e2193 docs(changelog): note Ubuntu PPA packaging scaffold
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-17 09:58:12 +08:00
cflmyandCursor 9bc7ef3b5a feat(packaging): scaffold Ubuntu PPA debian/ for marqdo CLI
Add control/rules/changelog and ppa-build-source.sh (cargo vendor + debuild -S).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-17 09:57:11 +08:00
cflmyandCursor 9431ad835c docs: link ext-cdn in doc/README
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-17 09:31:06 +08:00
cflmyandCursor 2984d7d4f8 feat(ext): prefer Cloudflare R2 CDN and allow separate ext pack versions
Download order is CDN (ext.marqdo.com) → GitHub → proxy; pack SemVer from
ext/VERSION / CDN latest; add upload-ext-r2.py and release-skill notes.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-17 09:28:28 +08:00
cflmyandCursor 6324036e41 fix(ext): mirror fallback for Release downloads and default ~/.marqdo/ext
Try proxy.cflmy.top when github.com fails; honor MARQDO_EXT_DOWNLOAD_BASE;
resolve ext imports from the ext-add install root without exporting MARQDO_EXT.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-17 08:57:24 +08:00
35 changed files with 1284 additions and 171 deletions

No files matched your search

+136 -111
View File
@@ -1,16 +1,14 @@
---
name: marqdo-release
description: >-
Run a full Marqdo product release: detect latest version and ask the user for
the next tag, update CHANGELOG/README/skills/public docs, bump Cargo versions,
sync VS Code extension on branch vscode-extension when needed, build or
trigger install assets (Windows CLI + **required Linux native/CLI zips**,
stdlib, ext, public zip, VSIX), push via origin or
https://proxy.cflmy.top/github.com/cflmy/marqdo.git, publish GitHub Release
with detailed notes, and recover from network failures via that proxy or HK
SSH jump (scripts/push-via-hk-jump.py). Use when the user asks to release,
cut a version, publish vX.Y.Z, ship a GitHub release, 发版, 发布新版本, or tag
marqdo.
Run a Marqdo product release: detect latest version and ask the user for the
next tag (or confirm mode), update CHANGELOG/README/skills/public docs, bump
Cargo and/or ext/VERSION, sync VS Code extension when needed, publish GitHub
Release assets, upload extension packs to Cloudflare R2 (ext.marqdo.com) and
GitHub, ship Ubuntu CLI via PPA (scripts/ppa-ship.sh), and print manual
commands when sudo/GPG/network steps need the user. Use when the user asks to
release, cut a version, publish vX.Y.Z, ext-only pack, PPA, 发版, 发布新版本,
or tag marqdo.
---
# Marqdo release
@@ -19,166 +17,193 @@ Canonical release playbook for **cflmy/marqdo**. Read this skill **before** tagg
## Hard rules
1. **Never invent the next version.** Detect current versions → **stop and ask the user** for the next SemVer (`X.Y.Z`). Do not proceed until they confirm.
1. **Never invent the next version.** Detect current versions → **stop and ask the user** for the next SemVer (`X.Y.Z`) and **release mode** if unclear. Do not proceed until they confirm (unless they already stated both in the same message).
2. **Never force-push** `main` / tags. **Never** `--no-verify` unless the user explicitly orders it.
3. **Never commit `vscode-extension/` on `main`** (gitignored). Extension source lives only on branch **`vscode-extension`**. See `doc/design/vscode-extension-commit.md`.
4. **Do not release from a dirty tree** (except intentional release commits you create in this flow).
5. **Tag format** is always `vX.Y.Z` matching root `Cargo.toml` `version = "X.Y.Z"`.
6. Prefer **tag push → GitHub Actions** (`.github/workflows/release.yml`) for **both** Windows and **Linux** install assets. Local packaging is fallback / verification.
7. After network errors: apply [reference.md § Proxy](reference.md). **First try** `https://proxy.cflmy.top/github.com/cflmy/marqdo.git` (this clone’s usual `origin`). Retry; do not silently skip uploads. If the reverse-proxy still drops long pushes, use **HK jump** (`hk.cflmy.de`) via `scripts/push-via-hk-jump.py` — see reference § **Proxy** / **HK SSH jump**.
8. **Do not ship a GitHub Release without the Linux extension/native zip.** Job `linux` must attach `marqdo-VER-native-x86_64-unknown-linux-gnu.zip` (and the Linux CLI bundle). Missing Linux packages = release incomplete.
5. **Tag format:** CLI/full → `vX.Y.Z` matching root `Cargo.toml`. Ext-only → `ext-vX.Y.Z` matching `ext/VERSION` (Cargo may stay unchanged).
6. Prefer **tag push → GitHub Actions** (`.github/workflows/release.yml`) for Windows/Linux install assets. Local packaging is fallback / verification.
7. After network errors: apply [reference.md § Proxy](reference.md). **First try** `https://proxy.cflmy.top/github.com/cflmy/marqdo.git`. Retry; do not silently skip uploads. If needed: **HK jump** (`scripts/push-via-hk-jump.py`).
8. **Do not ship a full GitHub Release without the Linux extension/native zip** when the mode includes ext packs. Missing Linux packages = incomplete for that mode.
9. **Extension packs always publish to both** Cloudflare R2 (`https://ext.marqdo.com` via `scripts/upload-ext-r2.py`) **and** GitHub (Release assets / `ext-v*` tag). Credentials only in `~/.marqdo/r2.env` / CI — **never** commit. See [ext-cdn.md](../../../doc/design/ext-cdn.md).
10. **CLI Ubuntu package = PPA** (`ppa:cflmy/marqdo`) via `./scripts/ppa-ship.sh` / `ppa-build-source.sh`. GitHub zips remain for Windows and non-apt Linux.
11. **When a step needs the user** (sudo password, GPG passphrase, Launchpad UI, interactive `dput`): **stop, state what failed, and give the exact commands** to run in their terminal. Do not pretend the step succeeded.
## Defaults (after version is confirmed)
## Release modes
Unless the user overrides, apply these automatically — **do not re-ask**:
Ask (or take from user message) which mode:
| Mode | Bumps | Git tag | GitHub assets | R2 CDN | Ubuntu PPA |
|------|-------|---------|---------------|--------|------------|
| **full** (default) | `Cargo.toml` + `ext/VERSION` (same SemVer) | `vX.Y.Z` | CLI + ext + native + VSIX + … | yes | yes |
| **cli** | `Cargo.toml` only | `vX.Y.Z` | CLI (+ optional attach prior ext) | no (unless user asks) | yes |
| **ext** | `ext/VERSION` only | `ext-vX.Y.Z` (or attach to latest CLI release) | ext.zip + native-*.zip | **required** | no |
Defaults after version+mode confirmed — **do not re-ask**:
| Setting | Default |
|---------|---------|
| Sync / bump `vscode-extension` to same `VER` | **yes** |
| Push `main`, create annotated tag, push tag immediately (trigger CI) | **yes** |
| Release notes language | **中英双语** (Chinese first, then English section, or paired ZH/EN under each heading) |
| Skip VSIX | **no** |
| Docs-only release | **no** (full asset pipeline via CI) |
| Sync / bump `vscode-extension` to same `VER` | **yes** for full/cli; **no** for ext-only |
| Push `main`, create annotated tag, push tag (trigger CI) | **yes** for full/cli |
| Ext packs → R2 **and** GitHub | **yes** whenever ext ships |
| Ubuntu PPA upload after CLI tag | **yes** for full/cli |
| Release notes language | **中英双语** |
| Deploy gh-pages after release | **no** (only if user asks) |
Phase 0 asks **only** for the next version (and optional overrides). Example ask:
Phase 0 ask example:
> 当前最新是 **vA.B.C**。请指定下一个版本号(如 `0.3.2`)。其余按默认:同步扩展、立刻打 tag 发布、中英双语说明。
> 当前最新 CLI **vA.B.C**,扩展包 **E.F.G**。请指定下一个版本号与模式(`full` / `cli` / `ext`,如 `1.0.3 full`)。
## Phase 0 — Detect versions (mandatory stop)
Run in repo root (parallel OK):
```bash
python3 .cursor/skills/marqdo-release/scripts/detect_version.py
# or: rg / git tag / gh release list — see reference.md
```
Report briefly, then **ask only for the next SemVer** (defaults above). Wait for confirmation. Normalize to `VER=X.Y.Z` and `TAG=vX.Y.Z`.
Report briefly, then **ask for SemVer + mode** (unless already given). Normalize `VER` / `TAG`.
## Phase 1 — Preflight
Copy and track:
```
Release progress:
- [ ] Phase 0: version confirmed by user (TAG=…)
- [ ] Phase 0: version + mode confirmed
- [ ] Phase 1: preflight green
- [ ] Phase 2: docs + version bumps committed
- [ ] Phase 3: extension branch (if needed)
- [ ] Phase 3: extension branch (full/cli)
- [ ] Phase 4: main pushed; tag created & pushed
- [ ] Phase 5: CI / assets / release notes verified
- [ ] Phase 6: post-release checks
- [ ] Phase 5: CI / GitHub assets / notes
- [ ] Phase 6: R2 upload (if ext ships)
- [ ] Phase 7: Ubuntu PPA (full/cli)
- [ ] Phase 8: post-release checks
```
Checks:
1. `git status` clean or only release WIP you control.
2. `git checkout main && git pull --ff-only origin main`.
3. Quick gate: `cargo test --test gold structure_hello` (and any release-critical tests the user named). Full gold suite if time allows.
4. `CHANGELOG.md` has meaningful **Unreleased** notes; if empty, draft from `git log vLAST..HEAD --oneline` and confirm with user.
5. Confirm no secrets in the commit (`*.env`, credentials).
Checks: clean tree; `main` ff-only; smoke tests; CHANGELOG Unreleased; no secrets.
## Phase 2 — Docs & version bumps (main)
Bump **in lockstep**:
| File | Change |
|------|--------|
| `Cargo.toml` | `version = "VER"` |
| `crates/marqdo-wasm/Cargo.toml` | same `VER` |
| `CHANGELOG.md` | Move `## Unreleased` → `## vVER — YYYY-MM-DD`; leave fresh empty `## Unreleased` |
| `README.md` | 「现状 / 如何使用最新」version strings, checkout tag, release URL |
| `.cursor/skills/marqdo/SKILL.md` | Version-sensitive bullets if any (WASM/web status); keep rules accurate |
| `public/**` | User-facing tutorials if features shipped (welcome, features/*, extensions). Prefer small factual updates over rewrites |
| `doc/roadmap/*.md` | Mark shipped waves done if this release closes them |
| `.github/workflows/release.yml` | Update **Highlights** body template to this release (CI still uses hardcoded blurb—keep in sync) |
Public HTML: regenerate before packaging / pages deploy:
| File | full | cli | ext |
|------|------|-----|-----|
| `Cargo.toml` + `crates/marqdo-wasm/Cargo.toml` | bump | bump | — |
| `ext/VERSION` | bump | — | bump |
| `CHANGELOG.md` | `## vVER` | `## vVER` | `## Ext pack vVER` |
| `README.md` / `public/**` install + download table | update | update | update ext notes |
| `.cursor/skills/marqdo/SKILL.md` | if surface changed | same | rare |
| `.github/workflows/release.yml` Highlights stub | yes | yes | — |
```bash
./scripts/build-public.sh
# outputs under public/ (gitignored HTML); used by release zip / gh-pages
./scripts/build-public.sh # when public/ sources changed
```
Commit on `main` (HEREDOC message), e.g.:
```text
release: vVER — <one-line highlight>
EOF
```
Push `main` (this repo’s `origin` is usually the reverse proxy):
Commit + push `main` (proxy origin). On push failure → reference § Proxy; if still blocked, **print** for the user:
```bash
# origin typically: https://proxy.cflmy.top/github.com/cflmy/marqdo.git
git push origin main
# one-shot if origin still points at github.com:
git push https://proxy.cflmy.top/github.com/cflmy/marqdo.git HEAD:main
# or: python3 scripts/push-via-hk-jump.py
```
If that times out: [reference § HK SSH jump](reference.md).
## Phase 3 — VS Code / Cursor extension (full/cli only)
## Phase 3 — VS Code / Cursor extension
Default: bump `vscode-extension` package.json to `VER`, compile, push branch `vscode-extension`, return to `main`.
**Default: always** bump extension to `VER` on branch `vscode-extension` (skip only if user said so).
1. `git fetch origin vscode-extension`
2. Work on `vscode-extension` branch (not main):
- Bump `vscode-extension/package.json` `version` to `VER`.
- Update `engines` / CLI version hints in extension README if present.
- `npm ci && npm run compile`
3. Commit & `git push origin vscode-extension`.
4. Return to `main`. **Do not** add `vscode-extension/` to main.
## Phase 4 — Tag & trigger CI
## Phase 4 — Tag & trigger CI (full/cli)
```bash
git tag -a "TAG" -m "TAG"
git push origin "TAG"
git tag -a "vVER" -m "vVER"
git push origin "vVER"
```
This runs `.github/workflows/release.yml`:
Ext-only:
- **windows** — CLI exe/zip, stdlib zip, ext zip, source zip, public zip, VSIX, Windows native zip
- **linux** (`needs: windows`) — **required** Linux native plugins zip + Linux CLI bundle (`ext/` + `native/*.so`)
```bash
git tag -a "ext-vVER" -m "ext-vVER"
git push origin "ext-vVER"
```
If tag push fails: [reference.md § Proxy](reference.md) (`proxy.cflmy.top` first), retry once; then HK jump.
(or build zips locally / via workflow and `gh release upload`).
## Phase 5 — Release notes & assets
If tag push needs the user:
1. Wait for Actions success: `gh run list --workflow=Release --limit 3` / open the run URL. **Both** `windows` and `linux` jobs must be green.
2. Ensure GitHub Release for `TAG` exists with assets (workflow uses `softprops/action-gh-release`).
3. Confirm Linux assets exist: `marqdo-VER-native-x86_64-unknown-linux-gnu.zip` and `marqdo-VER-x86_64-unknown-linux-gnu.zip`. If missing, do **not** call the release done — re-run / upload.
4. **Rewrite release body** (workflow body is a stub) with **中英双语** — see [reference.md § Notes template](reference.md). `gh release edit TAG --notes-file …`.
4. Include: Highlights, Breaking, Install (`ext add`, plugins build), Downloads table, WASM (`marqdo wasm build`), links to CHANGELOG/README, extension branch note.
5. Optional Linux local extras (not required if CI green): `marqdo wasm build` smoke; document in notes that wasm is built from source via CLI.
```bash
git push https://proxy.cflmy.top/github.com/cflmy/marqdo.git vVER
```
Local Windows fallback: `scripts/release-full.ps1 -Tag TAG -Upload` (see script header).
## Phase 5 — GitHub Release notes & assets
## Phase 6 — Post-release
1. Wait for Actions (full/cli): both `windows` and `linux` green when ext natives required.
2. Rewrite release body 中英双语 — [reference.md § Notes](reference.md).
3. Confirm Linux `marqdo-VER-native-x86_64-unknown-linux-gnu.zip` when mode ships ext.
1. `gh release view TAG` — confirm assets + notes.
2. `git status` clean; `main` synced.
3. Tell user: release URL `https://github.com/cflmy/marqdo/releases/tag/TAG`.
4. Optional: `./scripts/deploy-public.ps1` / pages workflow if user wants user-site refresh.
5. Open fresh `## Unreleased` already done in Phase 2.
## Phase 6 — Extension CDN (R2) — required when ext ships
```bash
# credentials: ~/.marqdo/r2.env (never commit)
python3 scripts/upload-ext-r2.py --version VER --from-dir dist/
curl -fsS https://ext.marqdo.com/latest/VERSION
```
Also ensure the **same** zips are on GitHub (`vVER` or `ext-vVER`). If R2 credentials missing, **tell the user**:
```bash
# put keys in ~/.marqdo/r2.env then:
python3 scripts/upload-ext-r2.py --version VER --from-dir dist/
```
## Phase 7 — Ubuntu PPA (full/cli)
Linux CLI for apt users. After `Cargo.toml` is at `VER` and `debian/changelog` has a matching `VER-1ppaN` stanza:
```bash
./scripts/ppa-ship.sh noble
```
Packaging notes / noble Cargo.lock v4: [ubuntu-ppa.md](../../../doc/design/ubuntu-ppa.md).
**If the agent cannot sudo / sign / dput**, print exactly:
```bash
cd ~/work/marqdo
./scripts/ppa-ship.sh noble
# or build then upload:
./scripts/ppa-build-source.sh noble
dput ppa:cflmy/marqdo ../build-area/marqdo_*~noble_source.changes
```
Watch: https://launchpad.net/~cflmy/+archive/ubuntu/marqdo/+builds
On Launchpad failure: read the build log, fix `debian/` or bundle script, bump `1ppaN`, re-upload. Tell the user the new `dput` command.
## Phase 8 — Post-release
1. `gh release view TAG` — assets + notes.
2. CDN `https://ext.marqdo.com/vVER/` (if ext shipped).
3. PPA build green (if cli shipped).
4. Tell user URLs: GitHub release, CDN, PPA install one-liner.
5. Optional gh-pages only if asked.
## Extension-only release (mode=ext)
1. User confirms **ext** SemVer (may differ from CLI).
2. Bump `ext/VERSION`; CHANGELOG → `## Ext pack vVER`.
3. Build/package `marqdo-VER-ext.zip` + `marqdo-VER-native-*.zip` (CI or local).
4. Upload to **R2 and GitHub** (required both).
5. Do **not** retag CLI / do **not** PPA unless user also asked for cli.
## Failure matrix (short)
| Failure | Action |
|---------|--------|
| `gh` / git TLS or Clash `7890` refused | Unset bad proxy; `required_permissions: ["all"]`; see [reference § Proxy](reference.md) |
| GitHub blocked | **`origin` via proxy.cflmy.top** (see below), then HK jump |
| Tag exists | Stop; ask user to bump or delete tag (no force on shared tags without explicit order) |
| CI red | Fix on main, move tag only if user explicitly allows delete+re-push tag |
| VSIX missing | Fetch `vscode-extension`, build locally, `gh release upload TAG dist/*.vsix` |
| Linux native/CLI zip missing | Incomplete release — fix `linux` job or `gh release upload`; do not skip |
| `gh` / git TLS or Clash `7890` refused | Unset bad proxy; see [reference § Proxy](reference.md) |
| GitHub blocked | proxy.cflmy.top, then HK jump; **print commands** if agent cannot push |
| Tag exists | Stop; ask user |
| CI red | Fix on main; retag only if user allows |
| R2 upload fails | Print `upload-ext-r2.py` command; do not skip silently |
| PPA / sudo / GPG | Print `./scripts/ppa-ship.sh noble` (or `dput …`); do not skip silently |
| Launchpad `Cargo.lock` v4 / old cargo | Ensure bundled `third_party/rust` via current `ppa-build-source.sh`; bump `1ppaN` |
| Linux native zip missing (full/ext) | Incomplete — fix linux job or upload |
## Do not
- Ship without user-confirmed version.
- Hardcode release highlights only in Actions YAML and forget CHANGELOG.
- Commit `target/`, `dist/`, `*.wasm`, or `public/**/*.html` unless project policy changes.
- Ship without user-confirmed version (and mode when ambiguous).
- Ship ext packs to only one of R2 / GitHub.
- Call a full/cli release done without attempting PPA **or** giving the user the PPA commands.
- Commit `target/`, `dist/`, `*.wasm`, `third_party/rust/`, or `public/**/*.html` unless policy changes.
- Merge extension tree into `main`.
+1 -1
View File
@@ -19,7 +19,7 @@ Canonical design (repo): [`markdown-mapping-v0.3.md`](../../doc/design/markdown-
**Authoring / library craft** (GFM-first, no json glue, how to develop Marqdo): use **[marqdo-dev](../marqdo-dev/SKILL.md)** whenever writing or refactoring `.mq.md`, `lib/*`, or `ext/*`.
**Product release** (tag, CHANGELOG, README, public docs, VSIX, GitHub assets): use [marqdo-release](../marqdo-release/SKILL.md) — detect latest version, **ask the user for the next SemVer**, then follow that playbook. Do not invent a version number.
**Product release** (tag, CHANGELOG, README, public docs, VSIX, GitHub assets, **ext → R2+GitHub**, **CLI → Ubuntu PPA**): use [marqdo-release](../marqdo-release/SKILL.md) — detect versions, **ask for SemVer + mode (`full` / `cli` / `ext`)**, then follow that playbook. Ext-only releases are allowed. If sudo/GPG/PPA steps need the user, print the exact commands.
## Hard rules (do not violate)
+9
View File
@@ -0,0 +1,9 @@
# Ext CDN / R2 (Cloudflare) — copy to ~/.marqdo/r2.env and chmod 600
# Never commit real keys. Public downloads: https://ext.marqdo.com
R2_ACCOUNT_ID=
R2_BUCKET=marqdo
R2_ENDPOINT=https://<accountid>.r2.cloudflarestorage.com
R2_ACCESS_KEY_ID=
R2_SECRET_ACCESS_KEY=
R2_PUBLIC_BASE=https://ext.marqdo.com
+2 -2
View File
@@ -124,8 +124,8 @@ jobs:
## Marqdo ${{ github.ref_name }}
### 亮点 / Highlights
- **v1.0.2**:表驱动页面引言 `compose_intro` / `引言装配`;web-site Demo 与后台门禁加固
- **v1.0.2**: table-bound page intro (`compose_intro`); web-site demo + desk auth polish
- **v1.0.3**:Ubuntu PPA + 扩展 CDN(ext.marqdo.com);CLI/扩展可分发;noble 捆绑 Rust 工具链
- **v1.0.3**: Ubuntu PPA + extension CDN; independent CLI/ext packs; bundled Rust for noble
- **v1.0.1**:修复 Go `libweb` 动态路由 `{param}` 查询代入、详情 Markdown、CSS 数字误报
- **Linux 包必出**:`*-native-x86_64-unknown-linux-gnu.zip` + Linux CLI bundle
- Details: [CHANGELOG](https://github.com/cflmy/marqdo/blob/main/CHANGELOG.md) · [mapping v0.3](https://github.com/cflmy/marqdo/blob/main/doc/design/markdown-mapping-v0.3.md)
+6 -1
View File
@@ -24,7 +24,8 @@
.marqdo/
**/.marqdo/
.marqdo-ext/
.env
/vendor/
/.cargo/config.toml
# Generated HTML beside tracked public/*.mq.md (sources stay in git)
/public/index.html
@@ -77,5 +78,9 @@ plugins/*/tmp/
# Go web plugin c-shared build output (./scripts/build-web-plugin.sh)
/plugins/web/build/
# PPA build: bundled Rust toolchain (packed into .orig by ppa-build-source.sh only)
/third_party/rust/
/third_party/
# ephemeral sqlite / test artifacts
tests/ext/:memory:
+33
View File
@@ -8,6 +8,39 @@
### Changed
## v1.0.3 — 2026-09-17
### Highlights
**分发渠道齐备**:Ubuntu PPA 装 CLI;扩展包走 CDN(`ext.marqdo.com`)与 GitHub;CLI 与扩展可分开发版。本版 CLI 与扩展包 **同步为 1.0.3**。
**语法**:未绑定的 GFM 表格是可丢弃的集合值,不再报 `unrecognized statement`(文档页可直接写安装表)。
```bash
# Ubuntu
sudo add-apt-repository ppa:cflmy/marqdo
sudo apt update && sudo apt install marqdo
marqdo ext add web
# 或源码 / Releases
git checkout v1.0.3 && cargo build --release
```
### Added
- Ubuntu PPA packaging: `debian/` + `scripts/ppa-ship.sh` (一键装工具/打包/dput) + `ppa-build-source.sh` + [ubuntu-ppa.md](doc/design/ubuntu-ppa.md).
- **Extension CDN** (`https://ext.marqdo.com` / Cloudflare R2):`ext add` 下载顺序改为 CDN → GitHub → proxy;独立包版本见 `ext/VERSION` 与 CDN `latest/VERSION`;上传脚本 `scripts/upload-ext-r2.py`(凭据 `~/.marqdo/r2.env`)。设计见 [ext-cdn.md](doc/design/ext-cdn.md)。
- `ext add` 下载:GitHub 失败后自动回退 `https://proxy.cflmy.top/github.com/…`;可用 `MARQDO_EXT_DOWNLOAD_BASE` 指定首选镜像;加长读写超时。
- 用户文档(README / `public/00-欢迎` / `features/05-扩展`)标明安装渠道:Ubuntu PPA、GitHub Releases、源码;扩展走 CDN。
### Fixed
- `ext/` 导入默认搜索 `~/.marqdo/ext`(与 `marqdo ext add` 安装根一致),无需再手动 `export MARQDO_EXT`。
- PPA/noble:捆绑 `third_party/rust`(现代 cargo)以构建 Cargo.lock v4;修复首次 Launchpad 构建失败(distro cargo 1.75)。
- 未绑定的 GFM 表格不再报 `unrecognized statement`:作为集合值求值后丢弃(文档中可写表)。
### Changed
- 允许 **CLI 与扩展包分开发版**:扩展包 SemVer 可与 `Cargo.toml` 不同(`MARQDO_EXT_VERSION` / `ext/VERSION` / CDN latest)。
- 发版 skill:支持 `full` / `cli` / `ext`;扩展必传 R2+GitHub;CLI 走 PPA;需用户操作时打印确切命令。
## v1.0.2 — 2026-09-16
### Highlights
Generated
+2 -2
View File
@@ -572,7 +572,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "marqdo"
version = "1.0.2"
version = "1.0.3"
dependencies = [
"anyhow",
"assert_cmd",
@@ -603,7 +603,7 @@ dependencies = [
[[package]]
name = "marqdo-wasm"
version = "1.0.2"
version = "1.0.3"
dependencies = [
"marqdo",
"serde_json",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "marqdo"
version = "1.0.2"
version = "1.0.3"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo — Markdown markers as an interpreted programming language"
+16 -6
View File
@@ -98,7 +98,7 @@ marqdo catalog public -o .marqdo
---
## 现状(v1.0.2)
## 现状(v1.0.3)
- 映射与解释器:**v0.3 语法宪法**(`**` 代码 / `*` 返回 / 叙述声明);Phase I 树遍历 + 字节码后端;金样例在 `tests/` 与 `tests/markup-v03/`
- **对象**:`#` = 类型/构造,`##`+ = 函数/方法;见 [objects.md](doc/design/objects.md)
@@ -120,12 +120,22 @@ marqdo catalog public -o .marqdo
- **浏览器 Marqdo(WASM)**:`marqdo wasm build` + 官方 bridge 自启(作者零业务 JS;桥内可含列表/路由/storage/ws/文件/Canvas/音频/Observer/拖放);`lib/browser` + GFM;`web.client_embed`;示例 [browser-hello](examples/browser-hello/) · [browser-app](examples/browser-app/) · [browser-media](examples/browser-media/) · [web-client-site](examples/web-client-site/)([ADR 0002](doc/adr/0002-browser-marqdo-wasm.md) · [D](doc/roadmap/browser-wasm-d.md) · [E](doc/roadmap/browser-wasm-e.md) · [F](doc/roadmap/browser-wasm-f.md))
- 选型:[ADR 0001 — Rust](doc/adr/0001-implementation-language.md) · [ADR 0002 — 浏览器 WASM](doc/adr/0002-browser-marqdo-wasm.md)(C0–C5 完结,见 [roadmap/browser-wasm.md](doc/roadmap/browser-wasm.md))· [ADR 0003 — 异步效应](doc/adr/0003-browser-async-effects.md)
### 如何使用最新 Marqdo(v1.0.2)
### 如何使用最新 Marqdo(v1.0.3)
**安装解释器(任选其一)**
| 方式 | 适用 | 命令 / 链接 |
|------|------|-------------|
| **Ubuntu PPA** | Ubuntu 24.04+(`noble` 等) | `sudo add-apt-repository ppa:cflmy/marqdo && sudo apt update && sudo apt install marqdo` |
| **GitHub Releases** | Windows / Linux 预编译 zip | https://github.com/cflmy/marqdo/releases |
| **源码** | 需本机 Rust | 见下方 `git clone` + `cargo build --release` |
**安装官方扩展**(与 CLI 可分开发版):`marqdo ext add …` 默认下载顺序为 **CDN** [`https://ext.marqdo.com`](https://ext.marqdo.com) → GitHub Releases → [`proxy.cflmy.top`](https://proxy.cflmy.top) 镜像。
```bash
# 1) 源码安装(跟 tag 或 main)
git clone https://github.com/cflmy/marqdo.git && cd marqdo
git checkout v1.0.2 # 或留在 main
git checkout v1.0.3 # 或留在 main
cargo build --release
export PATH="$PWD/target/release:$PATH"
marqdo version
@@ -138,7 +148,7 @@ marqdo view public --no-open
marqdo debug public --no-open
marqdo catalog public -o .marqdo
# 3) 安装官方扩展(Release 用户:无需本机 Rust/Go)
# 3) 安装官方扩展(Release / CDN 用户:无需本机 Rust/Go)
marqdo ext add web # 或:网页 — 自动下载 L1 + 预编译 native(Go libweb)
marqdo ext add agent # 或:智能体
marqdo ext add quantum # 或:量子
@@ -159,7 +169,7 @@ marqdo wasm build
# → dist/wasm/ … 见 examples/browser-hello/
# 6) Releases:Windows exe/zip/vsix;Linux CLI zip + native `.so` zip
# https://github.com/cflmy/marqdo/releases/tag/v1.0.2
# https://github.com/cflmy/marqdo/releases/tag/v1.0.3
```
开发期也可用 `cargo run -- …` 代替已安装的 `marqdo`:
@@ -171,7 +181,7 @@ cargo run -- view output public -o public
powershell -File ./scripts/build-public.ps1
```
**发布包(GitHub Releases)**:单独二进制 **已内置**官方 `lib/`(`import …:lib/…` 可直接导入)。仍提供带 `lib/` 的 zip 便于覆盖或离线分发。**扩展库**用 `marqdo ext add …` 安装(见 [CHANGELOG](CHANGELOG.md) 与 [ext-cli.md](doc/design/ext-cli.md));原生 `.so`/`.dll` 需本地编译或从带 `native/` 的安装目录解析。
**发布包**:GitHub Releases 的单独二进制 **已内置**官方 `lib/`。Ubuntu 用户优先 **PPA** 装 CLI。**扩展库**用 `marqdo ext add …`(CDN → GitHub);设计见 [ext-cdn.md](doc/design/ext-cdn.md) · [ubuntu-ppa.md](doc/design/ubuntu-ppa.md) · [ext-cli.md](doc/design/ext-cli.md)。
文档:用户站 [public/](public/) · 设计 [doc/](doc/) · OKF / catalog [catalog-cli.md](doc/design/catalog-cli.md) · 调试 [view-debug.md](doc/design/view-debug.md) · 变更 [CHANGELOG.md](CHANGELOG.md)
---
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "marqdo-wasm"
version = "1.0.2"
version = "1.0.3"
edition = "2021"
license = "Apache-2.0"
description = "Marqdo interpreter as WebAssembly (browser host bridge)"
+22
View File
@@ -0,0 +1,22 @@
Marqdo for Ubuntu (PPA)
=======================
This package installs the marqdo CLI (/usr/bin/marqdo).
Official language extensions are not bundled. After install:
marqdo ext add web
marqdo ext add quantum
Extension packs download from https://ext.marqdo.com (then GitHub / proxy).
Install this package from:
sudo add-apt-repository ppa:cflmy/marqdo
sudo apt update
sudo apt install marqdo
Also available: GitHub Releases (Windows/Linux zips) and building from source.
Documentation: https://github.com/cflmy/marqdo
PPA notes: doc/design/ubuntu-ppa.md in the upstream repository.
+20
View File
@@ -0,0 +1,20 @@
marqdo (1.0.3-1ppa1) noble; urgency=medium
* Marqdo 1.0.3: PPA + ext CDN; unbound GFM tables; bundled Rust for noble.
-- cflmy <pingan@cflmy.cn> Thu, 17 Sep 2026 10:35:00 +0800
marqdo (1.0.2-1ppa2) noble; urgency=medium
* Bundle Rust toolchain in orig (third_party/rust) so noble can build
Cargo.lock v4 offline; drop distro cargo/rustc Build-Depends.
* Architecture: amd64 (bundled toolchain is x86_64).
-- cflmy <pingan@cflmy.cn> Thu, 17 Sep 2026 10:20:00 +0800
marqdo (1.0.2-1ppa1) noble; urgency=medium
* Initial Ubuntu PPA packaging for Marqdo 1.0.2.
* CLI only; extensions via "marqdo ext add" / ext.marqdo.com.
-- cflmy <pingan@cflmy.cn> Thu, 17 Sep 2026 09:55:00 +0800
+29
View File
@@ -0,0 +1,29 @@
Source: marqdo
Section: devel
Priority: optional
Maintainer: cflmy <pingan@cflmy.cn>
Build-Depends:
debhelper-compat (= 13),
build-essential,
pkg-config,
libssl-dev,
ca-certificates
Standards-Version: 4.7.0
Homepage: https://github.com/cflmy/marqdo
Vcs-Git: https://github.com/cflmy/marqdo.git
Vcs-Browser: https://github.com/cflmy/marqdo
Rules-Requires-Root: no
Package: marqdo
Architecture: amd64
Depends: ${shlibs:Depends}, ${misc:Depends}, ca-certificates
Description: Markdown markers as an interpreted programming language
Marqdo treats Markdown markers as executable syntax (code / return /
control flow). This package installs the marqdo CLI.
.
Official extensions (web, quantum, …) are installed separately via
"marqdo ext add" and download from https://ext.marqdo.com .
.
Launchpad builds use a bundled Rust toolchain under third_party/rust
(Ubuntu noble cargo is too old for Cargo.lock v4). Rebuilds on newer
series may use distro rustc if cargo ≥ 1.78.
+24
View File
@@ -0,0 +1,24 @@
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: marqdo
Upstream-Contact: cflmy <pingan@cflmy.cn>
Source: https://github.com/cflmy/marqdo
Files: *
Copyright: 2024-2026 cflmy
License: Apache-2.0
License: Apache-2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
.
http://www.apache.org/licenses/LICENSE-2.0
.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
.
On Debian systems, the complete text of the Apache License version 2
can be found in "/usr/share/common-licenses/Apache-2.0".
Vendored Executable
+53
View File
@@ -0,0 +1,53 @@
#!/usr/bin/make -f
export DH_VERBOSE = 1
# Offline builds on Launchpad: vendored crates + optional bundled Rust toolchain.
export CARGO_HOME = $(CURDIR)/debian/cargo_home
export CARGO_NET_OFFLINE = true
# Noble (and older) distro cargo cannot read Cargo.lock v4 / modern crates.
# Prefer third_party/rust from the orig tarball when present (see ppa-build-source.sh).
BUNDLED_RUST = $(CURDIR)/third_party/rust
ifneq ($(wildcard $(BUNDLED_RUST)/bin/cargo),)
export PATH := $(BUNDLED_RUST)/bin:$(PATH)
endif
%:
dh $@
override_dh_auto_configure:
mkdir -p debian/cargo_home
@if [ -x "$(BUNDLED_RUST)/bin/cargo" ]; then \
echo "Using bundled Rust: $$($(BUNDLED_RUST)/bin/rustc --version)"; \
else \
echo "Using system Rust: $$(rustc --version 2>/dev/null || echo missing)"; \
fi
@if [ -f "$(CURDIR)/.cargo/config.toml" ] || [ -d "$(CURDIR)/vendor" ]; then \
echo "Using vendored Cargo dependencies"; \
else \
echo "WARNING: no vendor/ — online cargo fetch required (not for Launchpad)"; \
fi
override_dh_auto_build:
mkdir -p debian/cargo_home
@command -v cargo >/dev/null || { echo "error: cargo not found (bundle third_party/rust or install rustc/cargo)" >&2; exit 1; }
if [ -d "$(CURDIR)/vendor" ]; then \
CARGO_NET_OFFLINE=true cargo build --release --locked --offline \
-p marqdo; \
else \
cargo build --release --locked -p marqdo; \
fi
override_dh_auto_test:
@echo "Skipping full test suite on package build (run cargo test in CI)."
override_dh_auto_install:
install -d $(CURDIR)/debian/marqdo/usr/bin
install -m 0755 $(CURDIR)/target/release/marqdo \
$(CURDIR)/debian/marqdo/usr/bin/marqdo
override_dh_auto_clean:
dh_auto_clean
rm -rf debian/cargo_home
# keep vendor/ and third_party/rust (part of source package)
-cargo clean
+1
View File
@@ -0,0 +1 @@
3.0 (quilt)
+2
View File
@@ -0,0 +1,2 @@
# Ignore temporary files while editing patches
extend-diff-ignore = "^vendor/"
+2
View File
@@ -53,6 +53,8 @@
| [design/ext-agent.md](design/ext-agent.md) | **官方扩展 `ext/agent`(文档驱动:单步 / 多步工作簿 + 自写回;废弃 thin TOOL 循环)** |
| [design/ext-agent-plan.md](design/ext-agent-plan.md) | **`## 多步` / `## plan` 锁定设计**(工作簿文件、subtask 执行、修订循环、`step` 默认写回) |
| [design/ext-cli.md](design/ext-cli.md) | **`marqdo ext list/add/remove` 官方扩展安装器** |
| [design/ext-cdn.md](design/ext-cdn.md) | **扩展 CDN(R2 / ext.marqdo.com)与分开发版** |
| [design/ubuntu-ppa.md](design/ubuntu-ppa.md) | **Ubuntu PPA 打包与上传** |
| [design/ext-web.md](design/ext-web.md) | **官方扩展 `ext/web`(页面/库表/绑定/后台;ABI + CLI;设计锁定)** |
| [design/ext-web-go-rewrite.md](design/ext-web-go-rewrite.md) | **`plugins/web` Go 全量重写**(多语言 ABI · 网页制备完备 · anlian 验收) |
| [adr/0004-web-plugin-go.md](adr/0004-web-plugin-go.md) | **Accepted:web 原生插件实现语言为 Go** |
+66
View File
@@ -0,0 +1,66 @@
# Extension CDN (Cloudflare R2 → https://ext.marqdo.com)
| | |
|---|---|
| Status | **Accepted** — download prefer CDN; upload via `scripts/upload-ext-r2.py` |
| Date | 2026-09-17 |
| Related | [ext-cli.md](ext-cli.md) · [marqdo-release](../../.cursor/skills/marqdo-release/SKILL.md) |
## Why
CLI (语法解释器) and official **extension packs** (`ext/` L1 + native plugins) update on different cadences. Users behind unstable GitHub links should get packs from a stable object store first.
## Public URL layout
Bucket: `marqdo` · Custom domain: **https://ext.marqdo.com**
| Key | Meaning |
|-----|---------|
| `v{VER}/marqdo-{VER}-ext.zip` | L1 sources |
| `v{VER}/marqdo-{VER}-native-{triple}.zip` | Prebuilt plugins |
| `v{VER}/VERSION` | Pack SemVer echo |
| `latest/VERSION` | Recommended pack SemVer (plain text) |
| `latest/marqdo-…zip` | Same assets mirrored for convenience |
`{VER}` is the **extension pack** version (`ext/VERSION`), which **may differ** from CLI `Cargo.toml` version.
## Client download order (`marqdo ext add`)
1. `MARQDO_EXT_DOWNLOAD_BASE` (optional override)
2. **CDN** `https://ext.marqdo.com/v{VER}/…` (override host with `MARQDO_EXT_CDN`)
3. GitHub Releases
4. `https://proxy.cflmy.top/github.com/…`
Pack SemVer resolution (`release_version()`):
1. `MARQDO_EXT_VERSION`
2. CDN `GET …/latest/VERSION` (short timeout)
3. Embedded `ext/VERSION` at build time
4. CLI `CARGO_PKG_VERSION`
## Separate ext release (no CLI bump)
1. Bump `ext/VERSION` (e.g. `1.0.3`) — leave root `Cargo.toml` alone if CLI unchanged.
2. Build native zips (CI Linux/Windows jobs or local `scripts/build-web-plugin.sh` + cargo plugins).
3. Package `marqdo-{VER}-ext.zip` and `marqdo-{VER}-native-*.zip`.
4. Upload **both**:
- R2: `python3 scripts/upload-ext-r2.py --version VER --from-dir dist/`
- GitHub: attach to a GitHub Release (tag `ext-vVER` **or** reuse latest CLI release assets)
5. Confirm `https://ext.marqdo.com/latest/VERSION` and a sample zip URL return 200.
## Full product release (CLI + ext)
Keep `ext/VERSION` equal to (or compatible with) CLI SemVer unless documenting a deliberate skew. CI still publishes GitHub assets; **after** CI green, run `upload-ext-r2.py` so CDN is populated (or add a CI step with R2 secrets).
## Credentials
| Where | What |
|-------|------|
| Local | `~/.marqdo/r2.env` (`chmod 600`) — see `.env.r2.example` |
| CI (optional) | GitHub Actions secrets: `R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY`, `R2_ENDPOINT`, `R2_BUCKET` |
**Never** commit Account API tokens or S3 secret keys. If a token was pasted into chat, **rotate** it in the Cloudflare dashboard.
## Object ACL
Custom domain public reads require the bucket/objects to be publicly readable (R2 public bucket / r2.dev / custom domain). Uploads use S3 API credentials; downloads by `ext add` are anonymous HTTPS GETs.
+7 -1
View File
@@ -51,10 +51,16 @@ L1 sources:
| Variable | Meaning |
|----------|---------|
| `MARQDO_EXT_VERSION` | Override SemVer used for Release asset names (default: CLI `CARGO_PKG_VERSION`) |
| `MARQDO_EXT_VERSION` | Override **extension pack** SemVer (default: CDN `latest/VERSION` → embedded `ext/VERSION` → CLI version) |
| `MARQDO_EXT_CDN` | Public CDN base (default `https://ext.marqdo.com`) |
| `MARQDO_EXT_NO_DOWNLOAD=1` | Never hit the network (CI / air-gap); local/cargo only |
| `MARQDO_EXT_DOWNLOAD_BASE` | Optional extra base tried **before** CDN |
| `MARQDO_WEB_PLUGIN` / `MARQDO_AGENT_PLUGIN` / `MARQDO_QUANTUM_PLUGIN` | Explicit native path |
Download order for each Release asset: `MARQDO_EXT_DOWNLOAD_BASE` → **CDN `ext.marqdo.com`** → `github.com` → **`proxy.cflmy.top`**. Connect timeout 20s; read timeout 10min.
See [ext-cdn.md](ext-cdn.md) for R2 layout, separate ext releases, and `scripts/upload-ext-r2.py`.
Windows Release **zip** also ships `ext/` + `ext/native/*.dll` next to `marqdo.exe` so a portable unzip works offline after one download.
## Source for `add` (developers)
+1 -1
View File
@@ -211,7 +211,7 @@ v0.3:
| 体内 `1.` `2.` … | 分支 |
| 体内 `-` | 循环 |
| `N. *` | else 臂(此处 `*` 仍是「空条件」标记,**不是**返回值;见 §7.3) |
| GFM 表格 | 集合(不变) |
| GFM 表格 | 集合;`` `名` = `` 后绑定,或**未绑定**时求值后丢弃(文档中可写表) |
| 表达式 `[键](集合)` | **取元**(与 foreach 同形;见 [collection-access-link.md](collection-access-link.md)) |
| `修饰… [函数名] 实参…` | **方括号标记调用**(前置修饰 → `名=True`;见 [bracket-call-modifiers.md](bracket-call-modifiers.md)) |
| `` 集合[^键] `` | **过渡**:旧脚注取元(仍可用) |
+83
View File
@@ -0,0 +1,83 @@
# Ubuntu PPA packaging (Marqdo)
| | |
|---|---|
| Status | **Active** — `ppa:cflmy/marqdo`; noble builds need bundled Rust (Cargo.lock v4) |
| Maintainer | cflmy \<pingan@cflmy.cn\> |
| GPG | `505943294D04C803` (fingerprint `52A1…C803`) |
| PPA | `ppa:cflmy/marqdo` |
## Goal
Users install with:
```bash
sudo add-apt-repository ppa:cflmy/marqdo
sudo apt update
sudo apt install marqdo
```
CLI and extension packs stay independent: the `.deb` ships the **interpreter**; `marqdo ext add …` still pulls from **https://ext.marqdo.com** (then GitHub / proxy).
## Why bundled Rust?
Ubuntu **noble** ships cargo/rustc **1.75**, which cannot parse **Cargo.lock version 4** (`lock file version 4 requires -Znext-lockfile-bump`). Launchpad builders also have **no network**, so `ppa-build-source.sh` vendors crates **and** installs a modern toolchain into `third_party/rust` inside the `.orig.tar.xz`. `debian/rules` prefers that toolchain over distro cargo.
## One-time Launchpad setup
1. Create PPA: https://launchpad.net/~cflmy/+activate-ppa → name e.g. `marqdo`
2. OpenPGP key already on Launchpad (done)
3. Sign Ubuntu Code of Conduct if Launchpad asks
4. Enable the Ubuntu **series** you upload for (e.g. noble) under PPA → Change details
5. On the build machine:
```bash
sudo apt install build-essential debhelper devscripts dput lintian \
dpkg-dev quilt cargo rustc pkg-config curl xz-utils
```
## Package layout
| File | Role |
|------|------|
| `debian/control` | Metadata (amd64; no distro cargo BD) |
| `debian/changelog` | Upload revisions (`1ppaN`) |
| `debian/rules` | `dh` + bundled/system cargo build |
| `scripts/ppa-build-source.sh` | Vendor + bundle Rust + `debuild -S` |
| `scripts/ppa-ship.sh` | apt deps + build + `dput` |
## Versioning
- Upstream SemVer: `Cargo.toml` / git tag `vX.Y.Z`
- Debian revision: `X.Y.Z-1ppaN~SERIES` (SERIES = `noble`, …) — bump `1ppaN` on packaging-only reuploads
- **Ext pack** SemVer (`ext/VERSION`) is **not** the deb version
## One-shot (recommended)
```bash
cd ~/work/marqdo
./scripts/ppa-ship.sh noble # tools → debuild -S → dput
./scripts/ppa-ship.sh noble -n # build only
```
If automation cannot enter sudo/GPG, tell the user these commands (do not invent alternatives):
```bash
./scripts/ppa-build-source.sh noble
dput ppa:cflmy/marqdo ../build-area/marqdo_*~noble_source.changes
```
Watch: https://launchpad.net/~cflmy/+archive/ubuntu/marqdo/+builds
## Failure notes
| Symptom | Fix |
|---------|-----|
| `lock file version 4 requires -Znext-lockfile-bump` | Rebuild with current `ppa-build-source.sh` (bundles Rust); bump `1ppaN` |
| Same version rejected by Launchpad | Bump `debian/changelog` (`1ppa2`, …) |
| Wrong series | Enable series on PPA; rebuild with that series name |
## Out of scope (v1 deb)
- Shipping `ext/native/*.so` inside the deb (use CDN `ext add`)
- WASM / VSIX / non-amd64
+1
View File
@@ -0,0 +1 @@
1.0.3
+19
View File
@@ -13,6 +13,25 @@ AI already prefers Markdown-shaped output. The idea was: why keep docs and code
The language is still young: structure markers, bilingual core names (`print` / `打印`, …), importable stdlibs (see [`stdlib/`](stdlib/) on this site), local `view`, and static export. Contribute on [GitHub](https://github.com/cflmy/marqdo).
## Install
Pick one way to get the CLI:
| Method | Notes |
|--------|-------|
| Ubuntu PPA | `sudo add-apt-repository ppa:cflmy/marqdo` then `sudo apt install marqdo` |
| GitHub Releases | Prebuilt Windows / Linux zips at [releases](https://github.com/cflmy/marqdo/releases) |
| From source | `cargo build --release` (needs Rust) |
Official extensions version independently of the CLI:
```text
marqdo ext add web
marqdo ext add quantum
```
`ext add` tries CDN https://ext.marqdo.com first, then GitHub, then the proxy mirror. See [`features/05-extensions.mq.md`](features/05-extensions.mq.md).
中文文档请看同目录的 [`00-欢迎.mq.md`](00-欢迎.mq.md)(叙述与内置均用中文)。
A small function to say hello:
+19
View File
@@ -13,6 +13,25 @@ AI 偏爱 Markdown结构的输出。作者的念头是:何必再把文档和
语言仍在起步:结构标记、中英内核双名(`打印` / `print` …)、可导入的标准库(见本站 [`stdlib/`](stdlib/))、本地 `view` 与静态导出。欢迎到 [GitHub](https://github.com/cflmy/marqdo) 一起做。
## 安装
解释器(CLI)任选其一:
| 方式 | 说明 |
|------|------|
| Ubuntu PPA | `sudo add-apt-repository ppa:cflmy/marqdo` 后 `sudo apt install marqdo` |
| GitHub Releases | [releases](https://github.com/cflmy/marqdo/releases) 下载 Windows / Linux 预编译包 |
| 源码 | `cargo build --release`(需本机 Rust) |
官方扩展与 CLI 可分开更新。安装扩展:
```text
marqdo ext add web
marqdo ext add quantum
```
`ext add` 下载顺序:CDN https://ext.marqdo.com → GitHub → proxy 镜像。详见本站 [`features/05-扩展.mq.md`](features/05-扩展.mq.md)。
English docs: [`00-welcome.mq.md`](00-welcome.mq.md) (English prose and English builtins).
用一个小函数打个招呼:
+7 -3
View File
@@ -5,7 +5,7 @@ description: Optional packages — not stdlib
# main
ext/ holds official optional extensions. They are not part of lib/. Resolve via MARQDO_EXT, ./ext, or ext/ next to the binary.
ext/ holds official optional extensions. They are not part of lib/. Resolve via `~/.marqdo/ext`, MARQDO_EXT, ./ext, or ext/ next to the binary.
Install (v0.2+):
@@ -15,14 +15,18 @@ marqdo ext add llm
marqdo ext add agent
marqdo ext add web
marqdo ext add quantum
marqdo ext add linalg
```
For packages with a native plugin, build first then add:
`ext add` downloads L1 sources and prebuilt natives when available. Order: CDN https://ext.marqdo.com → GitHub Releases → proxy mirror. Pack SemVer is `ext/VERSION` and may differ from the CLI.
Rebuild natives locally only when developing plugins:
```text
cargo build --release -p marqdo_plugin_agent
cargo build --release -p marqdo_plugin_web
cargo build --release -p marqdo_plugin_quantum
cargo build --release -p marqdo_plugin_linalg
bash ./scripts/build-web-plugin.sh
marqdo ext add agent
marqdo ext add web
marqdo ext add quantum
+7 -3
View File
@@ -5,7 +5,7 @@ description: 可选包 — 不是标准库
# main
ext/ 存放官方可选扩展,不属于 lib/。解析顺序:MARQDO_EXT、./ext、或可执行文件旁的 ext/。
ext/ 存放官方可选扩展,不属于 lib/。解析顺序:`~/.marqdo/ext`、MARQDO_EXT、./ext、或可执行文件旁的 ext/。
安装(v0.2+):
@@ -15,14 +15,18 @@ marqdo ext add llm
marqdo ext add agent
marqdo ext add web
marqdo ext add quantum
marqdo ext add linalg
```
带原生插件的包需先编译再 add:
`ext add` 会下载 L1 与预编译 native(若有)。下载顺序:CDN https://ext.marqdo.com → GitHub Releases → proxy 镜像。扩展包版本见 `ext/VERSION`,可与 CLI 分开发版。
带原生插件的包若需本地重编:
```text
cargo build --release -p marqdo_plugin_agent
cargo build --release -p marqdo_plugin_web
cargo build --release -p marqdo_plugin_quantum
cargo build --release -p marqdo_plugin_linalg
bash ./scripts/build-web-plugin.sh
marqdo ext add agent
marqdo ext add web
marqdo ext add quantum
+155
View File
@@ -0,0 +1,155 @@
#!/usr/bin/env bash
# Build a signed Ubuntu *source* package for Launchpad PPA.
# Usage: ./scripts/ppa-build-source.sh [series]
# Example: ./scripts/ppa-build-source.sh noble
#
# Launchpad builders have no network and noble's cargo (1.75) cannot read
# Cargo.lock v4. This script therefore:
# 1) cargo vendor → vendor/
# 2) installs a modern Rust toolchain into third_party/rust (cached)
# 3) packs both into the .orig.tar.xz for offline debuild on Launchpad
set -euo pipefail
SERIES="${1:-noble}"
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$ROOT"
need() { command -v "$1" >/dev/null || { echo "missing tool: $1 (sudo apt install $2)"; exit 1; }; }
need cargo cargo
need debuild devscripts
need dpkg-parsechangelog dpkg-dev
need curl curl
need tar tar
need xz xz-utils
VER="$(sed -n 's/^version = "\(.*\)"/\1/p' Cargo.toml | head -1)"
# Debian revision comes from debian/changelog (e.g. 1.0.2-1ppa2); we append ~series.
CHG_VER="$(dpkg-parsechangelog -l debian/changelog -S Version)"
# Strip any previous ~series suffix from changelog version
BASE_DEB_VER="${CHG_VER%%~*}"
# If changelog still has old upstream, prefer Cargo VER for the upstream part
if [[ "$BASE_DEB_VER" != "$VER"-* ]]; then
# keep debian revision suffix after first '-' following VER if possible
REV="${BASE_DEB_VER#*-}"
if [[ "$BASE_DEB_VER" == *"-"* ]]; then
BASE_DEB_VER="${VER}-${REV}"
else
BASE_DEB_VER="${VER}-1ppa1"
fi
fi
DEB_VER="${BASE_DEB_VER}~${SERIES}"
PKG_ROOT="${TMPDIR:-/tmp}/marqdo-ppa/${VER}"
BUILD_AREA="${ROOT}/../build-area"
GPG_KEY="${MARQDO_PPA_GPG_KEY:-505943294D04C803}"
RUST_VER="${MARQDO_PPA_RUST_VERSION:-1.85.0}"
RUST_TRIPLE="${MARQDO_PPA_RUST_TRIPLE:-x86_64-unknown-linux-gnu}"
RUST_NAME="rust-${RUST_VER}-${RUST_TRIPLE}"
RUST_CACHE="${MARQDO_PPA_RUST_CACHE:-${HOME}/.cache/marqdo-ppa}"
RUST_TARBALL="${RUST_CACHE}/${RUST_NAME}.tar.gz"
RUST_URL="${MARQDO_PPA_RUST_URL:-https://static.rust-lang.org/dist/${RUST_NAME}.tar.gz}"
echo "==> upstream ${VER} → debian ${DEB_VER} (${SERIES})"
rm -rf "$PKG_ROOT"
mkdir -p "$PKG_ROOT" "$BUILD_AREA" "$RUST_CACHE"
# Clean export of sources (no target/, .git)
if command -v git >/dev/null && git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
git archive --format=tar HEAD | tar -x -C "$PKG_ROOT"
else
rsync -a --exclude target --exclude .git --exclude build-area "$ROOT"/ "$PKG_ROOT"/
fi
# Ensure debian/ from working tree (may be newer than last commit)
rsync -a "$ROOT/debian/" "$PKG_ROOT/debian/"
cd "$PKG_ROOT"
python3 - <<PY
from pathlib import Path
import re
p = Path("debian/changelog")
text = p.read_text(encoding="utf-8")
text2 = re.sub(
r"^marqdo \([^)]+\) [^;]+;",
"marqdo (${DEB_VER}) ${SERIES};",
text,
count=1,
flags=re.M,
)
p.write_text(text2, encoding="utf-8")
print("changelog:", p.read_text(encoding="utf-8").splitlines()[0])
PY
echo "==> cargo vendor (Launchpad offline)"
mkdir -p .cargo
cargo vendor --locked vendor >/tmp/marqdo-cargo-vendor.conf
cat > .cargo/config.toml <<'EOF'
[source.crates-io]
replace-with = "vendored-sources"
[source.vendored-sources]
directory = "vendor"
EOF
echo "==> bundle Rust ${RUST_VER} (${RUST_TRIPLE}) for Launchpad"
if [ ! -f "$RUST_TARBALL" ]; then
echo " downloading ${RUST_URL}"
curl -fL --retry 3 --retry-delay 2 -o "${RUST_TARBALL}.partial" "$RUST_URL"
mv "${RUST_TARBALL}.partial" "$RUST_TARBALL"
else
echo " cache hit ${RUST_TARBALL}"
fi
rm -rf third_party/rust
mkdir -p third_party
RUST_EXTRACT="$(mktemp -d "${TMPDIR:-/tmp}/marqdo-rust.XXXXXX")"
tar -xzf "$RUST_TARBALL" -C "$RUST_EXTRACT"
# Official dist tarball contains install.sh
INSTALL_SH="$(find "$RUST_EXTRACT" -maxdepth 2 -name install.sh | head -1)"
if [ -z "$INSTALL_SH" ]; then
echo "error: install.sh not found in ${RUST_TARBALL}" >&2
exit 1
fi
# --disable-ldconfig avoids needing root; strip docs to save space
bash "$INSTALL_SH" \
--prefix="$PKG_ROOT/third_party/rust" \
--without=rust-docs \
--disable-ldconfig
rm -rf "$RUST_EXTRACT"
"$PKG_ROOT/third_party/rust/bin/rustc" --version
"$PKG_ROOT/third_party/rust/bin/cargo" --version
# Always rebuild orig (vendor + rust toolchain change every run)
cd "$(dirname "$PKG_ROOT")"
ORIG="marqdo_${VER}.orig.tar.xz"
echo "==> packing ${ORIG} (vendor + third_party/rust)"
rm -f "$BUILD_AREA/$ORIG" "marqdo_${VER}.orig.tar.xz"
tar -cJf "$BUILD_AREA/$ORIG" \
--exclude=debian \
-C "$(dirname "$PKG_ROOT")" "$(basename "$PKG_ROOT")"
ln -sfn "$BUILD_AREA/$ORIG" "marqdo_${VER}.orig.tar.xz"
ls -lh "$BUILD_AREA/$ORIG"
cd "$PKG_ROOT"
echo "==> debuild -S (key ${GPG_KEY})"
debuild -S -sa -k"${GPG_KEY}" -d
# debuild writes next to package parent (/tmp/marqdo-ppa/); collect into build-area
mkdir -p "$BUILD_AREA"
shopt -s nullglob
for f in ../marqdo_"${DEB_VER}"* ../marqdo_"${VER}".orig.tar.*; do
[ -e "$f" ] || continue
if [ "$(realpath -m "$f")" = "$(realpath -m "$BUILD_AREA/$(basename "$f")")" ]; then
continue
fi
cp -f "$f" "$BUILD_AREA/"
done
shopt -u nullglob
echo "==> artifacts in $BUILD_AREA:"
ls -la "$BUILD_AREA"/marqdo_"${DEB_VER}"* "$BUILD_AREA"/marqdo_"${VER}".orig.tar.* 2>/dev/null || true
CHANGES="$BUILD_AREA/marqdo_${DEB_VER}_source.changes"
if [ -f "$CHANGES" ]; then
echo
echo "Upload with:"
echo " dput ppa:cflmy/marqdo $CHANGES"
echo " # or: ./scripts/ppa-ship.sh ${SERIES}"
fi
+107
View File
@@ -0,0 +1,107 @@
#!/usr/bin/env bash
# One-shot: install packaging tools → build signed source package → dput to Launchpad PPA.
#
# Run in YOUR terminal (needs sudo for apt + GPG passphrase for signing):
# ./scripts/ppa-ship.sh # default series: noble
# ./scripts/ppa-ship.sh resolute # match host /etc/os-release
# ./scripts/ppa-ship.sh noble --no-upload
# sudo ./scripts/ppa-ship.sh noble # also OK — build/sign/upload drop back to $SUDO_USER
#
# PPA: ppa:cflmy/marqdo | key: 505943294D04C803
set -euo pipefail
SERIES="${1:-noble}"
NO_UPLOAD=0
for arg in "${@:2}"; do
case "$arg" in
--no-upload|-n) NO_UPLOAD=1 ;;
*) echo "unknown arg: $arg" >&2; exit 2 ;;
esac
done
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$ROOT"
# Never cargo/debuild/dput as root — GPG lives in the human user's ~/.gnupg
if [ "$(id -u)" -eq 0 ]; then
if [ -z "${SUDO_USER:-}" ] || [ "$SUDO_USER" = root ]; then
echo "error: run as a normal user, or: sudo -u \$USER ./scripts/ppa-ship.sh" >&2
exit 1
fi
REAL_USER="$SUDO_USER"
REAL_HOME="$(getent passwd "$REAL_USER" | cut -d: -f6)"
echo "==> root detected; apt as root, build/upload as ${REAL_USER}"
run_user() {
local upath="${REAL_HOME}/.cargo/bin:/usr/local/bin:/usr/bin:/bin"
sudo -u "$REAL_USER" -H --preserve-env=MARQDO_PPA_GPG_KEY \
env HOME="$REAL_HOME" PATH="$upath" "$@"
}
APT=(apt-get)
else
REAL_USER="$(id -un)"
REAL_HOME="$HOME"
run_user() { "$@"; }
APT=(sudo apt-get)
fi
PPA_TARGET="${MARQDO_PPA:-ppa:cflmy/marqdo}"
PKGS=(build-essential debhelper devscripts dput lintian
dpkg-dev quilt cargo rustc pkg-config libssl-dev ca-certificates
python3 xz-utils)
need_apt=0
for bin in debuild dput dpkg-parsechangelog cargo gpg; do
if ! command -v "$bin" >/dev/null 2>&1; then
need_apt=1
break
fi
done
# dh_auto_* comes from debhelper even if debuild exists from a partial install
if ! dpkg -s debhelper >/dev/null 2>&1; then
need_apt=1
fi
if [ "$need_apt" -eq 1 ]; then
echo "==> installing packaging toolchain (sudo)…"
"${APT[@]}" update -qq
DEBIAN_FRONTEND=noninteractive "${APT[@]}" install -y "${PKGS[@]}"
else
echo "==> packaging tools already present"
fi
echo "==> building source package for series=${SERIES}"
run_user "$ROOT/scripts/ppa-build-source.sh" "$SERIES"
VER="$(sed -n 's/^version = "\(.*\)"/\1/p' "$ROOT/Cargo.toml" | head -1)"
BUILD_AREA="$(cd "$ROOT/.." && pwd)/build-area"
# Prefer changes matching this series; fall back to newest source.changes
CHANGES="$(ls -1t "$BUILD_AREA"/marqdo_*"~${SERIES}"_source.changes 2>/dev/null | head -1 || true)"
if [ -z "${CHANGES:-}" ]; then
CHANGES="$(ls -1t "$BUILD_AREA"/marqdo_*_source.changes 2>/dev/null | head -1 || true)"
fi
if [ -z "${CHANGES:-}" ] || [ ! -f "$CHANGES" ]; then
echo "error: no *_source.changes under $BUILD_AREA" >&2
ls -la "$BUILD_AREA" 2>/dev/null || true
exit 1
fi
echo "==> artifact: $CHANGES"
if command -v lintian >/dev/null 2>&1; then
echo "==> lintian (non-fatal)"
run_user lintian -i "$CHANGES" || true
fi
if [ "$NO_UPLOAD" -eq 1 ]; then
echo
echo "Skip upload (--no-upload). When ready:"
echo " dput ${PPA_TARGET} ${CHANGES}"
exit 0
fi
echo "==> dput ${PPA_TARGET}"
run_user dput --unchecked "${PPA_TARGET}" "$CHANGES"
echo
echo "Done. Watch builds:"
echo " https://launchpad.net/~cflmy/+archive/ubuntu/marqdo/+builds"
echo "Enable the Ubuntu series (${SERIES}) in PPA → Change details if Launchpad rejects the upload."
+222
View File
@@ -0,0 +1,222 @@
#!/usr/bin/env python3
"""Upload Marqdo extension packs to Cloudflare R2 (CDN: https://ext.marqdo.com).
No third-party deps (stdlib only) — AWS Signature V4 PutObject.
Credentials: ~/.marqdo/r2.env (chmod 600; never commit). Required:
R2_ENDPOINT, R2_BUCKET, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY
Optional: R2_PUBLIC_BASE, R2_REGION (default auto)
Usage:
python3 scripts/upload-ext-r2.py --version 1.0.2 dist/marqdo-1.0.2-ext.zip \\
dist/marqdo-1.0.2-native-x86_64-unknown-linux-gnu.zip
python3 scripts/upload-ext-r2.py --version 1.0.2 --from-dir dist/
"""
from __future__ import annotations
import argparse
import datetime as dt
import hashlib
import hmac
import os
import sys
import tempfile
import urllib.error
import urllib.request
from pathlib import Path
from urllib.parse import quote, urlparse
def load_env_file(path: Path) -> None:
if not path.is_file():
return
for line in path.read_text(encoding="utf-8").splitlines():
line = line.strip()
if not line or line.startswith("#") or "=" not in line:
continue
k, _, v = line.partition("=")
os.environ.setdefault(k.strip(), v.strip().strip("'").strip('"'))
def sha256_hex(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def hmac_sha256(key: bytes, msg: str) -> bytes:
return hmac.new(key, msg.encode("utf-8"), hashlib.sha256).digest()
def signing_key(secret: str, datestamp: str, region: str, service: str) -> bytes:
k_date = hmac_sha256(("AWS4" + secret).encode("utf-8"), datestamp)
k_region = hmac.new(k_date, region.encode("utf-8"), hashlib.sha256).digest()
k_service = hmac.new(k_region, service.encode("utf-8"), hashlib.sha256).digest()
return hmac.new(k_service, b"aws4_request", hashlib.sha256).digest()
def put_object(
*,
endpoint: str,
bucket: str,
key: str,
body: bytes,
content_type: str,
access_key: str,
secret_key: str,
region: str,
) -> None:
endpoint = endpoint.rstrip("/")
host = urlparse(endpoint).netloc
# Path-style: https://endpoint/bucket/key
canonical_uri = "/" + bucket + "/" + quote(key, safe="/~")
amz_date = dt.datetime.now(dt.timezone.utc).strftime("%Y%m%dT%H%M%SZ")
datestamp = amz_date[:8]
payload_hash = sha256_hex(body)
canonical_headers = (
f"content-type:{content_type}\n"
f"host:{host}\n"
f"x-amz-content-sha256:{payload_hash}\n"
f"x-amz-date:{amz_date}\n"
)
signed_headers = "content-type;host;x-amz-content-sha256;x-amz-date"
canonical_request = "\n".join(
[
"PUT",
canonical_uri,
"",
canonical_headers,
signed_headers,
payload_hash,
]
)
credential_scope = f"{datestamp}/{region}/s3/aws4_request"
string_to_sign = "\n".join(
[
"AWS4-HMAC-SHA256",
amz_date,
credential_scope,
sha256_hex(canonical_request.encode("utf-8")),
]
)
sig = hmac.new(
signing_key(secret_key, datestamp, region, "s3"),
string_to_sign.encode("utf-8"),
hashlib.sha256,
).hexdigest()
auth = (
f"AWS4-HMAC-SHA256 Credential={access_key}/{credential_scope}, "
f"SignedHeaders={signed_headers}, Signature={sig}"
)
url = endpoint + canonical_uri
req = urllib.request.Request(url, data=body, method="PUT")
req.add_header("Content-Type", content_type)
req.add_header("x-amz-content-sha256", payload_hash)
req.add_header("x-amz-date", amz_date)
req.add_header("Authorization", auth)
try:
with urllib.request.urlopen(req, timeout=300) as resp:
if resp.status not in (200, 201):
raise RuntimeError(f"PUT {key} HTTP {resp.status}")
except urllib.error.HTTPError as e:
detail = e.read().decode("utf-8", errors="replace")
raise RuntimeError(f"PUT {key} HTTP {e.code}: {detail}") from e
def content_type_for(path: Path) -> str:
if path.name == "VERSION" or path.suffix.lower() == ".txt":
return "text/plain; charset=utf-8"
if path.suffix.lower() == ".json":
return "application/json"
return "application/zip"
def upload_file(
path: Path,
key: str,
*,
endpoint: str,
bucket: str,
access_key: str,
secret_key: str,
region: str,
public_base: str,
) -> None:
body = path.read_bytes()
print(f"upload {path.name} → s3://{bucket}/{key} ({len(body)} bytes)")
put_object(
endpoint=endpoint,
bucket=bucket,
key=key,
body=body,
content_type=content_type_for(path),
access_key=access_key,
secret_key=secret_key,
region=region,
)
print(f" public: {public_base.rstrip('/')}/{key}")
def main() -> int:
load_env_file(Path.home() / ".marqdo" / "r2.env")
load_env_file(Path(".env"))
load_env_file(Path(".env.r2"))
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--version", required=True, help="Extension pack SemVer (no v prefix)")
ap.add_argument("--from-dir", type=Path, help="Upload all marqdo-{VER}-*.zip from dir")
ap.add_argument("files", nargs="*", type=Path, help="Files to upload")
ap.add_argument("--no-latest", action="store_true", help="Do not update latest/ pointers")
args = ap.parse_args()
ver = args.version.lstrip("v")
for key in ("R2_ENDPOINT", "R2_BUCKET", "R2_ACCESS_KEY_ID", "R2_SECRET_ACCESS_KEY"):
if not os.environ.get(key):
print(
f"missing {key}; write ~/.marqdo/r2.env (see .env.r2.example)",
file=sys.stderr,
)
return 1
endpoint = os.environ["R2_ENDPOINT"]
bucket = os.environ["R2_BUCKET"]
access_key = os.environ["R2_ACCESS_KEY_ID"]
secret_key = os.environ["R2_SECRET_ACCESS_KEY"]
region = os.environ.get("R2_REGION", "auto")
public_base = os.environ.get("R2_PUBLIC_BASE", "https://ext.marqdo.com")
files: list[Path] = [p for p in args.files if p.is_file()]
if args.from_dir:
files.extend(sorted(args.from_dir.glob(f"marqdo-{ver}-*.zip")))
if not files:
print("no files to upload", file=sys.stderr)
return 1
kw = dict(
endpoint=endpoint,
bucket=bucket,
access_key=access_key,
secret_key=secret_key,
region=region,
public_base=public_base,
)
for path in files:
upload_file(path, f"v{ver}/{path.name}", **kw)
if not args.no_latest:
upload_file(path, f"latest/{path.name}", **kw)
if not args.no_latest:
with tempfile.NamedTemporaryFile("w", encoding="utf-8", delete=False) as tf:
tf.write(ver + "\n")
tmp = Path(tf.name)
try:
upload_file(tmp, "latest/VERSION", **kw)
upload_file(tmp, f"v{ver}/VERSION", **kw)
finally:
tmp.unlink(missing_ok=True)
print("done.")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+173 -33
View File
@@ -1,16 +1,32 @@
//! Download official `ext/` L1 sources and native plugins from GitHub Releases.
//! Download official `ext/` L1 sources and native plugins.
//!
//! Enables `marqdo ext add web` without a local Rust toolchain or repo checkout.
//! Disable with `MARQDO_EXT_NO_DOWNLOAD=1`. Override tag with `MARQDO_EXT_VERSION=0.3.4`.
//!
//! Disable with `MARQDO_EXT_NO_DOWNLOAD=1`. Override pack SemVer with
//! `MARQDO_EXT_VERSION` (independent of CLI `CARGO_PKG_VERSION`).
//!
//! Download order (first success wins):
//! 1. `MARQDO_EXT_DOWNLOAD_BASE` (optional override)
//! 2. **CDN** `https://ext.marqdo.com` (Cloudflare R2 custom domain)
//! 3. GitHub Releases `github.com/cflmy/marqdo`
//! 4. GitHub via reverse proxy `proxy.cflmy.top`
use std::env;
use std::fs::{self, File};
use std::io::{self, Write};
use std::io::{self, Read, Write};
use std::path::{Path, PathBuf};
use std::sync::OnceLock;
use std::time::Duration;
use anyhow::{bail, Context, Result};
const REPO: &str = "cflmy/marqdo";
/// Public CDN for extension packs (R2 bucket `marqdo`, custom domain).
pub const EXT_CDN_BASE: &str = "https://ext.marqdo.com";
/// HTTPS reverse proxy used when github.com is slow or blocked.
const GITHUB_MIRROR_PREFIX: &str = "https://proxy.cflmy.top/github.com";
/// Packaged with the repo; may diverge from CLI SemVer for ext-only releases.
const EMBEDDED_EXT_VERSION: &str = include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/ext/VERSION"));
pub fn downloads_enabled() -> bool {
match env::var("MARQDO_EXT_NO_DOWNLOAD") {
@@ -22,17 +38,69 @@ pub fn downloads_enabled() -> bool {
}
}
/// SemVer without leading `v` (matches release asset names).
fn normalize_semver(raw: &str) -> Option<String> {
let v = raw.trim().trim_start_matches('v').trim();
if v.is_empty() {
return None;
}
// one line only
let v = v.lines().next()?.trim().trim_start_matches('v').trim();
if v.is_empty() {
return None;
}
Some(v.to_string())
}
/// SemVer of the **extension pack** (may differ from CLI version).
///
/// Resolution order:
/// 1. `MARQDO_EXT_VERSION`
/// 2. CDN `GET {EXT_CDN}/latest/VERSION` (short timeout; skipped if downloads disabled)
/// 3. Embedded `ext/VERSION` at build time
/// 4. CLI `CARGO_PKG_VERSION`
pub fn release_version() -> String {
if let Ok(v) = env::var("MARQDO_EXT_VERSION") {
let v = v.trim().trim_start_matches('v');
if !v.is_empty() {
return v.to_string();
if let Some(n) = normalize_semver(&v) {
return n;
}
}
if downloads_enabled() {
if let Some(n) = fetch_cdn_latest_version() {
return n;
}
}
if let Some(n) = normalize_semver(EMBEDDED_EXT_VERSION) {
return n;
}
env!("CARGO_PKG_VERSION").to_string()
}
fn fetch_cdn_latest_version() -> Option<String> {
static CACHED: OnceLock<Option<String>> = OnceLock::new();
CACHED
.get_or_init(|| {
let base = env::var("MARQDO_EXT_CDN")
.ok()
.map(|s| s.trim().trim_end_matches('/').to_string())
.filter(|s| !s.is_empty())
.unwrap_or_else(|| EXT_CDN_BASE.to_string());
let url = format!("{base}/latest/VERSION");
let agent = ureq::AgentBuilder::new()
.timeout_connect(Duration::from_secs(5))
.timeout_read(Duration::from_secs(10))
.user_agent(&user_agent())
.build();
let resp = agent.get(&url).call().ok()?;
if !(200..300).contains(&resp.status()) {
return None;
}
let mut body = String::new();
resp.into_reader().read_to_string(&mut body).ok()?;
normalize_semver(&body)
})
.clone()
}
pub fn host_target_triple() -> Option<&'static str> {
if cfg!(all(target_os = "windows", target_arch = "x86_64")) {
Some("x86_64-pc-windows-msvc")
@@ -61,13 +129,58 @@ fn user_agent() -> String {
format!("marqdo/{}", env!("CARGO_PKG_VERSION"))
}
fn http_agent() -> ureq::Agent {
ureq::AgentBuilder::new()
.timeout_connect(Duration::from_secs(20))
.timeout_read(Duration::from_secs(600))
.user_agent(&user_agent())
.build()
}
fn cdn_base() -> String {
env::var("MARQDO_EXT_CDN")
.ok()
.map(|s| s.trim().trim_end_matches('/').to_string())
.filter(|s| !s.is_empty())
.unwrap_or_else(|| EXT_CDN_BASE.to_string())
}
/// Candidate download URLs for a release asset.
///
/// Order: optional `MARQDO_EXT_DOWNLOAD_BASE` → CDN → GitHub → GitHub proxy.
pub fn release_asset_urls(ver: &str, filename: &str) -> Vec<String> {
let mut urls = Vec::new();
if let Ok(base) = env::var("MARQDO_EXT_DOWNLOAD_BASE") {
let base = base.trim().trim_end_matches('/');
if !base.is_empty() {
urls.push(format!("{base}/{REPO}/releases/download/v{ver}/{filename}"));
urls.push(format!("{base}/releases/download/v{ver}/{filename}"));
urls.push(format!("{base}/v{ver}/{filename}"));
urls.push(format!("{base}/{filename}"));
}
}
let cdn = cdn_base();
// Preferred public layout on ext.marqdo.com / R2
urls.push(format!("{cdn}/v{ver}/{filename}"));
urls.push(format!("{cdn}/releases/v{ver}/{filename}"));
urls.push(format!("{cdn}/{filename}"));
urls.push(format!(
"https://github.com/{REPO}/releases/download/v{ver}/{filename}"
));
urls.push(format!(
"{GITHUB_MIRROR_PREFIX}/{REPO}/releases/download/v{ver}/{filename}"
));
let mut seen = std::collections::HashSet::new();
urls.into_iter().filter(|u| seen.insert(u.clone())).collect()
}
fn download_to_file(url: &str, dest: &Path) -> Result<()> {
if let Some(parent) = dest.parent() {
fs::create_dir_all(parent)?;
}
let tmp = dest.with_extension("download-tmp");
let resp = ureq::get(url)
.set("User-Agent", &user_agent())
let resp = http_agent()
.get(url)
.set("Accept", "application/octet-stream")
.call()
.with_context(|| format!("GET {url}"))?;
@@ -86,6 +199,30 @@ fn download_to_file(url: &str, dest: &Path) -> Result<()> {
Ok(())
}
fn download_first_ok(urls: &[String], dest: &Path) -> Result<()> {
let mut errors: Vec<String> = Vec::new();
for (i, url) in urls.iter().enumerate() {
if i == 0 {
println!("downloading ({url})…");
} else {
println!("retry ({url})…");
}
match download_to_file(url, dest) {
Ok(()) => return Ok(()),
Err(e) => {
eprintln!("note: {e:#}");
errors.push(format!("{url} → {e:#}"));
let _ = fs::remove_file(dest.with_extension("download-tmp"));
}
}
}
bail!(
"all download mirrors failed for {}:\n - {}",
dest.file_name().and_then(|s| s.to_str()).unwrap_or("asset"),
errors.join("\n - ")
)
}
fn extract_zip(zip_path: &Path, dest_dir: &Path) -> Result<()> {
fs::create_dir_all(dest_dir)?;
let file = File::open(zip_path).with_context(|| format!("open {}", zip_path.display()))?;
@@ -121,10 +258,6 @@ fn extract_zip(zip_path: &Path, dest_dir: &Path) -> Result<()> {
Ok(())
}
fn release_asset_url(ver: &str, filename: &str) -> String {
format!("https://github.com/{REPO}/releases/download/v{ver}/{filename}")
}
/// Ensure L1 `ext/` tree is available (repo checkout or downloaded Release zip).
/// Returns a directory that contains `web/web.mq.md`, `ai/…`, etc.
pub fn ensure_ext_source_tree() -> Result<PathBuf> {
@@ -134,7 +267,6 @@ pub fn ensure_ext_source_tree() -> Result<PathBuf> {
if marker.is_file() {
return Ok(cache);
}
// Zip layout from CI: top-level `ext/…`
let nested = cache.join("ext").join("web").join("web.mq.md");
if nested.is_file() {
return Ok(cache.join("ext"));
@@ -143,16 +275,14 @@ pub fn ensure_ext_source_tree() -> Result<PathBuf> {
bail!("extension sources not found and downloads disabled (MARQDO_EXT_NO_DOWNLOAD)");
}
let zip_name = format!("marqdo-{ver}-ext.zip");
let url = release_asset_url(&ver, &zip_name);
let urls = release_asset_urls(&ver, &zip_name);
let zip_path = cache_root()?.join(&zip_name);
println!("downloading official ext sources ({url})…");
download_to_file(&url, &zip_path)?;
download_first_ok(&urls, &zip_path)?;
let stage = cache_root()?.join(format!("ext-src-v{ver}-extract"));
if stage.exists() {
let _ = fs::remove_dir_all(&stage);
}
extract_zip(&zip_path, &stage)?;
// Prefer `ext/` child if present
let src = if stage.join("ext").join("web").join("web.mq.md").is_file() {
stage.join("ext")
} else if stage.join("web").join("web.mq.md").is_file() {
@@ -165,7 +295,6 @@ pub fn ensure_ext_source_tree() -> Result<PathBuf> {
if cache.exists() {
let _ = fs::remove_dir_all(&cache);
}
// Move/copy tree into stable cache path
copy_dir_recursive(&src, &cache)?;
let _ = fs::remove_dir_all(&stage);
if !cache.join("web").join("web.mq.md").is_file() {
@@ -206,24 +335,22 @@ pub fn download_native_plugin(short: &str, lib_name: &str) -> Result<PathBuf> {
};
let ver = release_version();
let zip_name = format!("marqdo-{ver}-native-{triple}.zip");
let url = release_asset_url(&ver, &zip_name);
let urls = release_asset_urls(&ver, &zip_name);
let cache = cache_root()?.join(format!("native-v{ver}-{triple}"));
let lib_in_cache = cache.join("native").join(lib_name);
if lib_in_cache.is_file() {
return Ok(lib_in_cache);
}
// Also accept flat layout
let flat = cache.join(lib_name);
if flat.is_file() {
return Ok(flat);
}
let zip_path = cache_root()?.join(&zip_name);
println!("downloading prebuilt native plugins ({url})…");
download_to_file(&url, &zip_path).with_context(|| {
download_first_ok(&urls, &zip_path).with_context(|| {
format!(
"failed to download prebuilt plugins for {triple}. \
Build locally with `cargo build --release -p marqdo_plugin_{short}`, \
or check https://github.com/{REPO}/releases/tag/v{ver}"
Set MARQDO_EXT_CDN / MARQDO_EXT_DOWNLOAD_BASE, build locally, \
or check https://github.com/{REPO}/releases and https://ext.marqdo.com/v{ver}/"
)
})?;
if cache.exists() {
@@ -238,7 +365,6 @@ pub fn download_native_plugin(short: &str, lib_name: &str) -> Result<PathBuf> {
if flat.is_file() {
return Ok(flat);
}
// Search recursively for the lib name
if let Some(found) = find_named_file(&cache, lib_name)? {
return Ok(found);
}
@@ -283,17 +409,31 @@ mod tests {
use super::*;
#[test]
fn asset_url_shape() {
let u = release_asset_url("0.3.4", "marqdo-0.3.4-ext.zip");
assert_eq!(
u,
"https://github.com/cflmy/marqdo/releases/download/v0.3.4/marqdo-0.3.4-ext.zip"
);
fn asset_urls_prefer_cdn_then_github_then_proxy() {
let u = release_asset_urls("1.0.2", "marqdo-1.0.2-ext.zip");
assert!(u[0].starts_with("https://ext.marqdo.com/"));
assert!(u.iter().any(|x| x.contains("github.com/cflmy/marqdo")));
assert!(u
.iter()
.any(|x| x.contains("proxy.cflmy.top/github.com/cflmy/marqdo")));
let gh = u
.iter()
.position(|x| x.starts_with("https://github.com/"))
.unwrap();
let proxy = u
.iter()
.position(|x| x.contains("proxy.cflmy.top"))
.unwrap();
assert!(gh < proxy);
}
#[test]
fn embedded_ext_version_parses() {
assert!(normalize_semver(EMBEDDED_EXT_VERSION).is_some());
}
#[test]
fn triple_known_on_ci_hosts() {
// At least one of these builds is what we ship.
let t = host_target_triple();
if cfg!(any(
all(target_os = "linux", target_arch = "x86_64"),
+18 -5
View File
@@ -140,10 +140,18 @@ pub fn resolve_import(from_dir: &Path, rel: &str) -> Result<PathBuf> {
"cannot resolve import `{rel}` from {}",
from_dir.display()
);
if env::var_os("MARQDO_EXT").is_none() {
msg.push_str(
"\nhint: set MARQDO_EXT to the directory that contains web/ (or run `marqdo ext add web`)",
);
let user_ext = crate::ext_cli::default_user_ext_dir();
if env::var_os("MARQDO_EXT").is_none() && !user_ext.join("web").join("web.mq.md").is_file()
{
msg.push_str(&format!(
"\nhint: run `marqdo ext add web` (installs under {}), or set MARQDO_EXT",
user_ext.display()
));
} else if env::var_os("MARQDO_EXT").is_none() {
msg.push_str(&format!(
"\nhint: looked under {}; set MARQDO_EXT to override",
user_ext.display()
));
}
bail!("{msg}");
}
@@ -194,6 +202,8 @@ fn ext_search_roots() -> Vec<PathBuf> {
if let Ok(h) = env::var("MARQDO_EXT") {
roots.push(PathBuf::from(h));
}
// Default install root from `marqdo ext add` (~/.marqdo/ext)
roots.push(crate::ext_cli::default_user_ext_dir());
if let Ok(cwd) = env::current_dir() {
roots.push(cwd.join("ext"));
}
@@ -318,7 +328,10 @@ mod tests {
"{msg}"
);
assert!(
msg.contains("hint: set MARQDO_EXT to the directory that contains web/"),
msg.contains("hint:")
&& (msg.contains("marqdo ext add")
|| msg.contains("MARQDO_EXT")
|| msg.contains(".marqdo/ext")),
"{msg}"
);
}
+19
View File
@@ -323,6 +323,16 @@ impl<'a> Cursor<'a> {
fun.body.push(self.parse_loop_or_err()?);
continue;
}
// Unbound GFM table = collection value (not assigned); evaluate and discard.
if trimmed.starts_with('|') {
let span = Span {
line: l.line_no,
col: 1,
};
let value = self.consume_table()?;
fun.body.push(Stmt::Expr { value, span });
continue;
}
let stmt = self.parse_simple_stmt()?;
let end_body = matches!(stmt, Stmt::Return { .. }) && level >= 2;
@@ -382,6 +392,13 @@ impl<'a> Cursor<'a> {
return Ok(Stmt::Call { call, span });
}
// Unbound GFM table (header already consumed by bump above).
if trimmed.starts_with('|') {
self.i -= 1;
let value = self.consume_table()?;
return Ok(Stmt::Expr { value, span });
}
// Bare assign `` `x` = … ``
if trimmed.starts_with('`') {
if let Some(stmt) = self.parse_backtick_assign(trimmed, span.clone())? {
@@ -695,6 +712,8 @@ impl<'a> Cursor<'a> {
/// - first header `@` / `行` / `row` → List of row Maps (marker col excluded)
/// - else ≥2-col → Map (column-oriented)
///
/// Also used for **unbound** tables in a function body (value computed, not stored).
///
/// Data cells are expressions ([`parse_call_arg_value`]); headers stay key text.
fn consume_table(&mut self) -> Result<Expr> {
self.skip_noise();
+5
View File
@@ -132,6 +132,11 @@ fn structure_hello() {
assert_out("tests/structure/hello.mq.md", "Hello World!");
}
#[test]
fn structure_unbound_table() {
assert_out("tests/structure/unbound-table.mq.md", "ok");
}
#[test]
fn structure_object_handle() {
assert_out("tests/structure/object-handle.mq.md", "counter\n3\n4");
+15
View File
@@ -0,0 +1,15 @@
---
title: Unbound GFM table is a discarded value
---
# main
Documentation may include a table that is not bound to a name. It is still a
collection value; evaluating it must not error.
| Method | Notes |
|--------|-------|
| PPA | apt |
| Source | cargo |
> print text=ok