docs(release): add HK Paramiko jump for GitHub push recovery
Document the proven hk.cflmy.de CONNECT path and ship push-via-hk-jump.py so flaky Clash/reverse-proxy no longer blocks releases. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
1 parent
237962c3be
commit
a0e88919a6
3 files changed
+333
-9
No files matched your search
@@ -6,8 +6,9 @@ description: >-
|
||||
sync VS Code extension on branch vscode-extension when needed, build or
|
||||
trigger install assets (CLI, stdlib, ext, public zip, VSIX, wasm notes), push
|
||||
tag, publish GitHub Release with detailed notes, and recover from network
|
||||
failures via proxy. Use when the user asks to release, cut a version, publish
|
||||
vX.Y.Z, ship a GitHub release, 发版, 发布新版本, or tag marqdo.
|
||||
failures via proxy or HK SSH jump (scripts/push-via-hk-jump.py). Use when the
|
||||
user asks to release, cut a version, publish vX.Y.Z, ship a GitHub release,
|
||||
发版, 发布新版本, or tag marqdo.
|
||||
---
|
||||
|
||||
# Marqdo release
|
||||
@@ -22,7 +23,7 @@ Canonical release playbook for **cflmy/marqdo**. Read this skill **before** tagg
|
||||
4. **Do not release from a dirty tree** (except intentional release commits you create in this flow).
|
||||
5. **Tag format** is always `vX.Y.Z` matching root `Cargo.toml` `version = "X.Y.Z"`.
|
||||
6. Prefer **tag push → GitHub Actions** (`.github/workflows/release.yml`) for Windows CLI + VSIX + zips. Local packaging is fallback / verification.
|
||||
7. After network errors: apply [reference.md § Proxy](reference.md); retry; do not silently skip uploads.
|
||||
7. After network errors: apply [reference.md § Proxy](reference.md); retry; do not silently skip uploads. If GitHub stays unreachable but **HK jump** (`hk.cflmy.de`) works, use `scripts/push-via-hk-jump.py` (Paramiko CONNECT) — see reference § **HK SSH jump**.
|
||||
|
||||
## Defaults (after version is confirmed)
|
||||
|
||||
@@ -105,7 +106,7 @@ release: vVER — <one-line highlight>
|
||||
EOF
|
||||
```
|
||||
|
||||
Push `main`: `git push origin main` (with proxy/`all` perms if needed).
|
||||
Push `main`: `git push origin main` (with proxy/`all` perms if needed; on persistent GitHub failure use [reference § HK SSH jump](reference.md)).
|
||||
|
||||
## Phase 3 — VS Code / Cursor extension
|
||||
|
||||
@@ -152,7 +153,8 @@ Local Windows fallback: `scripts/release-full.ps1 -Tag TAG -Upload` (see script
|
||||
|
||||
| Failure | Action |
|
||||
|---------|--------|
|
||||
| `gh` / git TLS or proxy `7890` refused | Unset bad proxy; use working `HTTP(S)_PROXY` or `required_permissions: ["all"]`; see reference |
|
||||
| `gh` / git TLS or Clash `7890` refused | Unset bad proxy; `required_permissions: ["all"]`; see [reference § Proxy](reference.md) |
|
||||
| GitHub blocked / `proxy.cflmy.top` push timeout | **HK SSH jump**: `scripts/push-via-hk-jump.py` (ask user for HK password/key; never commit). See reference § **HK SSH jump** |
|
||||
| Tag exists | Stop; ask user to bump or delete tag (no force on shared tags without explicit order) |
|
||||
| CI red | Fix on main, move tag only if user explicitly allows delete+re-push tag |
|
||||
| VSIX missing | Fetch `vscode-extension`, build locally, `gh release upload TAG dist/*.vsix` |
|
||||
|
||||
@@ -206,14 +206,16 @@ marqdo wasm build
|
||||
|
||||
## Proxy & auth (network recovery)
|
||||
|
||||
Symptoms seen in this project: broken local proxy on `127.0.0.1:7890`, TLS failures, `gh` needing token, Cursor sandbox blocking git/gh.
|
||||
Symptoms seen in this project: broken local Clash on `127.0.0.1:7890`, flaky `proxy.cflmy.top` HTTPS (short GET OK, long `git push` / connect timeouts), TLS failures, `gh` needing token, Cursor agent sandbox with only `lo`.
|
||||
|
||||
### Diagnose
|
||||
|
||||
```bash
|
||||
env | grep -iE 'proxy|PROXY' || true
|
||||
curl -sI https://github.com | head -5
|
||||
curl -sI --max-time 15 https://github.com | head -5
|
||||
curl -sS --max-time 15 -o /dev/null -w 'proxy=%{http_code}\n' https://proxy.cflmy.top/ || true
|
||||
gh auth status
|
||||
ip -br addr | head -5 # agent may lack enp*; use required_permissions: ["all"]
|
||||
```
|
||||
|
||||
### Fix order
|
||||
@@ -222,6 +224,7 @@ gh auth status
|
||||
|
||||
```bash
|
||||
unset http_proxy https_proxy HTTP_PROXY HTTPS_PROXY ALL_PROXY all_proxy
|
||||
# also clear for one git call: git -c http.proxy= -c https.proxy= …
|
||||
```
|
||||
|
||||
2. **Set a working proxy** only if the user provides one (or known working):
|
||||
@@ -237,13 +240,15 @@ export HTTP_PROXY=http://HOST:PORT
|
||||
|
||||
5. **Git remote**: prefer `https://github.com/cflmy/marqdo.git`; SSH needs keys.
|
||||
|
||||
6. **Partial upload**:
|
||||
6. **HK SSH jump (preferred when GitHub is blocked but HK is reachable)** — see next subsection.
|
||||
|
||||
7. **Partial upload**:
|
||||
|
||||
```bash
|
||||
gh release upload "vVER" dist/FILE --clobber
|
||||
```
|
||||
|
||||
7. **Re-run CI** without retagging (if tag already correct):
|
||||
8. **Re-run CI** without retagging (if tag already correct):
|
||||
|
||||
```bash
|
||||
gh workflow run Release --ref "vVER" # only if workflow_dispatch exists; else fix + new patch tag
|
||||
@@ -251,6 +256,43 @@ gh workflow run Release --ref "vVER" # only if workflow_dispatch exists; else f
|
||||
|
||||
Current Release workflow is **tag-push only**. Red CI after tag → fix on `main`, then either new patch tag or **user-approved** delete+recreate tag (dangerous; prefer patch).
|
||||
|
||||
### HK SSH jump (Paramiko CONNECT)
|
||||
|
||||
**When to use:** local → GitHub fails (timeout / TLS / Clash flaky / reverse-proxy `proxy.cflmy.top` connect drops), but this host can reach an overseas box that has stable GitHub (default `hk.cflmy.de`). Proven path for v0.3.9 push/tag/`gh`.
|
||||
|
||||
**Do not** rely on OpenSSH + `sshpass` here — password auth has hung; use **Paramiko** (`python3-paramiko`).
|
||||
|
||||
**Never** commit the HK root password or tokens. Ask the user for the password (or a key), put it in env / a `600` temp file, scrub after.
|
||||
|
||||
```bash
|
||||
# one-shot password file (example — use the secret the user gave this session)
|
||||
printf '%s' "$HK_PASSWORD_FROM_USER" > /tmp/.hk_ssh_pw && chmod 600 /tmp/.hk_ssh_pw
|
||||
|
||||
export HK_SSH_HOST=hk.cflmy.de # or 83.229.126.119
|
||||
export HK_SSH_USER=root
|
||||
export HK_SSH_PASSWORD_FILE=/tmp/.hk_ssh_pw
|
||||
# optional: HK_JUMP_PORT=18081
|
||||
|
||||
SCRIPT=.cursor/skills/marqdo-release/scripts/push-via-hk-jump.py
|
||||
|
||||
# push / tag / extension (git gets CONNECT proxy + github.com askpass)
|
||||
python3 "$SCRIPT" -- git push origin main
|
||||
python3 "$SCRIPT" -- git push origin vscode-extension
|
||||
python3 "$SCRIPT" -- git push origin vVER
|
||||
|
||||
# gh (sets GH_TOKEN from git credential fill if unset)
|
||||
python3 "$SCRIPT" -- gh release edit vVER --notes-file /tmp/notes.md
|
||||
python3 "$SCRIPT" -- gh run list --workflow=Release --limit 3
|
||||
|
||||
# leave proxy up for manual commands
|
||||
python3 "$SCRIPT" --serve # then: export https_proxy=http://127.0.0.1:18081
|
||||
|
||||
rm -f /tmp/.hk_ssh_pw
|
||||
```
|
||||
|
||||
Script behavior: SSH to HK → local `http://127.0.0.1:$HK_JUMP_PORT` HTTP CONNECT via `direct-tcpip` → run `git`/`gh`/`curl` with that proxy. GitHub HTTPS credentials still come from the **local** `git credential` store (askpass), not from the jump host.
|
||||
|
||||
Optional path reverse-proxy (`https://proxy.cflmy.top/github.com/…` `insteadOf`) may work for short GETs; treat long `git push` failures as a signal to use the HK jump instead of retrying Clash forever.
|
||||
## Local Windows fallback
|
||||
|
||||
```powershell
|
||||
|
||||
+280
@@ -0,0 +1,280 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Push / run git|gh through an HK SSH jump host (Paramiko HTTP CONNECT).
|
||||
|
||||
When this machine cannot reach GitHub reliably but can SSH to a host that can
|
||||
(e.g. hk.cflmy.de), open a local HTTP CONNECT proxy over Paramiko and run the
|
||||
remaining command with http(s).proxy pointed at it.
|
||||
|
||||
Why Paramiko: OpenSSH/`sshpass` has hung on password auth in this environment;
|
||||
Paramiko password + direct-tcpip works.
|
||||
|
||||
Credentials (never commit):
|
||||
HK_SSH_PASSWORD password string
|
||||
HK_SSH_PASSWORD_FILE file containing password (mode 600 recommended)
|
||||
or SSH key via Paramiko agent/keys if password unset and key works
|
||||
|
||||
Optional env:
|
||||
HK_SSH_HOST default hk.cflmy.de
|
||||
HK_SSH_USER default root
|
||||
HK_JUMP_PORT default 18081 (local CONNECT listen)
|
||||
|
||||
Usage:
|
||||
HK_SSH_PASSWORD_FILE=/tmp/hk.pw \\
|
||||
python3 .cursor/skills/marqdo-release/scripts/push-via-hk-jump.py -- \\
|
||||
git push origin main
|
||||
|
||||
# keep proxy only (Ctrl+C to stop)
|
||||
... push-via-hk-jump.py --serve
|
||||
|
||||
# wrap gh
|
||||
... push-via-hk-jump.py -- gh release view v0.3.9
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import select
|
||||
import socket
|
||||
import socketserver
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
import paramiko
|
||||
except ImportError as e: # pragma: no cover
|
||||
raise SystemExit(
|
||||
"paramiko required: apt install python3-paramiko (or pip install paramiko)"
|
||||
) from e
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[4]
|
||||
|
||||
|
||||
class _Server(socketserver.ThreadingMixIn, socketserver.TCPServer):
|
||||
allow_reuse_address = True
|
||||
daemon_threads = True
|
||||
|
||||
|
||||
def load_password() -> str | None:
|
||||
path = os.environ.get("HK_SSH_PASSWORD_FILE")
|
||||
if path:
|
||||
return Path(path).read_text(encoding="utf-8").strip()
|
||||
pw = os.environ.get("HK_SSH_PASSWORD")
|
||||
return pw.strip() if pw else None
|
||||
|
||||
|
||||
def connect_hk() -> paramiko.SSHClient:
|
||||
host = os.environ.get("HK_SSH_HOST", "hk.cflmy.de")
|
||||
user = os.environ.get("HK_SSH_USER", "root")
|
||||
pw = load_password()
|
||||
client = paramiko.SSHClient()
|
||||
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||
kwargs: dict = {
|
||||
"hostname": host,
|
||||
"username": user,
|
||||
"timeout": 20,
|
||||
"auth_timeout": 25,
|
||||
"banner_timeout": 25,
|
||||
}
|
||||
if pw:
|
||||
kwargs.update(password=pw, allow_agent=False, look_for_keys=False)
|
||||
else:
|
||||
kwargs.update(allow_agent=True, look_for_keys=True)
|
||||
print(f"== HK jump {user}@{host} ==", flush=True)
|
||||
client.connect(**kwargs)
|
||||
_, stdout, stderr = client.exec_command(
|
||||
"hostname; curl -sS --max-time 20 -o /dev/null -w gh=%{http_code} https://github.com/; echo"
|
||||
)
|
||||
out = stdout.read().decode()
|
||||
err = stderr.read().decode()
|
||||
print(out, end="" if out.endswith("\n") else "\n", flush=True)
|
||||
if err.strip():
|
||||
print(err, file=sys.stderr, flush=True)
|
||||
return client
|
||||
|
||||
|
||||
def make_proxy(transport: paramiko.Transport, port: int) -> _Server:
|
||||
class Handler(socketserver.StreamRequestHandler):
|
||||
def handle(self) -> None:
|
||||
try:
|
||||
first = self.rfile.readline(65536)
|
||||
if not first:
|
||||
return
|
||||
parts = first.decode("latin1", "replace").split()
|
||||
if len(parts) < 2 or parts[0].upper() != "CONNECT":
|
||||
self.wfile.write(b"HTTP/1.1 405 Method Not Allowed\r\n\r\n")
|
||||
return
|
||||
while True:
|
||||
line = self.rfile.readline(65536)
|
||||
if line in (b"\r\n", b"\n", b""):
|
||||
break
|
||||
host, _, port_s = parts[1].partition(":")
|
||||
chan = transport.open_channel(
|
||||
"direct-tcpip",
|
||||
(host, int(port_s or "443")),
|
||||
self.connection.getpeername(),
|
||||
)
|
||||
self.wfile.write(b"HTTP/1.1 200 Connection Established\r\n\r\n")
|
||||
sock = self.connection
|
||||
sock.setblocking(False)
|
||||
chan.setblocking(False)
|
||||
try:
|
||||
while True:
|
||||
r, _, x = select.select([sock, chan], [], [sock, chan], 120)
|
||||
if x:
|
||||
break
|
||||
if sock in r:
|
||||
data = sock.recv(65536)
|
||||
if not data:
|
||||
break
|
||||
chan.sendall(data)
|
||||
if chan in r:
|
||||
data = chan.recv(65536)
|
||||
if not data:
|
||||
break
|
||||
sock.sendall(data)
|
||||
finally:
|
||||
try:
|
||||
chan.close()
|
||||
except Exception:
|
||||
pass
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return _Server(("127.0.0.1", port), Handler)
|
||||
|
||||
|
||||
def github_askpass() -> Path:
|
||||
"""Write a one-shot GIT_ASKPASS that answers with github.com store creds."""
|
||||
out = subprocess.check_output(
|
||||
["git", "credential", "fill"],
|
||||
input=b"protocol=https\nhost=github.com\n\n",
|
||||
).decode()
|
||||
data = dict(line.split("=", 1) for line in out.splitlines() if "=" in line)
|
||||
user = data.get("username") or "x-access-token"
|
||||
password = data.get("password") or ""
|
||||
if not password:
|
||||
raise SystemExit("no github.com credential in git credential store")
|
||||
ask = Path("/tmp/marqdo-git-askpass-hk.py")
|
||||
ask.write_text(
|
||||
"#!/usr/bin/env python3\n"
|
||||
"import sys\n"
|
||||
f"u={user!r}; p={password!r}\n"
|
||||
"print(u if 'username' in sys.argv[-1].lower() else p)\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
ask.chmod(0o700)
|
||||
return ask
|
||||
|
||||
|
||||
def run_with_proxy(proxy: str, argv: list[str]) -> int:
|
||||
env = os.environ.copy()
|
||||
for k in (
|
||||
"http_proxy",
|
||||
"https_proxy",
|
||||
"HTTP_PROXY",
|
||||
"HTTPS_PROXY",
|
||||
"ALL_PROXY",
|
||||
"all_proxy",
|
||||
):
|
||||
env.pop(k, None)
|
||||
env["http_proxy"] = proxy
|
||||
env["https_proxy"] = proxy
|
||||
env["HTTP_PROXY"] = proxy
|
||||
env["HTTPS_PROXY"] = proxy
|
||||
|
||||
cmd = list(argv)
|
||||
if cmd and cmd[0] == "git":
|
||||
ask = github_askpass()
|
||||
env["GIT_ASKPASS"] = str(ask)
|
||||
env["GIT_TERMINAL_PROMPT"] = "0"
|
||||
# Force this CONNECT proxy; clear gitconfig Clash 7890 for the session.
|
||||
cmd = [
|
||||
"git",
|
||||
"-c",
|
||||
f"http.proxy={proxy}",
|
||||
"-c",
|
||||
f"https.proxy={proxy}",
|
||||
"-c",
|
||||
"credential.helper=",
|
||||
"-c",
|
||||
"http.version=HTTP/1.1",
|
||||
"-c",
|
||||
"http.postBuffer=524288000",
|
||||
*cmd[1:],
|
||||
]
|
||||
try:
|
||||
print("+", " ".join(cmd), flush=True)
|
||||
return subprocess.call(cmd, cwd=ROOT, env=env)
|
||||
finally:
|
||||
ask.unlink(missing_ok=True)
|
||||
|
||||
if cmd and cmd[0] == "gh" and not env.get("GH_TOKEN"):
|
||||
# session token from git credential store
|
||||
out = subprocess.check_output(
|
||||
["git", "credential", "fill"],
|
||||
input=b"protocol=https\nhost=github.com\n\n",
|
||||
).decode()
|
||||
data = dict(line.split("=", 1) for line in out.splitlines() if "=" in line)
|
||||
if data.get("password"):
|
||||
env["GH_TOKEN"] = data["password"]
|
||||
|
||||
print("+", " ".join(cmd), flush=True)
|
||||
return subprocess.call(cmd, cwd=ROOT, env=env)
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
args = argv[1:]
|
||||
if not args or args[0] in ("-h", "--help"):
|
||||
print(__doc__)
|
||||
return 0 if args else 1
|
||||
|
||||
serve_only = args == ["--serve"]
|
||||
if args[0] == "--":
|
||||
args = args[1:]
|
||||
if not serve_only and not args:
|
||||
print("missing command after --", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
port = int(os.environ.get("HK_JUMP_PORT", "18081"))
|
||||
client = connect_hk()
|
||||
transport = client.get_transport()
|
||||
assert transport is not None
|
||||
server = make_proxy(transport, port)
|
||||
threading.Thread(target=server.serve_forever, daemon=True).start()
|
||||
proxy = f"http://127.0.0.1:{port}"
|
||||
print(f"== CONNECT proxy {proxy} ==", flush=True)
|
||||
# smoke
|
||||
subprocess.call(
|
||||
[
|
||||
"curl",
|
||||
"-sS",
|
||||
"--connect-timeout",
|
||||
"10",
|
||||
"--max-time",
|
||||
"25",
|
||||
"--proxy",
|
||||
proxy,
|
||||
"-o",
|
||||
"/dev/null",
|
||||
"-w",
|
||||
"github=%{http_code}\\n",
|
||||
"https://github.com/",
|
||||
]
|
||||
)
|
||||
|
||||
try:
|
||||
if serve_only:
|
||||
print("serving; Ctrl+C to stop", flush=True)
|
||||
while transport.is_active():
|
||||
time.sleep(5)
|
||||
return 1
|
||||
return run_with_proxy(proxy, args)
|
||||
finally:
|
||||
server.shutdown()
|
||||
client.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main(sys.argv))
|
||||
Reference in new issue
Block a user