feat(regex): add host-backed groups, alternation and captures
This commit is contained in:
1 parent
94a13880c4
commit
28b178fc70
16 files changed
+289
-8
No files matched your search
+4
-2
@@ -58,7 +58,7 @@ byte-identical).
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
490 passed, 0 failed, 2 ignored; includes 7/7 Linux security tests
|
||||
491 passed, 0 failed, 2 ignored; includes 7/7 Linux security tests
|
||||
```
|
||||
|
||||
The documentation and test migration is local work; re-run this block after it
|
||||
@@ -209,7 +209,9 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
runs an exact `--allow-process PROGRAM` grant with 5 s timeout and 1 MiB caps
|
||||
(ADR-0044, not an OS sandbox); `random.字节` reads explicitly granted
|
||||
`random.bytes` entropy (ADR-0046). File watch, binary offset reads, streaming
|
||||
incremental JSON/CSV streaming and regex groups/alternation remain open.
|
||||
incremental JSON/CSV streaming, pure-mode regex groups/alternation and regex
|
||||
look-around/backreferences remain open; project mode has host-backed groups
|
||||
and captures (ADR-0050).
|
||||
- Bundled standard library (CM-A10, ADR-0028): `stdlib/` ships
|
||||
`format`/`text`/`io`/`fs`/`data`/`path`/`regex`/`datetime`/`clock`/`env`/`process`/`random`/`compress`; explicit `std:` imports resolve only inside the bundle;
|
||||
a SHA-256 digest over `*.mq.md` is verified before loading; Analysis records
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
# ADR-0050: Host-backed extended regular expressions
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-21
|
||||
Relates to: ADR-0041
|
||||
Specs: `spec/evaluation.md`
|
||||
Evidence: `docs/evidence/2026-11-21-regex-extension.md`
|
||||
|
||||
## Context
|
||||
|
||||
The pure `.mq.md` regex subset covers literals, `.`, quantifiers, anchors and
|
||||
character classes, but not groups, alternation or captures. Implementing a
|
||||
full backtracking engine in `.mq.md` would be a large, hard-to-audit
|
||||
duplicate. The host already has a safe, well-tested `regex` crate.
|
||||
|
||||
## Decision
|
||||
|
||||
Add project-mode operations:
|
||||
|
||||
```text
|
||||
resource.request("regex.match", [pattern, text]) -> {ok:true, value:Bool}
|
||||
resource.request("regex.captures", [pattern, text]) -> {ok:true, value:{matched:Bool, groups:List<Text|None>}}
|
||||
```
|
||||
|
||||
* Both require two Text arguments; invalid patterns are recoverable
|
||||
`regex.invalid`.
|
||||
* `captures` includes group 0 (the whole match); unmatched optional groups are
|
||||
`None`.
|
||||
* No new capability grant: regex compilation/matching is deterministic CPU
|
||||
work and does not disclose external state. All `resource.request` operations
|
||||
remain project-mode only, so `pure`/`strict` refuse them.
|
||||
* `stdlib/regex.mq.md` exposes `扩展匹配`, `扩展搜索` and `捕获`; the existing
|
||||
pure `匹配`/`搜索` subset stays unchanged for pure mode.
|
||||
* The `regex` crate is Rust-only, with no OS dependency. No new builtin,
|
||||
`HostOp` or parser node is added.
|
||||
|
||||
## Consequences
|
||||
|
||||
* Groups, alternation, `\d` classes and captures are available in project mode.
|
||||
* Pure mode still has only the documented subset; this is an explicit
|
||||
mode-dependent capability, not a silent downgrade.
|
||||
* Look-around, backreferences and regex captures in pure `.mq.md` remain open.
|
||||
@@ -18,7 +18,7 @@ platform".
|
||||
| Dimension | Current state | Source |
|
||||
|---|---|---|
|
||||
| Implemented | Rust workspace `0.1.0`: Markdown/frontend/IR, tree-walk runtime, `Memory`, capability host, Tool/Analysis/Trace, Plugin ABI, Foreign kernels, bundled stdlib, CLI composition root | `Cargo.toml`, `crates/`, `stdlib/` |
|
||||
| Tested locally | `cargo test --workspace --locked`: **490 passed, 0 failed, 2 ignored** on Linux x86_64 at the baseline | `docs/STATUS.md` §3 |
|
||||
| Tested locally | `cargo test --workspace --locked`: **491 passed, 0 failed, 2 ignored** on Linux x86_64 at the baseline | `docs/STATUS.md` §3 |
|
||||
| Bundled stdlib released | In-repo bundle `marqdo-stdlib@0.1.0`, SHA-256 digest verified before loading; it is **not** an external registry package | `stdlib/stdlib.json`, `stdlib/README.md` |
|
||||
| Multi-platform verified | Linux local gate is the only executed lane here; Windows/macOS native lanes are **UNVERIFIED**, and the GitHub security lane remains a **VM-only** gate | `docs/STATUS.md` §1, §4 |
|
||||
|
||||
@@ -95,7 +95,7 @@ review RFC, not a new crate per provider.
|
||||
| `std:data/json` | `json.解析/解析多个/格式化`; nested Value, 128-depth bound, recoverable errors | implemented (recursive, multi-value; no streaming) | `stdlib_product` |
|
||||
| `std:data/csv` | `csv.解析行/解析/格式化行/格式化`; quoted single/multi-row, CRLF, quoted newlines | implemented (no streaming) | `stdlib_product` |
|
||||
| `std:path` | `path.连接/文件名/目录名/扩展名/规范/是否绝对`; lexical only | implemented (no I/O) | `stdlib_product` |
|
||||
| `std:regex` | `regex.匹配/搜索` with `^ $ . * + ? [...]` escapes | implemented subset; no groups/alternation | `stdlib_product` |
|
||||
| `std:regex` | pure `匹配/搜索` subset; host `扩展匹配/捕获` with groups/alternation/captures | implemented both paths; pure path has no groups | `resource_streams`, `stdlib_product` |
|
||||
| `std:datetime` | `datetime.格式化(毫秒)` / `datetime.解析(文本)`; UTC ISO-8601, pre-1970 supported | implemented (pure) | `stdlib_product` |
|
||||
| `std:clock` | `clock.现在()` -> `resource.request("clock.now", [])` | implemented (project mode; pure/strict denied) | `resource_streams`, `stdlib_product` |
|
||||
| `std:env` | `env.读取/存在` -> `resource.request("env.get", [name])`; exact `--allow-env` grant | implemented (project mode; no `env.list`) | `resource_streams`, `stdlib_product` |
|
||||
@@ -134,7 +134,7 @@ that a phase is complete.
|
||||
| P1 | CLI split into `commands/{sandbox,run,check,inspect,init,package,tool}.rs`; host split into `host/{resource_ops,providers,effects,dispatch}.rs` plus `process.rs`; `commands.rs` keeps shared project/runtime helpers and `host.rs` keeps the `Host` trait methods and capability checks | split; shared helpers and trait methods remain by design |
|
||||
| P2 | HTTP parsing is Unicode-codepoint oriented; a general byte body is not the only path | open; convert before a general network release |
|
||||
| P2 | DNS, UDP, TCP listener and TLS client endpoints are implemented; no TLS server/mTLS or custom-CA CLI | open |
|
||||
| P2 | `std:path`, regex subset, `std:datetime`, `std:clock`, `std:env`, `std:process`, `std:random` and gzip compression are implemented; no streaming/zstd/brotli | open |
|
||||
| P2 | `std:path`, pure+host regex, `std:datetime`, `std:clock`, `std:env`, `std:process`, `std:random` and gzip compression are implemented; no streaming/zstd/brotli | open |
|
||||
| P2 | Memory limits: 16 MiB per region, 64 MiB per run | current budget, not a product limit |
|
||||
| P3 | No concurrency tasks, atomics or cancellable long-running scheduling | open; separate ADR |
|
||||
| P3 | No native optimizer/JIT or columnar numeric backend | open; benchmark-first |
|
||||
@@ -173,5 +173,6 @@ that a phase is complete.
|
||||
- [ADR-0047: DNS/UDP/TCP listener](../adr/0047-net-dns-udp-listener.md) — accepted project-mode network endpoints.
|
||||
- [ADR-0048: TLS client endpoint](../adr/0048-tls-resource-endpoint.md) — accepted validated rustls client; system + embedder DER roots.
|
||||
- [ADR-0049: gzip compression](../adr/0049-compression-resource-operations.md) — accepted bounded gzip byte-list codec.
|
||||
- [ADR-0050: host regex extension](../adr/0050-host-regex-extension.md) — accepted project-mode groups/alternation/captures.
|
||||
- [Compute-machine RFC-0001](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) and [core/stdlib RFC-0002](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) — design input.
|
||||
- [Architecture audit and refactor RFC](Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md) — review input for this migration.
|
||||
@@ -54,6 +54,7 @@ The implemented first slices of these RFCs are recorded in `CURRENT.md` and
|
||||
| DNS/UDP/TCP-listener endpoints | [0047](../adr/0047-net-dns-udp-listener.md) | Accepted (project only, exact host grant, no TLS) |
|
||||
| TLS client endpoint | [0048](../adr/0048-tls-resource-endpoint.md) | Accepted (rustls, validated, project only) |
|
||||
| gzip compression operations | [0049](../adr/0049-compression-resource-operations.md) | Accepted (bounded byte-list gzip) |
|
||||
| Host regex extension | [0050](../adr/0050-host-regex-extension.md) | Accepted (project-mode groups/alternation/captures) |
|
||||
| Table physical storage (T1/T2/T3), strict stdin/stderr/file/network resources | — | **Not decided**; do not implement by implication |
|
||||
|
||||
## Evidence and status
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
# Evidence: host-backed regex extension
|
||||
|
||||
Date: 2026-11-21 (Asia/Shanghai)
|
||||
Source baseline: `94a13880c417739d1d8cc270097c376ef69d87df`
|
||||
Repository: `marqdo-cli`
|
||||
Decision record: [ADR-0050](../adr/0050-host-regex-extension.md)
|
||||
|
||||
## Implemented
|
||||
|
||||
- `resource.request("regex.match", [pattern, text])` -> Bool.
|
||||
- `resource.request("regex.captures", [pattern, text])` -> `{matched, groups}`
|
||||
with group 0 and `None` for unmatched groups.
|
||||
- Invalid patterns are recoverable `regex.invalid`; no new grant; project mode
|
||||
only.
|
||||
- `stdlib/regex.mq.md` exposes `扩展匹配`, `扩展搜索`, `捕获`; pure
|
||||
`匹配`/`搜索` subset unchanged.
|
||||
|
||||
## Tests
|
||||
|
||||
`stdlib/tests/regex.mq.md` covers `^(a+)(b|c)$` matching/captures.
|
||||
`crates/marqdo-host/tests/resource_streams.rs` pins groups/alternation,
|
||||
captures and invalid-pattern recovery (34 host tests).
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace 491 passed, 0 failed, 2 ignored
|
||||
cargo test -p marqdo-host --test resource_streams 34 passed
|
||||
cargo test -p marqdo-cli --test stdlib_product 16 passed
|
||||
python3 tools/pack-stdlib.py PASS
|
||||
```
|
||||
|
||||
## Not covered
|
||||
|
||||
No pure-mode groups/alternation/captures, look-around, backreferences or
|
||||
regex replace/split. The host engine is only reachable in project mode.
|
||||
Reference in new issue
Block a user