feat(net): add DNS, UDP, TCP listener and TLS endpoints
This commit is contained in:
1 parent
ee0c6cc80a
commit
ff3f19a2ea
20 files changed
+1327
-61
No files matched your search
+7
-6
@@ -58,7 +58,7 @@ byte-identical).
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
481 passed, 0 failed, 2 ignored; includes 7/7 Linux security tests
|
||||
488 passed, 0 failed, 2 ignored; includes 7/7 Linux security tests
|
||||
```
|
||||
|
||||
The documentation and test migration is local work; re-run this block after it
|
||||
@@ -267,11 +267,12 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
4. `fs.write`, `net.request`, `process`, `clock`, `random` are not executed
|
||||
broker services; unsupported requests must return `capability.unavailable`.
|
||||
5. Resource streams cover project-file read/write (including `std:fs/file`
|
||||
read/write/copy), non-streaming `resource.request` `fs.*` metadata/directory
|
||||
operations, stdout, compatible print, trusted in-process providers and
|
||||
IP-literal TCP; append/seek/watch, process, device, DNS/TLS/HTTP, UDP and
|
||||
provider packaging remain open. Strict permits only stdio and refuses the
|
||||
rest (CM-A11 lane PASS).
|
||||
read/write/copy/append/read_at/walk), non-streaming `resource.request`
|
||||
`fs.*` metadata/directory operations, stdout, compatible print, trusted
|
||||
in-process providers and TCP/UDP/listener/TLS endpoints with DNS
|
||||
resolution after an exact host grant. File watch, device and provider
|
||||
packaging remain open. Strict permits only stdio and refuses the rest (CM-A11 lane
|
||||
PASS).
|
||||
6. uv/npm/venv provisioning is not implemented. Only a pre-provisioned,
|
||||
digest-pinned directory with a regular-file executable is supported.
|
||||
7. No Windows/macOS RC archive exists from this host. The release-candidate
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
# ADR-0047: DNS resolution, UDP and TCP listener endpoints
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-21
|
||||
Relates to: ADR-0034, ADR-0041
|
||||
Specs: `spec/security.md`, `spec/evaluation.md`
|
||||
Evidence: `docs/evidence/2026-11-21-net-endpoints.md`
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0034 opened only an outbound IP-literal TCP stream. That left DNS names,
|
||||
datagrams and inbound listeners unavailable, and the `--allow-net` grant was
|
||||
documented as IP-only. The audit RFC's P2 network gap requires real endpoints,
|
||||
not a `TODO`.
|
||||
|
||||
## Decision
|
||||
|
||||
* `resource.open("tcp", "host:port", ["read","write"])` now accepts a DNS name
|
||||
or IP literal. The **host** (not the resolved IP) is checked against the
|
||||
exact `--allow-net HOST` grant **before** `ToSocketAddrs`; every resolved
|
||||
address is tried with a 5 s connect timeout.
|
||||
* Add `resource.open("udp", "host:port", ["read","write"])`: a connected
|
||||
`UdpSocket` from an ephemeral local port. `resource.read` returns one
|
||||
datagram (truncated to the destination `Memory` bound, `eof=false`);
|
||||
`resource.write` sends one datagram.
|
||||
* Add `resource.open("tcp-listener", "host:port", ["accept"])` plus the
|
||||
`resource.accept(listener)` builtin. `accept` polls a non-blocking listener
|
||||
for up to 5 s, returns a new TCP `Resource` in `{ok:true, value:Resource}`,
|
||||
or `{ok:false, code:"tcp.timeout"}`/`tcp.accept` on failure.
|
||||
* All three are project-mode only. `pure`/`strict` return `permission.denied`;
|
||||
the listener still requires the exact host grant.
|
||||
* Host names are accepted but **not** pinned to an IP: a DNS answer may change
|
||||
between grant and connect. No TLS, certificate validation, server sockets
|
||||
outside project mode or UDP broadcast/multicast is added.
|
||||
* The grant flag remains `--allow-net HOST` (repeatable); its help text now says
|
||||
host name or IP literal.
|
||||
|
||||
## Consequences
|
||||
|
||||
* DNS, UDP and inbound TCP are reachable from `.mq.md` without a new builtin
|
||||
per transport. `resource.accept` is the only new builtin and returns an
|
||||
opaque handle like `resource.open`.
|
||||
* TLS still needs a separate decision (certificate validation and trust
|
||||
anchors); this ADR does not claim TLS.
|
||||
* `resource.read` on UDP is datagram-based; a zero-length datagram is a
|
||||
successful empty read, and there is no stream EOF.
|
||||
@@ -0,0 +1,39 @@
|
||||
# ADR-0048: TLS client resource endpoint
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-21
|
||||
Relates to: ADR-0034, ADR-0047
|
||||
Specs: `spec/security.md`, `spec/evaluation.md`
|
||||
Evidence: `docs/evidence/2026-11-21-tls-endpoint.md`
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0047 added DNS, UDP and TCP listeners but deliberately left TLS out: a TLS
|
||||
client needs a trust-anchor policy and certificate validation, not just a
|
||||
socket. The audit RFC lists TLS as a P2 provider gap.
|
||||
|
||||
## Decision
|
||||
|
||||
* `resource.open("tls", "host:port", ["read","write"])` connects with TCP,
|
||||
validates the server certificate with `rustls`, and returns an opaque
|
||||
`tls` Resource. `resource.read`/`write`/`close` work like TCP.
|
||||
* The exact host grant (`--allow-net HOST`) is checked before DNS resolution.
|
||||
`pure`/`strict` refuse; only project mode is supported.
|
||||
* Trust roots are the platform native roots plus any DER certificates added by
|
||||
an embedder through `StdHost::with_tls_ca`. There is no CLI flag for a private
|
||||
CA in this build; the API is for embedders/tests.
|
||||
* Server-name verification uses the descriptor host, not the resolved IP.
|
||||
Certificate or handshake failures are fatal `host.error`; a failed handshake
|
||||
closes the connection and does not return a Resource.
|
||||
* TLS is client-only. No server TLS acceptor, mutual TLS, ALPN, session
|
||||
resumption, OCSP or custom root CLI is added.
|
||||
* `rustls` is used instead of `native-tls` so the build does not require
|
||||
system OpenSSL headers; it remains a normal Rust dependency.
|
||||
|
||||
## Consequences
|
||||
|
||||
* `resource.open("tls")` is a real validated TLS client endpoint and is covered
|
||||
by a local self-signed test that adds an explicit DER root.
|
||||
* A future CLI `--tls-ca` or provider-managed trust store needs a separate
|
||||
grant/ADR; this decision does not silently trust unknown certificates.
|
||||
* TLS server sockets and mutual TLS remain open.
|
||||
@@ -18,7 +18,7 @@ platform".
|
||||
| Dimension | Current state | Source |
|
||||
|---|---|---|
|
||||
| Implemented | Rust workspace `0.1.0`: Markdown/frontend/IR, tree-walk runtime, `Memory`, capability host, Tool/Analysis/Trace, Plugin ABI, Foreign kernels, bundled stdlib, CLI composition root | `Cargo.toml`, `crates/`, `stdlib/` |
|
||||
| Tested locally | `cargo test --workspace --locked`: **481 passed, 0 failed, 2 ignored** on Linux x86_64 at the baseline | `docs/STATUS.md` §3 |
|
||||
| Tested locally | `cargo test --workspace --locked`: **488 passed, 0 failed, 2 ignored** on Linux x86_64 at the baseline | `docs/STATUS.md` §3 |
|
||||
| Bundled stdlib released | In-repo bundle `marqdo-stdlib@0.1.0`, SHA-256 digest verified before loading; it is **not** an external registry package | `stdlib/stdlib.json`, `stdlib/README.md` |
|
||||
| Multi-platform verified | Linux local gate is the only executed lane here; Windows/macOS native lanes are **UNVERIFIED**, and the GitHub security lane remains a **VM-only** gate | `docs/STATUS.md` §1, §4 |
|
||||
|
||||
@@ -101,6 +101,7 @@ review RFC, not a new crate per provider.
|
||||
| `std:env` | `env.读取/存在` -> `resource.request("env.get", [name])`; exact `--allow-env` grant | implemented (project mode; no `env.list`) | `resource_streams`, `stdlib_product` |
|
||||
| `std:process` | `process.运行(程序, 参数列表)` -> `resource.request("process.run", ...)`; exact `--allow-process` grant, 5 s timeout, 1 MiB caps | implemented (project mode; not an OS sandbox) | `resource_streams`, `stdlib_product` |
|
||||
| `std:random` | `random.字节/整数` -> `resource.request("random.bytes", [count])`; exact `--allow-random` grant | implemented (project mode; byte-list RNG) | `resource_streams`, `stdlib_product` |
|
||||
| network endpoints | `tcp`/`udp`/`tcp-listener`/`tls` open + `resource.accept`; exact host grant, DNS after grant, TLS certificate validation | implemented (project mode) | `resource_streams` |
|
||||
| file metadata / directory / rename / remove / mkdir | not part of the current Resource contract | **not implemented**; needs a separate Resource Operation ADR | — |
|
||||
| TCP | `resource.open("tcp", ip:port, ["read","write"])`; exact IP grant, 5 s timeout | implemented; no DNS/TLS/server socket/UDP | `tcp_endpoint`, `http_over_tcp` |
|
||||
| HTTP/1.1 | `stdlib/net/http.mq.md`: status/header/chunked-byte parse, request build | implemented over an already-authorized TCP stream; not a DNS/TLS client | `stdlib_http`, `http_over_tcp`, `stdlib_product` |
|
||||
@@ -131,7 +132,7 @@ that a phase is complete.
|
||||
| P1 | JSON/CSV product parsers are implemented; no streaming/multi-line CSV or depth-bounded JSON walker | open |
|
||||
| P1 | CLI split into `commands/{sandbox,run,check,inspect,init,package,tool}.rs`; host split into `host/{resource_ops,providers,effects,dispatch}.rs` plus `process.rs`; `commands.rs` keeps shared project/runtime helpers and `host.rs` keeps the `Host` trait methods and capability checks | split; shared helpers and trait methods remain by design |
|
||||
| P2 | HTTP parsing is Unicode-codepoint oriented; a general byte body is not the only path | open; convert before a general network release |
|
||||
| P2 | No DNS, TLS, listener or UDP provider | open |
|
||||
| P2 | DNS, UDP, TCP listener and TLS client endpoints are implemented; no TLS server/mTLS or custom-CA CLI | open |
|
||||
| P2 | `std:path`, a regex subset, `std:datetime`, `std:clock`, `std:env`, `std:process` and `std:random` are implemented; no compression | open |
|
||||
| P2 | Memory limits: 16 MiB per region, 64 MiB per run | current budget, not a product limit |
|
||||
| P3 | No concurrency tasks, atomics or cancellable long-running scheduling | open; separate ADR |
|
||||
@@ -168,5 +169,7 @@ that a phase is complete.
|
||||
- [ADR-0044: `process.run` resource operation](../adr/0044-process-resource-operation.md) — accepted exact-path process grant; not a sandbox.
|
||||
- [ADR-0045: fs append/read_at/walk](../adr/0045-fs-append-read-at-walk.md) — accepted bounded content and directory walk.
|
||||
- [ADR-0046: `random.bytes`](../adr/0046-random-resource-operation.md) — accepted explicitly granted OS entropy.
|
||||
- [ADR-0047: DNS/UDP/TCP listener](../adr/0047-net-dns-udp-listener.md) — accepted project-mode network endpoints.
|
||||
- [ADR-0048: TLS client endpoint](../adr/0048-tls-resource-endpoint.md) — accepted validated rustls client; system + embedder DER roots.
|
||||
- [Compute-machine RFC-0001](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) and [core/stdlib RFC-0002](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) — design input.
|
||||
- [Architecture audit and refactor RFC](Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md) — review input for this migration.
|
||||
@@ -51,6 +51,8 @@ The implemented first slices of these RFCs are recorded in `CURRENT.md` and
|
||||
| `process.run` resource operation | [0044](../adr/0044-process-resource-operation.md) | Accepted (exact program grant, project only, not a sandbox) |
|
||||
| `fs.append`/`fs.read_at`/`fs.walk` | [0045](../adr/0045-fs-append-read-at-walk.md) | Accepted (bounded content, directory walk) |
|
||||
| `random.bytes` resource operation | [0046](../adr/0046-random-resource-operation.md) | Accepted (explicit `--allow-random`, project only) |
|
||||
| DNS/UDP/TCP-listener endpoints | [0047](../adr/0047-net-dns-udp-listener.md) | Accepted (project only, exact host grant, no TLS) |
|
||||
| TLS client endpoint | [0048](../adr/0048-tls-resource-endpoint.md) | Accepted (rustls, validated, project only) |
|
||||
| Table physical storage (T1/T2/T3), strict stdin/stderr/file/network resources | — | **Not decided**; do not implement by implication |
|
||||
|
||||
## Evidence and status
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
# Evidence: DNS, UDP and TCP listener endpoints
|
||||
|
||||
Date: 2026-11-21 (Asia/Shanghai)
|
||||
Source baseline: `ee0c6cc80a2782c4e88de63195eaf41a27077dcf`
|
||||
Repository: `marqdo-cli`
|
||||
Decision record: [ADR-0047](../adr/0047-net-dns-udp-listener.md)
|
||||
|
||||
## Implemented
|
||||
|
||||
- `resource.open("tcp", "host:port", ["read","write"])`: exact host grant is
|
||||
checked **before** `ToSocketAddrs`; every resolved address is tried with a
|
||||
5 s connect timeout.
|
||||
- `resource.open("udp", "host:port", ["read","write"])`: connected datagram
|
||||
socket from an ephemeral local port; `read` returns one datagram with
|
||||
`eof=false`; `write` sends one datagram.
|
||||
- `resource.open("tcp-listener", "host:port", ["accept"])` plus
|
||||
`resource.accept(listener)`: non-blocking 5 s poll, returns a new TCP
|
||||
`Resource` in `{ok:true,value:Resource}` or `tcp.timeout`.
|
||||
- All project-mode only; pure/strict refuse; no TLS.
|
||||
|
||||
## Tests
|
||||
|
||||
`crates/marqdo-host/tests/resource_streams.rs` (30 tests) covers TCP DNS
|
||||
round-trip via `localhost`, UDP datagram round-trip, listener accept/read/write,
|
||||
host-grant denial and project-only refusal.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace 486 passed, 0 failed, 2 ignored
|
||||
cargo test -p marqdo-host --test resource_streams 30 passed
|
||||
python3 tools/pack-stdlib.py PASS
|
||||
```
|
||||
|
||||
## Not covered
|
||||
|
||||
No TLS/certificate validation, no server sockets outside project mode, no UDP
|
||||
broadcast/multicast, no listener port discovery API, no connection pool and no
|
||||
DNS pinning. TLS remains a separate ADR.
|
||||
@@ -0,0 +1,39 @@
|
||||
# Evidence: TLS client endpoint
|
||||
|
||||
Date: 2026-11-21 (Asia/Shanghai)
|
||||
Source baseline: `ee0c6cc80a2782c4e88de63195eaf41a27077dcf`
|
||||
Repository: `marqdo-cli`
|
||||
Decision record: [ADR-0048](../adr/0048-tls-resource-endpoint.md)
|
||||
|
||||
## Implemented
|
||||
|
||||
`resource.open("tls", "host:port", ["read","write"])` is a project-mode
|
||||
validated TLS client:
|
||||
|
||||
- exact host grant before DNS;
|
||||
- TCP connect with 5 s timeout, then `rustls` handshake using platform native
|
||||
roots plus `StdHost::with_tls_ca` DER roots;
|
||||
- server name is the descriptor host; certificate/handshake failure is fatal
|
||||
`host.error`;
|
||||
- the resulting `tls` Resource supports `resource.read/write/close`;
|
||||
- no TLS server, mTLS, ALPN or custom-CA CLI.
|
||||
|
||||
## Tests
|
||||
|
||||
`crates/marqdo-host/tests/resource_streams.rs` starts a local `rustls` server
|
||||
with an `rcgen` self-signed certificate, adds that DER as an explicit root and
|
||||
round-trips a 4-byte payload. A second test pins project-only/host-grant
|
||||
refusal (32 host tests).
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace 488 passed, 0 failed, 2 ignored
|
||||
cargo test -p marqdo-host --test resource_streams 32 passed
|
||||
python3 tools/pack-stdlib.py PASS
|
||||
```
|
||||
|
||||
## Not covered
|
||||
|
||||
No TLS server/mTLS, no ALPN, no OCSP/CRL, no client certificates, no CLI
|
||||
private-CA flag and no session resumption.
|
||||
Reference in new issue
Block a user