feat(net): add DNS, UDP, TCP listener and TLS endpoints

This commit is contained in:
Marqdo Agent committed 2026-11-21 10:43:08 +08:00
1 parent ee0c6cc80a
commit ff3f19a2ea
20 files changed
+1327 -61

No files matched your search

+7 -6
View File
@@ -58,7 +58,7 @@ byte-identical).
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
481 passed, 0 failed, 2 ignored; includes 7/7 Linux security tests
488 passed, 0 failed, 2 ignored; includes 7/7 Linux security tests
```
The documentation and test migration is local work; re-run this block after it
@@ -267,11 +267,12 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
4. `fs.write`, `net.request`, `process`, `clock`, `random` are not executed
broker services; unsupported requests must return `capability.unavailable`.
5. Resource streams cover project-file read/write (including `std:fs/file`
read/write/copy), non-streaming `resource.request` `fs.*` metadata/directory
operations, stdout, compatible print, trusted in-process providers and
IP-literal TCP; append/seek/watch, process, device, DNS/TLS/HTTP, UDP and
provider packaging remain open. Strict permits only stdio and refuses the
rest (CM-A11 lane PASS).
read/write/copy/append/read_at/walk), non-streaming `resource.request`
`fs.*` metadata/directory operations, stdout, compatible print, trusted
in-process providers and TCP/UDP/listener/TLS endpoints with DNS
resolution after an exact host grant. File watch, device and provider
packaging remain open. Strict permits only stdio and refuses the rest (CM-A11 lane
PASS).
6. uv/npm/venv provisioning is not implemented. Only a pre-provisioned,
digest-pinned directory with a regular-file executable is supported.
7. No Windows/macOS RC archive exists from this host. The release-candidate
+46
View File
@@ -0,0 +1,46 @@
# ADR-0047: DNS resolution, UDP and TCP listener endpoints
Status: **Accepted**
Date: 2026-11-21
Relates to: ADR-0034, ADR-0041
Specs: `spec/security.md`, `spec/evaluation.md`
Evidence: `docs/evidence/2026-11-21-net-endpoints.md`
## Context
ADR-0034 opened only an outbound IP-literal TCP stream. That left DNS names,
datagrams and inbound listeners unavailable, and the `--allow-net` grant was
documented as IP-only. The audit RFC's P2 network gap requires real endpoints,
not a `TODO`.
## Decision
* `resource.open("tcp", "host:port", ["read","write"])` now accepts a DNS name
or IP literal. The **host** (not the resolved IP) is checked against the
exact `--allow-net HOST` grant **before** `ToSocketAddrs`; every resolved
address is tried with a 5 s connect timeout.
* Add `resource.open("udp", "host:port", ["read","write"])`: a connected
`UdpSocket` from an ephemeral local port. `resource.read` returns one
datagram (truncated to the destination `Memory` bound, `eof=false`);
`resource.write` sends one datagram.
* Add `resource.open("tcp-listener", "host:port", ["accept"])` plus the
`resource.accept(listener)` builtin. `accept` polls a non-blocking listener
for up to 5 s, returns a new TCP `Resource` in `{ok:true, value:Resource}`,
or `{ok:false, code:"tcp.timeout"}`/`tcp.accept` on failure.
* All three are project-mode only. `pure`/`strict` return `permission.denied`;
the listener still requires the exact host grant.
* Host names are accepted but **not** pinned to an IP: a DNS answer may change
between grant and connect. No TLS, certificate validation, server sockets
outside project mode or UDP broadcast/multicast is added.
* The grant flag remains `--allow-net HOST` (repeatable); its help text now says
host name or IP literal.
## Consequences
* DNS, UDP and inbound TCP are reachable from `.mq.md` without a new builtin
per transport. `resource.accept` is the only new builtin and returns an
opaque handle like `resource.open`.
* TLS still needs a separate decision (certificate validation and trust
anchors); this ADR does not claim TLS.
* `resource.read` on UDP is datagram-based; a zero-length datagram is a
successful empty read, and there is no stream EOF.
+39
View File
@@ -0,0 +1,39 @@
# ADR-0048: TLS client resource endpoint
Status: **Accepted**
Date: 2026-11-21
Relates to: ADR-0034, ADR-0047
Specs: `spec/security.md`, `spec/evaluation.md`
Evidence: `docs/evidence/2026-11-21-tls-endpoint.md`
## Context
ADR-0047 added DNS, UDP and TCP listeners but deliberately left TLS out: a TLS
client needs a trust-anchor policy and certificate validation, not just a
socket. The audit RFC lists TLS as a P2 provider gap.
## Decision
* `resource.open("tls", "host:port", ["read","write"])` connects with TCP,
validates the server certificate with `rustls`, and returns an opaque
`tls` Resource. `resource.read`/`write`/`close` work like TCP.
* The exact host grant (`--allow-net HOST`) is checked before DNS resolution.
`pure`/`strict` refuse; only project mode is supported.
* Trust roots are the platform native roots plus any DER certificates added by
an embedder through `StdHost::with_tls_ca`. There is no CLI flag for a private
CA in this build; the API is for embedders/tests.
* Server-name verification uses the descriptor host, not the resolved IP.
Certificate or handshake failures are fatal `host.error`; a failed handshake
closes the connection and does not return a Resource.
* TLS is client-only. No server TLS acceptor, mutual TLS, ALPN, session
resumption, OCSP or custom root CLI is added.
* `rustls` is used instead of `native-tls` so the build does not require
system OpenSSL headers; it remains a normal Rust dependency.
## Consequences
* `resource.open("tls")` is a real validated TLS client endpoint and is covered
by a local self-signed test that adds an explicit DER root.
* A future CLI `--tls-ca` or provider-managed trust store needs a separate
grant/ADR; this decision does not silently trust unknown certificates.
* TLS server sockets and mutual TLS remain open.
+5 -2
View File
@@ -18,7 +18,7 @@ platform".
| Dimension | Current state | Source |
|---|---|---|
| Implemented | Rust workspace `0.1.0`: Markdown/frontend/IR, tree-walk runtime, `Memory`, capability host, Tool/Analysis/Trace, Plugin ABI, Foreign kernels, bundled stdlib, CLI composition root | `Cargo.toml`, `crates/`, `stdlib/` |
| Tested locally | `cargo test --workspace --locked`: **481 passed, 0 failed, 2 ignored** on Linux x86_64 at the baseline | `docs/STATUS.md` §3 |
| Tested locally | `cargo test --workspace --locked`: **488 passed, 0 failed, 2 ignored** on Linux x86_64 at the baseline | `docs/STATUS.md` §3 |
| Bundled stdlib released | In-repo bundle `marqdo-stdlib@0.1.0`, SHA-256 digest verified before loading; it is **not** an external registry package | `stdlib/stdlib.json`, `stdlib/README.md` |
| Multi-platform verified | Linux local gate is the only executed lane here; Windows/macOS native lanes are **UNVERIFIED**, and the GitHub security lane remains a **VM-only** gate | `docs/STATUS.md` §1, §4 |
@@ -101,6 +101,7 @@ review RFC, not a new crate per provider.
| `std:env` | `env.读取/存在` -> `resource.request("env.get", [name])`; exact `--allow-env` grant | implemented (project mode; no `env.list`) | `resource_streams`, `stdlib_product` |
| `std:process` | `process.运行(程序, 参数列表)` -> `resource.request("process.run", ...)`; exact `--allow-process` grant, 5 s timeout, 1 MiB caps | implemented (project mode; not an OS sandbox) | `resource_streams`, `stdlib_product` |
| `std:random` | `random.字节/整数` -> `resource.request("random.bytes", [count])`; exact `--allow-random` grant | implemented (project mode; byte-list RNG) | `resource_streams`, `stdlib_product` |
| network endpoints | `tcp`/`udp`/`tcp-listener`/`tls` open + `resource.accept`; exact host grant, DNS after grant, TLS certificate validation | implemented (project mode) | `resource_streams` |
| file metadata / directory / rename / remove / mkdir | not part of the current Resource contract | **not implemented**; needs a separate Resource Operation ADR | — |
| TCP | `resource.open("tcp", ip:port, ["read","write"])`; exact IP grant, 5 s timeout | implemented; no DNS/TLS/server socket/UDP | `tcp_endpoint`, `http_over_tcp` |
| HTTP/1.1 | `stdlib/net/http.mq.md`: status/header/chunked-byte parse, request build | implemented over an already-authorized TCP stream; not a DNS/TLS client | `stdlib_http`, `http_over_tcp`, `stdlib_product` |
@@ -131,7 +132,7 @@ that a phase is complete.
| P1 | JSON/CSV product parsers are implemented; no streaming/multi-line CSV or depth-bounded JSON walker | open |
| P1 | CLI split into `commands/{sandbox,run,check,inspect,init,package,tool}.rs`; host split into `host/{resource_ops,providers,effects,dispatch}.rs` plus `process.rs`; `commands.rs` keeps shared project/runtime helpers and `host.rs` keeps the `Host` trait methods and capability checks | split; shared helpers and trait methods remain by design |
| P2 | HTTP parsing is Unicode-codepoint oriented; a general byte body is not the only path | open; convert before a general network release |
| P2 | No DNS, TLS, listener or UDP provider | open |
| P2 | DNS, UDP, TCP listener and TLS client endpoints are implemented; no TLS server/mTLS or custom-CA CLI | open |
| P2 | `std:path`, a regex subset, `std:datetime`, `std:clock`, `std:env`, `std:process` and `std:random` are implemented; no compression | open |
| P2 | Memory limits: 16 MiB per region, 64 MiB per run | current budget, not a product limit |
| P3 | No concurrency tasks, atomics or cancellable long-running scheduling | open; separate ADR |
@@ -168,5 +169,7 @@ that a phase is complete.
- [ADR-0044: `process.run` resource operation](../adr/0044-process-resource-operation.md) — accepted exact-path process grant; not a sandbox.
- [ADR-0045: fs append/read_at/walk](../adr/0045-fs-append-read-at-walk.md) — accepted bounded content and directory walk.
- [ADR-0046: `random.bytes`](../adr/0046-random-resource-operation.md) — accepted explicitly granted OS entropy.
- [ADR-0047: DNS/UDP/TCP listener](../adr/0047-net-dns-udp-listener.md) — accepted project-mode network endpoints.
- [ADR-0048: TLS client endpoint](../adr/0048-tls-resource-endpoint.md) — accepted validated rustls client; system + embedder DER roots.
- [Compute-machine RFC-0001](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) and [core/stdlib RFC-0002](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) — design input.
- [Architecture audit and refactor RFC](Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md) — review input for this migration.
+2
View File
@@ -51,6 +51,8 @@ The implemented first slices of these RFCs are recorded in `CURRENT.md` and
| `process.run` resource operation | [0044](../adr/0044-process-resource-operation.md) | Accepted (exact program grant, project only, not a sandbox) |
| `fs.append`/`fs.read_at`/`fs.walk` | [0045](../adr/0045-fs-append-read-at-walk.md) | Accepted (bounded content, directory walk) |
| `random.bytes` resource operation | [0046](../adr/0046-random-resource-operation.md) | Accepted (explicit `--allow-random`, project only) |
| DNS/UDP/TCP-listener endpoints | [0047](../adr/0047-net-dns-udp-listener.md) | Accepted (project only, exact host grant, no TLS) |
| TLS client endpoint | [0048](../adr/0048-tls-resource-endpoint.md) | Accepted (rustls, validated, project only) |
| Table physical storage (T1/T2/T3), strict stdin/stderr/file/network resources | — | **Not decided**; do not implement by implication |
## Evidence and status
+39
View File
@@ -0,0 +1,39 @@
# Evidence: DNS, UDP and TCP listener endpoints
Date: 2026-11-21 (Asia/Shanghai)
Source baseline: `ee0c6cc80a2782c4e88de63195eaf41a27077dcf`
Repository: `marqdo-cli`
Decision record: [ADR-0047](../adr/0047-net-dns-udp-listener.md)
## Implemented
- `resource.open("tcp", "host:port", ["read","write"])`: exact host grant is
checked **before** `ToSocketAddrs`; every resolved address is tried with a
5 s connect timeout.
- `resource.open("udp", "host:port", ["read","write"])`: connected datagram
socket from an ephemeral local port; `read` returns one datagram with
`eof=false`; `write` sends one datagram.
- `resource.open("tcp-listener", "host:port", ["accept"])` plus
`resource.accept(listener)`: non-blocking 5 s poll, returns a new TCP
`Resource` in `{ok:true,value:Resource}` or `tcp.timeout`.
- All project-mode only; pure/strict refuse; no TLS.
## Tests
`crates/marqdo-host/tests/resource_streams.rs` (30 tests) covers TCP DNS
round-trip via `localhost`, UDP datagram round-trip, listener accept/read/write,
host-grant denial and project-only refusal.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace 486 passed, 0 failed, 2 ignored
cargo test -p marqdo-host --test resource_streams 30 passed
python3 tools/pack-stdlib.py PASS
```
## Not covered
No TLS/certificate validation, no server sockets outside project mode, no UDP
broadcast/multicast, no listener port discovery API, no connection pool and no
DNS pinning. TLS remains a separate ADR.
+39
View File
@@ -0,0 +1,39 @@
# Evidence: TLS client endpoint
Date: 2026-11-21 (Asia/Shanghai)
Source baseline: `ee0c6cc80a2782c4e88de63195eaf41a27077dcf`
Repository: `marqdo-cli`
Decision record: [ADR-0048](../adr/0048-tls-resource-endpoint.md)
## Implemented
`resource.open("tls", "host:port", ["read","write"])` is a project-mode
validated TLS client:
- exact host grant before DNS;
- TCP connect with 5 s timeout, then `rustls` handshake using platform native
roots plus `StdHost::with_tls_ca` DER roots;
- server name is the descriptor host; certificate/handshake failure is fatal
`host.error`;
- the resulting `tls` Resource supports `resource.read/write/close`;
- no TLS server, mTLS, ALPN or custom-CA CLI.
## Tests
`crates/marqdo-host/tests/resource_streams.rs` starts a local `rustls` server
with an `rcgen` self-signed certificate, adds that DER as an explicit root and
round-trips a 4-byte payload. A second test pins project-only/host-grant
refusal (32 host tests).
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace 488 passed, 0 failed, 2 ignored
cargo test -p marqdo-host --test resource_streams 32 passed
python3 tools/pack-stdlib.py PASS
```
## Not covered
No TLS server/mTLS, no ALPN, no OCSP/CRL, no client certificates, no CLI
private-CA flag and no session resumption.