feat(host): add IP-literal blocking TCP resource endpoint (P5)

- resource.open("tcp", ip:port, ["read","write"]) with exact IP grants,
  project mode only, 5s connect/read/write timeouts; no DNS/TLS/UDP/server
  sockets (ADR-0034).
- Add TCP stream storage/dispatch in the resource table; CLI --allow-net IP;
  pure/strict preflight rejects the grant and the language endpoint.
- Host loopback echo/grant/DNS/strict tests and a .mq.md CLI echo test.
- Update specs, status, architecture, README, changelog and evidence.

Gate: fmt, clippy -D warnings, 426 tests passed.
This commit is contained in:
cflmy committed 2026-11-20 23:05:06 +08:00
1 parent 1d87f98539
commit 00bdcad1ff
15 files changed
+435 -4

No files matched your search

+7 -1
View File
@@ -135,6 +135,9 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
PluginHandle `plugin/export` are read-only observation members (host id
private); `stdlib/format.mq.md` reproduces builtin `打印`/`repr` for all 15
value types, including Formula/Code/FunctionRef and Code UTF-8 byte length.
- Blocking TCP endpoint (P5 first slice, ADR-0034): project-mode
`resource.open("tcp", ip:port, ["read","write"])` with exact IP grant and
timeouts; no DNS/TLS/UDP/server sockets; strict/pure refuse. CLI `--allow-net`.
- In-process ResourceProvider v1 (CM-A09 first slice, ADR-0033): trusted
providers register additional resource kinds without parser/runtime/HostOp
changes; project-mode + name grant required; pure/strict refuse provider
@@ -195,7 +198,7 @@ behavior. This table records only what is in this tree now.
| P2 Marqdo-written allocator/data structures + minimal structure observation | **PARTIAL, gate NOT passed** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), a quoted/escaped CSV row parser (68), table group (69), and an Int matrix multiply (70); the first-fit allocator also detects double free (`alloc.double_free`). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). P2 remaining: publication as a separate external stdlib repository. The bundled `stdlib/` tree, `std:` imports, digest verification, Analysis identity and frozen-CLI closure test are implemented (ADR-0028). C01–C06/C11/C12 each have `.mq.md` evidence, including JSON floats and a JSON formatter. |
| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict permits only stdio/stdout and refuses file/network resources. Fixture 66 + CLI/host tests. CM-A07 **Compat-C** is implemented: when a program calls `打印/print`, the loader injects the bundled `stdlib/io.mq.md` and the runtime routes to `io.写值` (all 15 value types, UTF-8 via `Memory`, `resource.write(stdout)`); `HostOp::Print` and the legacy `Value::display()` print path are removed, and a missing bundle fails with `kernel.unavailable`. Strict permits only the existing stdio console channel. Still missing: directories, network, generic provider registry, Table storage, strict file/network resources. |
| P4 Table logical/physical migration | **PARTIAL** | ADR-0032/CM-A08: `Value::Table` logical semantics unchanged (T0 `Vec<Vec<Value>>`); `stdlib/data/table.mq.md` provides select/filter/sort/append/set/join/group as value-returning `.mq.md` functions with input immutability. No T1/T2/T3 physical storage, column store or Memory-backed table. |
| P5 third-party Provider / TCP / HTTP | **PARTIAL** | ADR-0033 in-process `ResourceProvider` v1: explicitly trusted providers register by kind, project-mode only, gated by `Capabilities.allow_resource_provider`, using the existing `resource.*`/`Result` ABI; a mock read/write provider is tested. No C/Worker provider transport, network/socket/TCP/HTTP, clock/random/process/device providers, or provider package installation. |
| P5 third-party Provider / TCP / HTTP | **PARTIAL** | ADR-0034 blocking TCP resource endpoint (`resource.open("tcp", "<ip>:<port>", ["read","write"])`, exact IP grant, project-only, 5s timeouts; loopback host+CLI tests). ADR-0033 in-process `ResourceProvider` v1: explicitly trusted providers register by kind, project-mode only, gated by `Capabilities.allow_resource_provider`, using the existing `resource.*`/`Result` ABI; a mock read/write provider is tested. No C/Worker provider transport, DNS/TLS/HTTP framing, UDP, clock/random/process/device providers, or provider package installation. |
| P6 performance / native backends | **NOT IMPLEMENTED** | no bytecode/JIT/GPU/QPU work |
The P1 memory domain is pure computation: it needs no grant and is available in
@@ -236,6 +239,9 @@ user-written allocator is explicitly **not** claimed.
- [`evidence/2026-11-20-provider-abi-cm-a09.md`](evidence/2026-11-20-provider-abi-cm-a09.md)
— ADR-0033 in-process ResourceProvider v1, grants/mode checks, mock provider
read/write and registration rejection cases.
- [`evidence/2026-11-20-tcp-endpoint.md`](evidence/2026-11-20-tcp-endpoint.md)
— ADR-0034 blocking TCP endpoint, IP grants, DNS refusal, loopback host/CLI
echo tests.
- [`evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md`](evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md)
— G0..G12 matrix and RC decision.
- [`security-gate.md`](security-gate.md) — mechanism, attack matrix and strict
+84
View File
@@ -0,0 +1,84 @@
# ADR-0034: Blocking TCP resource endpoint v1 (CM-A09/P5 first slice)
Status: **Accepted**
Date: 2026-11-20
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
(CM-A09, CM-A11, P5), ADR-0025, ADR-0033
Specs: `spec/evaluation.md`, `spec/security.md`
Evidence: `docs/evidence/2026-11-20-tcp-endpoint.md`
## Context
CM-A09/P5 require a real external endpoint reached through the stable resource
ABI, not a new `HostOp` or Rust builtin per application. The in-process
provider slice (ADR-0033) covers trusted adapters; it does not exercise a real
OS byte stream. This ADR adds the first OS endpoint: a blocking TCP stream.
The scope is deliberately narrow: HTTP/1.1, TLS, DNS, UDP, sockets as a broad
API and nonblocking/async I/O remain standard-library or later work.
## Decision
### 1. Resource kind and descriptor
```text
resource.open("tcp", "<ip-literal>:<port>", ["read", "write"]) -> Resource
```
* descriptor must parse as a `SocketAddr` containing an **IP literal**. DNS
names are rejected with `resource.unsupported`; no resolver is called.
* rights must be exactly `["read","write"]` in this build. A read-only or
write-only TCP handle is future work.
* `resource.read`/`write`/`close` behave like other streams:
* read returns `{count, eof}`; `count=0` is transport EOF;
* write returns a possibly short count;
* close calls `shutdown(Both)` and marks the handle closed;
* closed/forged/stale handles use the existing `resource.closed` /
`runtime.resource` rules.
* connect, read and write have 5-second timeouts. This is blocking I/O on the
interpreter thread; no async runtime is introduced.
### 2. Authorization
* `project` mode only. `pure` and `strict` refuse TCP (`permission.denied`);
strict remains fail-closed (CM-A11), and the G-C lane verifies the refusal.
* `Capabilities.check_net(ip)` requires an exact IP-literal grant. The CLI adds
`--allow-net <IP>` (repeatable); `--allow-net` is refused alongside other
external grants in pure/strict preflight.
* Granting a hostname or a network range does not authorize a different IP;
there is no DNS-based allowlist bypass because DNS is not used.
* No TLS, no proxy, no HTTP parsing in the host. `.mq.md` code composes
protocols over the byte stream.
### 3. Trace
`resource.open/read/write/close` effect events are recorded. The trace records
the IP:port for open but never payload bytes or credentials. A TCP endpoint is
an external effect and is not replayed as pure computation.
### 4. Explicit non-goals
* DNS, TLS, HTTP/1.1 or HTTP/2 host implementations, proxies, redirects.
* UDP, ICMP, raw sockets, socket options, nonblocking/async/select.
* File descriptors exposed to the language, `setsockopt`, `bind`/`listen` server
sockets.
* Strict-mode TCP and any claim that in-process `std::net` is OS isolation.
* Network provider package installation or a generic socket registry.
## Consequences
* `.mq.md` can implement a line protocol/HTTP client parser on top of the TCP
byte stream using the existing Memory/Text/Result primitives.
* The runtime still has no `net.request` builtin or new `HostOp`; a CLI test
opens a local echo server, writes/reads `ping` through `resource.*`, and
closes.
* Blocking timeouts bound a hung peer but do not preempt a blocking syscall
deeper than the OS timeout; this is documented.
* Network tests use loopback only; external network access is not required for
the gate.
## Verification
`docs/evidence/2026-11-20-tcp-endpoint.md` records host tests (loopback echo,
grant/pure/strict/DNS refusal) and a CLI `.mq.md` echo test with
`--allow-net 127.0.0.1`, plus the full workspace gate.
+1
View File
@@ -30,6 +30,7 @@ implementation status lives in `docs/STATUS.md`; frozen behavior is in
| CM-A07 Compat-C remove legacy print formatting | [0031](../adr/0031-print-compat-c-remove-host-formatting.md) | Accepted |
| CM-A08 table logical value / stdlib algorithms / T0 storage | [0032](../adr/0032-table-logical-value-and-stdlib-algorithms.md) | Accepted |
| CM-A09 in-process ResourceProvider v1 | [0033](../adr/0033-in-process-resource-provider-v1.md) | Accepted (trusted in-process slice) |
| P5 blocking TCP resource endpoint v1 | [0034](../adr/0034-tcp-resource-endpoint.md) | Accepted (IP-literal, no DNS/TLS) |
| Table physical storage (T1/T2/T3), C/Worker provider transport, strict file/network resources | — | Not decided; do not implement by implication |
## What a future phase must not do
+78
View File
@@ -0,0 +1,78 @@
# RFC-CM P5 evidence: blocking TCP resource endpoint v1
Date: 2026-11-20 (Asia/Shanghai)
Repository: `marqdo-cli`
Baseline: `1d87f98` plus this step's changes.
Decision record: [ADR-0034](../adr/0034-tcp-resource-endpoint.md)
Related: [provider v1](2026-11-20-provider-abi-cm-a09.md), [STATUS §5](../STATUS.md).
Scope: IP-literal blocking TCP only; no DNS/TLS/UDP/server sockets.
## Environment and gate
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
```text
cargo fmt --all -- --check PASS
cargo clippy --workspace --all-targets -- -D warnings PASS
cargo test --workspace --locked PASS
426 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
```
No GitHub CI is available from this checkout; this is local evidence.
## Contract
```text
resource.open("tcp", "<ip-literal>:<port>", ["read", "write"])
```
* project mode only; `pure`/`strict` refuse;
* exact IP-literal grant (`Capabilities.check_net`), CLI `--allow-net <IP>`;
* DNS names rejected with `resource.unsupported` (no resolver is called);
* connect/read/write timeouts 5 seconds; blocking I/O on the interpreter thread;
* `resource.read` returns `{count, eof}`, `write` a possibly short count,
`close` shuts the socket down; closed/forged handles use existing rules;
* trace records `resource.open/read/write/close` effects (IP:port on open, no
payload bytes).
No new `HostOp`, `Value`, builtin or Markdown syntax.
## Tests
Host (`crates/marqdo-host/tests/resource_streams.rs`, 2 TCP tests):
```text
tcp_resource_round_trips_with_a_network_grant
loopback echo: write ping -> read ping -> close
tcp_requires_network_grant_and_rejects_dns_names
no --allow-net -> permission.denied
"example.test:80" -> resource.unsupported
pure/strict -> permission.denied
```
CLI (`crates/cli/tests/resource_streams.rs`):
```text
tcp_stream_round_trips_through_mqmd_with_a_network_grant
local TcpListener; .mq.md writes/reads 4 bytes through Memory;
run --allow-net 127.0.0.1 --json -> [4,4,112,105,110,103,true]
```
## Not done
* DNS, TLS, HTTP framing, proxies, UDP, raw sockets, socket options.
* Nonblocking/async/select and server bind/listen.
* Strict-mode TCP and any claim that `std::net` in-process is OS isolation.
* Provider package installation/registry for network endpoints.
## Reproduce
```bash
export PATH=/root/.cargo/bin:$PATH
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
cargo test -p marqdo-host --test resource_streams tcp
cargo test -p marqdo-cli --test resource_streams tcp
cargo test --workspace --locked
```