feat(host): add IP-literal blocking TCP resource endpoint (P5)
- resource.open("tcp", ip:port, ["read","write"]) with exact IP grants,
project mode only, 5s connect/read/write timeouts; no DNS/TLS/UDP/server
sockets (ADR-0034).
- Add TCP stream storage/dispatch in the resource table; CLI --allow-net IP;
pure/strict preflight rejects the grant and the language endpoint.
- Host loopback echo/grant/DNS/strict tests and a .mq.md CLI echo test.
- Update specs, status, architecture, README, changelog and evidence.
Gate: fmt, clippy -D warnings, 426 tests passed.
This commit is contained in:
1 parent
1d87f98539
commit
00bdcad1ff
15 files changed
+435
-4
No files matched your search
+7
-1
@@ -135,6 +135,9 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
PluginHandle `plugin/export` are read-only observation members (host id
|
||||
private); `stdlib/format.mq.md` reproduces builtin `打印`/`repr` for all 15
|
||||
value types, including Formula/Code/FunctionRef and Code UTF-8 byte length.
|
||||
- Blocking TCP endpoint (P5 first slice, ADR-0034): project-mode
|
||||
`resource.open("tcp", ip:port, ["read","write"])` with exact IP grant and
|
||||
timeouts; no DNS/TLS/UDP/server sockets; strict/pure refuse. CLI `--allow-net`.
|
||||
- In-process ResourceProvider v1 (CM-A09 first slice, ADR-0033): trusted
|
||||
providers register additional resource kinds without parser/runtime/HostOp
|
||||
changes; project-mode + name grant required; pure/strict refuse provider
|
||||
@@ -195,7 +198,7 @@ behavior. This table records only what is in this tree now.
|
||||
| P2 Marqdo-written allocator/data structures + minimal structure observation | **PARTIAL, gate NOT passed** | `.mq.md`-only evidence: bump allocator with u32 codec, linked list, failed-allocation atomicity (48–50); CM-A04 structure primitives and a table filter (51–54, ADR-0023); a first-fit free-list allocator with block reuse, splitting, address-adjacent coalescing and `Result` maps (55, ADR-0024), a `Memory`-backed binary tree with recursive insert/in-order traversal (56), a `.mq.md` CSV row splitter (57), an open-addressing hash table with linear probing, update and `hash.full` (58), a `.mq.md` UTF-8 encoder over `Memory` (59), a validating UTF-8 decoder rejecting truncated, bad-continuation and surrogate/overlong input (60), a `Memory`-backed graph adjacency list (61), a recursive JSON parser for nested arrays/objects, strings with escapes, integers, booleans, null and error Results (62), a `Memory`-backed stack bytecode VM with PUSH/ADD/MUL/JMP/JZ/HALT (63), big-endian 16/32/64-bit plus LEB128 integer codecs (64), a `.mq.md` table sort rebuilding a new table from `table.row/columns/cell` (65), a table join (67), a quoted/escaped CSV row parser (68), table group (69), and an Int matrix multiply (70); the first-fit allocator also detects double free (`alloc.double_free`). Evidence: [`memory-p2-closure`](evidence/2026-11-20-memory-p2-closure.md), [`structure-api-p2`](evidence/2026-11-20-structure-api-p2.md), [`result-and-free-list-p2`](evidence/2026-11-20-result-and-free-list-p2.md). P2 remaining: publication as a separate external stdlib repository. The bundled `stdlib/` tree, `std:` imports, digest verification, Analysis identity and frozen-CLI closure test are implemented (ADR-0028). C01–C06/C11/C12 each have `.mq.md` evidence, including JSON floats and a JSON formatter. |
|
||||
| P3 Resource/Stream ABI | **PARTIAL** | ADR-0025/0026 CM-A06 slices: opaque `Resource`, `resource.open/read/write/close`, stdout write in pure/project, granted project-file read and buffered project-file write in project, EOF/closed/handle errors, wire/host rejection; strict permits only stdio/stdout and refuses file/network resources. Fixture 66 + CLI/host tests. CM-A07 **Compat-C** is implemented: when a program calls `打印/print`, the loader injects the bundled `stdlib/io.mq.md` and the runtime routes to `io.写值` (all 15 value types, UTF-8 via `Memory`, `resource.write(stdout)`); `HostOp::Print` and the legacy `Value::display()` print path are removed, and a missing bundle fails with `kernel.unavailable`. Strict permits only the existing stdio console channel. Still missing: directories, network, generic provider registry, Table storage, strict file/network resources. |
|
||||
| P4 Table logical/physical migration | **PARTIAL** | ADR-0032/CM-A08: `Value::Table` logical semantics unchanged (T0 `Vec<Vec<Value>>`); `stdlib/data/table.mq.md` provides select/filter/sort/append/set/join/group as value-returning `.mq.md` functions with input immutability. No T1/T2/T3 physical storage, column store or Memory-backed table. |
|
||||
| P5 third-party Provider / TCP / HTTP | **PARTIAL** | ADR-0033 in-process `ResourceProvider` v1: explicitly trusted providers register by kind, project-mode only, gated by `Capabilities.allow_resource_provider`, using the existing `resource.*`/`Result` ABI; a mock read/write provider is tested. No C/Worker provider transport, network/socket/TCP/HTTP, clock/random/process/device providers, or provider package installation. |
|
||||
| P5 third-party Provider / TCP / HTTP | **PARTIAL** | ADR-0034 blocking TCP resource endpoint (`resource.open("tcp", "<ip>:<port>", ["read","write"])`, exact IP grant, project-only, 5s timeouts; loopback host+CLI tests). ADR-0033 in-process `ResourceProvider` v1: explicitly trusted providers register by kind, project-mode only, gated by `Capabilities.allow_resource_provider`, using the existing `resource.*`/`Result` ABI; a mock read/write provider is tested. No C/Worker provider transport, DNS/TLS/HTTP framing, UDP, clock/random/process/device providers, or provider package installation. |
|
||||
| P6 performance / native backends | **NOT IMPLEMENTED** | no bytecode/JIT/GPU/QPU work |
|
||||
|
||||
The P1 memory domain is pure computation: it needs no grant and is available in
|
||||
@@ -236,6 +239,9 @@ user-written allocator is explicitly **not** claimed.
|
||||
- [`evidence/2026-11-20-provider-abi-cm-a09.md`](evidence/2026-11-20-provider-abi-cm-a09.md)
|
||||
— ADR-0033 in-process ResourceProvider v1, grants/mode checks, mock provider
|
||||
read/write and registration rejection cases.
|
||||
- [`evidence/2026-11-20-tcp-endpoint.md`](evidence/2026-11-20-tcp-endpoint.md)
|
||||
— ADR-0034 blocking TCP endpoint, IP grants, DNS refusal, loopback host/CLI
|
||||
echo tests.
|
||||
- [`evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md`](evidence/RELEASE_CANDIDATE_0.1.0_8af7262.md)
|
||||
— G0..G12 matrix and RC decision.
|
||||
- [`security-gate.md`](security-gate.md) — mechanism, attack matrix and strict
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
# ADR-0034: Blocking TCP resource endpoint v1 (CM-A09/P5 first slice)
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-20
|
||||
Relates to: `docs/MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md`
|
||||
(CM-A09, CM-A11, P5), ADR-0025, ADR-0033
|
||||
Specs: `spec/evaluation.md`, `spec/security.md`
|
||||
Evidence: `docs/evidence/2026-11-20-tcp-endpoint.md`
|
||||
|
||||
## Context
|
||||
|
||||
CM-A09/P5 require a real external endpoint reached through the stable resource
|
||||
ABI, not a new `HostOp` or Rust builtin per application. The in-process
|
||||
provider slice (ADR-0033) covers trusted adapters; it does not exercise a real
|
||||
OS byte stream. This ADR adds the first OS endpoint: a blocking TCP stream.
|
||||
|
||||
The scope is deliberately narrow: HTTP/1.1, TLS, DNS, UDP, sockets as a broad
|
||||
API and nonblocking/async I/O remain standard-library or later work.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Resource kind and descriptor
|
||||
|
||||
```text
|
||||
resource.open("tcp", "<ip-literal>:<port>", ["read", "write"]) -> Resource
|
||||
```
|
||||
|
||||
* descriptor must parse as a `SocketAddr` containing an **IP literal**. DNS
|
||||
names are rejected with `resource.unsupported`; no resolver is called.
|
||||
* rights must be exactly `["read","write"]` in this build. A read-only or
|
||||
write-only TCP handle is future work.
|
||||
* `resource.read`/`write`/`close` behave like other streams:
|
||||
* read returns `{count, eof}`; `count=0` is transport EOF;
|
||||
* write returns a possibly short count;
|
||||
* close calls `shutdown(Both)` and marks the handle closed;
|
||||
* closed/forged/stale handles use the existing `resource.closed` /
|
||||
`runtime.resource` rules.
|
||||
* connect, read and write have 5-second timeouts. This is blocking I/O on the
|
||||
interpreter thread; no async runtime is introduced.
|
||||
|
||||
### 2. Authorization
|
||||
|
||||
* `project` mode only. `pure` and `strict` refuse TCP (`permission.denied`);
|
||||
strict remains fail-closed (CM-A11), and the G-C lane verifies the refusal.
|
||||
* `Capabilities.check_net(ip)` requires an exact IP-literal grant. The CLI adds
|
||||
`--allow-net <IP>` (repeatable); `--allow-net` is refused alongside other
|
||||
external grants in pure/strict preflight.
|
||||
* Granting a hostname or a network range does not authorize a different IP;
|
||||
there is no DNS-based allowlist bypass because DNS is not used.
|
||||
* No TLS, no proxy, no HTTP parsing in the host. `.mq.md` code composes
|
||||
protocols over the byte stream.
|
||||
|
||||
### 3. Trace
|
||||
|
||||
`resource.open/read/write/close` effect events are recorded. The trace records
|
||||
the IP:port for open but never payload bytes or credentials. A TCP endpoint is
|
||||
an external effect and is not replayed as pure computation.
|
||||
|
||||
### 4. Explicit non-goals
|
||||
|
||||
* DNS, TLS, HTTP/1.1 or HTTP/2 host implementations, proxies, redirects.
|
||||
* UDP, ICMP, raw sockets, socket options, nonblocking/async/select.
|
||||
* File descriptors exposed to the language, `setsockopt`, `bind`/`listen` server
|
||||
sockets.
|
||||
* Strict-mode TCP and any claim that in-process `std::net` is OS isolation.
|
||||
* Network provider package installation or a generic socket registry.
|
||||
|
||||
## Consequences
|
||||
|
||||
* `.mq.md` can implement a line protocol/HTTP client parser on top of the TCP
|
||||
byte stream using the existing Memory/Text/Result primitives.
|
||||
* The runtime still has no `net.request` builtin or new `HostOp`; a CLI test
|
||||
opens a local echo server, writes/reads `ping` through `resource.*`, and
|
||||
closes.
|
||||
* Blocking timeouts bound a hung peer but do not preempt a blocking syscall
|
||||
deeper than the OS timeout; this is documented.
|
||||
* Network tests use loopback only; external network access is not required for
|
||||
the gate.
|
||||
|
||||
## Verification
|
||||
|
||||
`docs/evidence/2026-11-20-tcp-endpoint.md` records host tests (loopback echo,
|
||||
grant/pure/strict/DNS refusal) and a CLI `.mq.md` echo test with
|
||||
`--allow-net 127.0.0.1`, plus the full workspace gate.
|
||||
@@ -30,6 +30,7 @@ implementation status lives in `docs/STATUS.md`; frozen behavior is in
|
||||
| CM-A07 Compat-C remove legacy print formatting | [0031](../adr/0031-print-compat-c-remove-host-formatting.md) | Accepted |
|
||||
| CM-A08 table logical value / stdlib algorithms / T0 storage | [0032](../adr/0032-table-logical-value-and-stdlib-algorithms.md) | Accepted |
|
||||
| CM-A09 in-process ResourceProvider v1 | [0033](../adr/0033-in-process-resource-provider-v1.md) | Accepted (trusted in-process slice) |
|
||||
| P5 blocking TCP resource endpoint v1 | [0034](../adr/0034-tcp-resource-endpoint.md) | Accepted (IP-literal, no DNS/TLS) |
|
||||
| Table physical storage (T1/T2/T3), C/Worker provider transport, strict file/network resources | — | Not decided; do not implement by implication |
|
||||
|
||||
## What a future phase must not do
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
# RFC-CM P5 evidence: blocking TCP resource endpoint v1
|
||||
|
||||
Date: 2026-11-20 (Asia/Shanghai)
|
||||
Repository: `marqdo-cli`
|
||||
Baseline: `1d87f98` plus this step's changes.
|
||||
Decision record: [ADR-0034](../adr/0034-tcp-resource-endpoint.md)
|
||||
Related: [provider v1](2026-11-20-provider-abi-cm-a09.md), [STATUS §5](../STATUS.md).
|
||||
Scope: IP-literal blocking TCP only; no DNS/TLS/UDP/server sockets.
|
||||
|
||||
## Environment and gate
|
||||
|
||||
Ubuntu 26.04 development container, Linux x86_64, Rust 1.100.0.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace --locked PASS
|
||||
426 passed, 0 failed, 2 ignored (existing plugin-sdk doc-tests)
|
||||
```
|
||||
|
||||
No GitHub CI is available from this checkout; this is local evidence.
|
||||
|
||||
## Contract
|
||||
|
||||
```text
|
||||
resource.open("tcp", "<ip-literal>:<port>", ["read", "write"])
|
||||
```
|
||||
|
||||
* project mode only; `pure`/`strict` refuse;
|
||||
* exact IP-literal grant (`Capabilities.check_net`), CLI `--allow-net <IP>`;
|
||||
* DNS names rejected with `resource.unsupported` (no resolver is called);
|
||||
* connect/read/write timeouts 5 seconds; blocking I/O on the interpreter thread;
|
||||
* `resource.read` returns `{count, eof}`, `write` a possibly short count,
|
||||
`close` shuts the socket down; closed/forged handles use existing rules;
|
||||
* trace records `resource.open/read/write/close` effects (IP:port on open, no
|
||||
payload bytes).
|
||||
|
||||
No new `HostOp`, `Value`, builtin or Markdown syntax.
|
||||
|
||||
## Tests
|
||||
|
||||
Host (`crates/marqdo-host/tests/resource_streams.rs`, 2 TCP tests):
|
||||
|
||||
```text
|
||||
tcp_resource_round_trips_with_a_network_grant
|
||||
loopback echo: write ping -> read ping -> close
|
||||
|
||||
tcp_requires_network_grant_and_rejects_dns_names
|
||||
no --allow-net -> permission.denied
|
||||
"example.test:80" -> resource.unsupported
|
||||
pure/strict -> permission.denied
|
||||
```
|
||||
|
||||
CLI (`crates/cli/tests/resource_streams.rs`):
|
||||
|
||||
```text
|
||||
tcp_stream_round_trips_through_mqmd_with_a_network_grant
|
||||
local TcpListener; .mq.md writes/reads 4 bytes through Memory;
|
||||
run --allow-net 127.0.0.1 --json -> [4,4,112,105,110,103,true]
|
||||
```
|
||||
|
||||
## Not done
|
||||
|
||||
* DNS, TLS, HTTP framing, proxies, UDP, raw sockets, socket options.
|
||||
* Nonblocking/async/select and server bind/listen.
|
||||
* Strict-mode TCP and any claim that `std::net` in-process is OS isolation.
|
||||
* Provider package installation/registry for network endpoints.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
export PATH=/root/.cargo/bin:$PATH
|
||||
export CARGO_HOME=/root/work/marqdo-cli/.cache/cargo
|
||||
export CARGO_TARGET_DIR=/root/work/marqdo-cli/.cache/target
|
||||
cargo test -p marqdo-host --test resource_streams tcp
|
||||
cargo test -p marqdo-cli --test resource_streams tcp
|
||||
cargo test --workspace --locked
|
||||
```
|
||||
Reference in new issue
Block a user