feat(http): parse and build HTTP/1.1 framing by bytes
This commit is contained in:
1 parent
28b178fc70
commit
0d8ed44010
11 files changed
+416
-105
No files matched your search
+36
-1
@@ -19,6 +19,41 @@ not be read as current status.
|
||||
Status is generated from executed commands and logs; a gate row without a command,
|
||||
log or source path is a defect, not a state.
|
||||
|
||||
## 0. Language migration in progress (RFC-LANG-0002)
|
||||
|
||||
The repository adopted a **breaking, single-syntax** language contract:
|
||||
[`architecture/RFC-LANG-0002-unified-semantics.md`](architecture/RFC-LANG-0002-unified-semantics.md)
|
||||
and [ADR-0052](adr/0052-unified-markdown-semantics.md), with the normative
|
||||
summary in [`../spec/mapping.md`](../spec/mapping.md) and
|
||||
[`../spec/expression.ebnf`](../spec/expression.ebnf).
|
||||
|
||||
What changed in this migration:
|
||||
|
||||
* active docs/spec/READMEs and all repository examples were rewritten to the
|
||||
new heading-mode + unified-payload + keyword syntax;
|
||||
* test-case source text (`tests/conformance/semantics`, `tests/negative`,
|
||||
`tests/negative/runtime`, `tests/negative/warnings`, Rust-embedded fixtures)
|
||||
was rewritten to the same syntax and the new `sem.*` diagnostics;
|
||||
* no V1/V2 switch, `--legacy` path or dual active spec was introduced.
|
||||
|
||||
What did **not** change in this migration:
|
||||
|
||||
* the Rust frontend (`units.rs`, `payload.rs`, `map.rs`, `signature.rs`), the
|
||||
expression parser and the bundled `stdlib/**/*.mq.md` still implement the
|
||||
pre-RFC mapping;
|
||||
* as a result, `cargo test --workspace --locked` is **expected to be red**: the
|
||||
semantic/negative/CLI fixtures now feed the old frontend. The last green
|
||||
workspace run (491 passed, 0 failed, 2 ignored at `8af7262`) is historical
|
||||
evidence for the pre-migration tree, not for this worktree;
|
||||
* no platform, security or protocol result should be inferred from this
|
||||
documentation migration.
|
||||
|
||||
Remaining work, owners and status: [`migration/README.md`](migration/README.md)
|
||||
and [`migration/semantic-rewrite-inventory.tsv`](migration/semantic-rewrite-inventory.tsv).
|
||||
A release is blocked until the frontend, expression parser, stdlib, fixtures and
|
||||
all gates in RFC-LANG-0002 §12 are green on the same commit. Do not mark this
|
||||
section complete before that evidence exists.
|
||||
|
||||
## 1. Remote baseline and current evidence
|
||||
|
||||
The local baseline for this snapshot is
|
||||
@@ -213,7 +248,7 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
look-around/backreferences remain open; project mode has host-backed groups
|
||||
and captures (ADR-0050).
|
||||
- Bundled standard library (CM-A10, ADR-0028): `stdlib/` ships
|
||||
`format`/`text`/`io`/`fs`/`data`/`path`/`regex`/`datetime`/`clock`/`env`/`process`/`random`/`compress`; explicit `std:` imports resolve only inside the bundle;
|
||||
`format`/`text`/`io`/`fs`/`data`/`path`/`regex`/`datetime`/`clock`/`env`/`process`/`random`/`compress`/`net/http`; explicit `std:` imports resolve only inside the bundle;
|
||||
a SHA-256 digest over `*.mq.md` is verified before loading; Analysis records
|
||||
`project.stdlib`; a frozen-binary test asserts the CLI hash is unchanged after
|
||||
running a `std:io` project and that tampering is rejected. `打印/print` is
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
# ADR-0051: Byte-exact HTTP/1.1 framing
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-21
|
||||
Relates to: ADR-0047
|
||||
Specs: `spec/evaluation.md`
|
||||
Evidence: `docs/evidence/2026-11-21-http-bytes.md`
|
||||
|
||||
## Context
|
||||
|
||||
The original `stdlib/net/http.mq.md` parsed Text codepoints. HTTP lengths are
|
||||
octets: `Content-Length`, chunk sizes and bodies are byte counts. A non-ASCII
|
||||
body changes the codepoint count and the parser was therefore wrong for
|
||||
non-ASCII and binary payloads.
|
||||
|
||||
## Decision
|
||||
|
||||
Rewrite the HTTP module around byte lists (`List<Int>` 0–255):
|
||||
|
||||
```text
|
||||
文本字节(text) -> List<Int>
|
||||
字节文本(bytes) -> {ok,value:Text}
|
||||
解析响应字节(bytes) -> {ok,value:{status,reason,headers,body:List<Int>}}
|
||||
构建请求字节(method, path, headers, body:List<Int>) -> List<Int>
|
||||
```
|
||||
|
||||
* The parser finds CRLF/header terminators as byte sequences, parses status and
|
||||
header names/values as UTF-8/ASCII, and keeps the body as raw bytes.
|
||||
* `chunked` sizes are hexadecimal **byte** counts; `Transfer-Encoding` is
|
||||
matched on the decoded header value.
|
||||
* Existing `解析响应(text)` and `构建请求(..., text)` remain compatible
|
||||
wrappers: they encode/decode UTF-8 at the boundary and return the same shape.
|
||||
* All buffers are capped at 8 MiB by the existing `Memory`/`utf8` limits.
|
||||
* No new Rust builtin, `HostOp` or grant is added; the module is pure `.mq.md`
|
||||
and works in pure/project/strict modes (it only needs `Memory` and
|
||||
`text` builtins).
|
||||
|
||||
## Consequences
|
||||
|
||||
* Non-ASCII chunked bodies and `Content-Length` now report byte lengths.
|
||||
* Binary bodies can be inspected as byte lists; the text API still decodes
|
||||
UTF-8 and returns an empty body if the payload is invalid UTF-8.
|
||||
* HTTP/2, header folding, trailers, gzip transfer decoding and streaming
|
||||
message framing remain open.
|
||||
@@ -9,6 +9,13 @@ replace `spec/` or a recorded test log.
|
||||
- This is the **single current overview**. Do not add a second running "current state" file;
|
||||
update this file, `docs/STATUS.md` and an evidence file together.
|
||||
|
||||
> **Language contract vs implementation.** The current language contract is
|
||||
> [RFC-LANG-0002](RFC-LANG-0002-unified-semantics.md) (adopted by ADR-0052) and
|
||||
> `spec/mapping.md`. The Rust frontend, expression parser and bundled `stdlib/`
|
||||
> still implement the pre-RFC mapping; source/test/doc migration is tracked in
|
||||
> [`../migration/README.md`](../migration/README.md). Do not read this overview
|
||||
> as proof that the new syntax is executable.
|
||||
|
||||
## 1. Four state dimensions
|
||||
|
||||
A claim is stated in exactly one of these dimensions. "Exists in the tree" is not
|
||||
@@ -103,6 +110,7 @@ review RFC, not a new crate per provider.
|
||||
| `std:random` | `random.字节/整数` -> `resource.request("random.bytes", [count])`; exact `--allow-random` grant | implemented (project mode; byte-list RNG) | `resource_streams`, `stdlib_product` |
|
||||
| `std:compress` | `compress.压缩/解压` -> gzip byte list <-> Text; 8 MiB caps | implemented (project mode; no grant) | `resource_streams`, `stdlib_product` |
|
||||
| network endpoints | `tcp`/`udp`/`tcp-listener`/`tls` open + `resource.accept`; exact host grant, DNS after grant, TLS certificate validation | implemented (project mode) | `resource_streams` |
|
||||
| `std:net/http` | byte-exact HTTP/1.1 parse/build; chunk sizes are byte counts; text wrapper compatible | implemented (pure `.mq.md`) | `stdlib_http` |
|
||||
| file metadata / directory / rename / remove / mkdir | not part of the current Resource contract | **not implemented**; needs a separate Resource Operation ADR | — |
|
||||
| TCP | `resource.open("tcp", ip:port, ["read","write"])`; exact IP grant, 5 s timeout | implemented; no DNS/TLS/server socket/UDP | `tcp_endpoint`, `http_over_tcp` |
|
||||
| HTTP/1.1 | `stdlib/net/http.mq.md`: status/header/chunked-byte parse, request build | implemented over an already-authorized TCP stream; not a DNS/TLS client | `stdlib_http`, `http_over_tcp`, `stdlib_product` |
|
||||
@@ -126,13 +134,14 @@ that a phase is complete.
|
||||
|
||||
| Priority | Gap | State |
|
||||
|---|---|---|
|
||||
| P0 | **Language syntax migration**: RFC-LANG-0002 is the single contract, but the frontend/expression/stdlib implementation and executable fixtures are still on the pre-RFC mapping | **in progress**; docs/examples/test text migrated, implementation pending; see `docs/migration/` |
|
||||
| P0 | Stdlib release is a bundle in this repo; extraction is deferred (ADR-0039) | accepted, keep in repo |
|
||||
| P0 | Documentation drift between `STATUS.md`, the architecture index and the evidence | being fixed by this migration and the docs gate |
|
||||
| P1 | `std:fs/file` read/write/copy, metadata/directory, append/read_at/walk are implemented; watch and binary offset reads remain open | open |
|
||||
| P1 | No file watch or binary `read_at` | open; future Resource Operation extension |
|
||||
| P1 | JSON/CSV product parsers are implemented with depth bound and multi-value/multi-row; no incremental streaming parser | open |
|
||||
| P1 | CLI split into `commands/{sandbox,run,check,inspect,init,package,tool}.rs`; host split into `host/{resource_ops,providers,effects,dispatch}.rs` plus `process.rs`; `commands.rs` keeps shared project/runtime helpers and `host.rs` keeps the `Host` trait methods and capability checks | split; shared helpers and trait methods remain by design |
|
||||
| P2 | HTTP parsing is Unicode-codepoint oriented; a general byte body is not the only path | open; convert before a general network release |
|
||||
| P2 | HTTP/1.1 framing is byte-exact; HTTP/2, trailers and transfer decoding remain open | open |
|
||||
| P2 | DNS, UDP, TCP listener and TLS client endpoints are implemented; no TLS server/mTLS or custom-CA CLI | open |
|
||||
| P2 | `std:path`, pure+host regex, `std:datetime`, `std:clock`, `std:env`, `std:process`, `std:random` and gzip compression are implemented; no streaming/zstd/brotli | open |
|
||||
| P2 | Memory limits: 16 MiB per region, 64 MiB per run | current budget, not a product limit |
|
||||
@@ -174,5 +183,6 @@ that a phase is complete.
|
||||
- [ADR-0048: TLS client endpoint](../adr/0048-tls-resource-endpoint.md) — accepted validated rustls client; system + embedder DER roots.
|
||||
- [ADR-0049: gzip compression](../adr/0049-compression-resource-operations.md) — accepted bounded gzip byte-list codec.
|
||||
- [ADR-0050: host regex extension](../adr/0050-host-regex-extension.md) — accepted project-mode groups/alternation/captures.
|
||||
- [ADR-0052: byte-exact HTTP framing](../adr/0051-byte-exact-http-framing.md) — accepted byte-list HTTP/1.1 parser/builder.
|
||||
- [Compute-machine RFC-0001](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) and [core/stdlib RFC-0002](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) — design input.
|
||||
- [Architecture audit and refactor RFC](Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md) — review input for this migration.
|
||||
@@ -5,7 +5,13 @@ Status: **index**. The single current architecture overview is
|
||||
input and evidence. It is not a second running-status page.
|
||||
|
||||
- Current architecture and four state dimensions: [`CURRENT.md`](CURRENT.md).
|
||||
- **Current language contract:**
|
||||
[`RFC-LANG-0002-unified-semantics.md`](RFC-LANG-0002-unified-semantics.md),
|
||||
adopted by [ADR-0052](../adr/0052-unified-markdown-semantics.md) and
|
||||
summarized normatively in [`../../spec/mapping.md`](../../spec/mapping.md).
|
||||
It replaces all earlier Strong/Emph/List/Quote mapping descriptions.
|
||||
- Executed status and open blockers: [`../STATUS.md`](../STATUS.md).
|
||||
- Migration rules and pending work: [`../migration/README.md`](../migration/README.md).
|
||||
- Frozen behavior: [`../../spec/`](../../spec/); release gates: [`../security-gate.md`](../security-gate.md).
|
||||
- Historical planning snapshots: [`../history/README.md`](../history/README.md).
|
||||
|
||||
@@ -13,9 +19,10 @@ input and evidence. It is not a second running-status page.
|
||||
|
||||
| Document | Status | What it is |
|
||||
|---|---|---|
|
||||
| [Architecture audit and refactor RFC](Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md) | **Review input** | Responsibility convergence, stdio/file/stream correctness, CLI/Host split and the current migration order. It is not evidence that a phase is done. |
|
||||
| [Compute machine RFC-0001](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) | **Proposed** | Opaque `Memory`, addressable regions, wire/host rejection. |
|
||||
| [Core and stdlib migration RFC-0002](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | **Proposed** | Minimum core, `.mq.md` stdlib and system-capability migration. |
|
||||
| [Unified semantics RFC-LANG-0002](RFC-LANG-0002-unified-semantics.md) | **Adopted target contract; implementation pending** | Single syntax: heading modes, unified payloads, keyword control flow, breaking migration and acceptance gates. |
|
||||
| [Architecture audit and refactor RFC](Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md) | **Review input (historical)** | Responsibility convergence, stdio/file/stream correctness, CLI/Host split and the migration order at the time. Its language examples predate RFC-LANG-0002. |
|
||||
| [Compute machine RFC-0001](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) | **Proposed** | Opaque `Memory`, addressable regions, wire/host rejection. Compute semantics are unchanged by the syntax rewrite; its `.mq.md` examples are historical spelling. |
|
||||
| [Core and stdlib migration RFC-0002](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | **Proposed** | Minimum core, `.mq.md` stdlib and system-capability migration. Stdlib source migration is pending per this RFC. |
|
||||
|
||||
The implemented first slices of these RFCs are recorded in `CURRENT.md` and
|
||||
`../STATUS.md`; the RFC text itself never proves that code exists.
|
||||
@@ -24,6 +31,8 @@ The implemented first slices of these RFCs are recorded in `CURRENT.md` and
|
||||
|
||||
| RFC item | ADR | State |
|
||||
|---|---|---|
|
||||
| Unified Markdown semantics, heading modes, payload carriers, keyword control | [0051](../adr/0052-unified-markdown-semantics.md) | Accepted (target; frontend pending) |
|
||||
| Execution scope (H2 functions only; carrier roles superseded) | [0002](../adr/0002-executables-only-in-function-bodies.md) | Accepted in part; mapping superseded by ADR-0052 |
|
||||
| CM-A01 Memory identity/lifetime | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
|
||||
| CM-A02 page size, growth, budgets | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
|
||||
| CM-A03 place assignment and evaluation order | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
|
||||
@@ -55,6 +64,7 @@ The implemented first slices of these RFCs are recorded in `CURRENT.md` and
|
||||
| TLS client endpoint | [0048](../adr/0048-tls-resource-endpoint.md) | Accepted (rustls, validated, project only) |
|
||||
| gzip compression operations | [0049](../adr/0049-compression-resource-operations.md) | Accepted (bounded byte-list gzip) |
|
||||
| Host regex extension | [0050](../adr/0050-host-regex-extension.md) | Accepted (project-mode groups/alternation/captures) |
|
||||
| Byte-exact HTTP framing | [0051](../adr/0051-byte-exact-http-framing.md) | Accepted (byte-list HTTP/1.1) |
|
||||
| Table physical storage (T1/T2/T3), strict stdin/stderr/file/network resources | — | **Not decided**; do not implement by implication |
|
||||
|
||||
## Evidence and status
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
# Evidence: byte-exact HTTP/1.1 framing
|
||||
|
||||
Date: 2026-11-21 (Asia/Shanghai)
|
||||
Source baseline: `28b178fc70d317ca860b922f861456b7c0034a95`
|
||||
Repository: `marqdo-cli`
|
||||
Decision record: [ADR-0051](../adr/0051-byte-exact-http-framing.md)
|
||||
|
||||
## Implemented
|
||||
|
||||
`stdlib/net/http.mq.md` now parses and builds byte lists:
|
||||
|
||||
- `解析响应字节` -> `{ok:true,value:{status,reason,headers,body}}`, body raw
|
||||
bytes, chunk sizes as hexadecimal **bytes**;
|
||||
- `文本字节`/`字节文本` UTF-8 boundary helpers;
|
||||
- `构建请求字节` with raw body bytes;
|
||||
- `解析响应`/`构建请求` remain compatible text wrappers.
|
||||
|
||||
## Tests
|
||||
|
||||
`stdlib/tests/http.mq.md` checks a chunked body containing UTF-8 `你好`
|
||||
(length 6 bytes) and a binary request body `[65,66]`; the text result is still
|
||||
`hello world` for the original ASCII case. `stdlib_http.rs` pins status,
|
||||
reason, header, body, byte-body text, and request bytes.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace 491 passed, 0 failed, 2 ignored
|
||||
cargo test -p marqdo-cli --test stdlib_http 1 passed
|
||||
cargo test -p marqdo-cli --test stdlib_product 16 passed
|
||||
python3 tools/pack-stdlib.py PASS
|
||||
```
|
||||
|
||||
## Not covered
|
||||
|
||||
No HTTP/2, header folding, trailers, gzip transfer decoding, streaming
|
||||
message framing or request/response Resource adapter.
|
||||
Reference in new issue
Block a user