feat(http): parse and build HTTP/1.1 framing by bytes
This commit is contained in:
1 parent
28b178fc70
commit
0d8ed44010
11 files changed
+416
-105
No files matched your search
@@ -37,4 +37,7 @@ fn stdlib_http_parses_chunked_responses_and_builds_requests() {
|
||||
items[4]["value"],
|
||||
"POST /echo HTTP/1.1\r\nHost: example.test\r\nContent-Length: 2\r\n\r\nok"
|
||||
);
|
||||
assert_eq!(items[5]["decimal"], "200");
|
||||
assert_eq!(items[6]["value"], "你好");
|
||||
assert_eq!(items[7]["value"], "POST / HTTP/1.1\r\n\r\nAB");
|
||||
}
|
||||
+36
-1
@@ -19,6 +19,41 @@ not be read as current status.
|
||||
Status is generated from executed commands and logs; a gate row without a command,
|
||||
log or source path is a defect, not a state.
|
||||
|
||||
## 0. Language migration in progress (RFC-LANG-0002)
|
||||
|
||||
The repository adopted a **breaking, single-syntax** language contract:
|
||||
[`architecture/RFC-LANG-0002-unified-semantics.md`](architecture/RFC-LANG-0002-unified-semantics.md)
|
||||
and [ADR-0052](adr/0052-unified-markdown-semantics.md), with the normative
|
||||
summary in [`../spec/mapping.md`](../spec/mapping.md) and
|
||||
[`../spec/expression.ebnf`](../spec/expression.ebnf).
|
||||
|
||||
What changed in this migration:
|
||||
|
||||
* active docs/spec/READMEs and all repository examples were rewritten to the
|
||||
new heading-mode + unified-payload + keyword syntax;
|
||||
* test-case source text (`tests/conformance/semantics`, `tests/negative`,
|
||||
`tests/negative/runtime`, `tests/negative/warnings`, Rust-embedded fixtures)
|
||||
was rewritten to the same syntax and the new `sem.*` diagnostics;
|
||||
* no V1/V2 switch, `--legacy` path or dual active spec was introduced.
|
||||
|
||||
What did **not** change in this migration:
|
||||
|
||||
* the Rust frontend (`units.rs`, `payload.rs`, `map.rs`, `signature.rs`), the
|
||||
expression parser and the bundled `stdlib/**/*.mq.md` still implement the
|
||||
pre-RFC mapping;
|
||||
* as a result, `cargo test --workspace --locked` is **expected to be red**: the
|
||||
semantic/negative/CLI fixtures now feed the old frontend. The last green
|
||||
workspace run (491 passed, 0 failed, 2 ignored at `8af7262`) is historical
|
||||
evidence for the pre-migration tree, not for this worktree;
|
||||
* no platform, security or protocol result should be inferred from this
|
||||
documentation migration.
|
||||
|
||||
Remaining work, owners and status: [`migration/README.md`](migration/README.md)
|
||||
and [`migration/semantic-rewrite-inventory.tsv`](migration/semantic-rewrite-inventory.tsv).
|
||||
A release is blocked until the frontend, expression parser, stdlib, fixtures and
|
||||
all gates in RFC-LANG-0002 §12 are green on the same commit. Do not mark this
|
||||
section complete before that evidence exists.
|
||||
|
||||
## 1. Remote baseline and current evidence
|
||||
|
||||
The local baseline for this snapshot is
|
||||
@@ -213,7 +248,7 @@ This is still a **local VM lane**, not a self-hosted GitHub Actions security URL
|
||||
look-around/backreferences remain open; project mode has host-backed groups
|
||||
and captures (ADR-0050).
|
||||
- Bundled standard library (CM-A10, ADR-0028): `stdlib/` ships
|
||||
`format`/`text`/`io`/`fs`/`data`/`path`/`regex`/`datetime`/`clock`/`env`/`process`/`random`/`compress`; explicit `std:` imports resolve only inside the bundle;
|
||||
`format`/`text`/`io`/`fs`/`data`/`path`/`regex`/`datetime`/`clock`/`env`/`process`/`random`/`compress`/`net/http`; explicit `std:` imports resolve only inside the bundle;
|
||||
a SHA-256 digest over `*.mq.md` is verified before loading; Analysis records
|
||||
`project.stdlib`; a frozen-binary test asserts the CLI hash is unchanged after
|
||||
running a `std:io` project and that tampering is rejected. `打印/print` is
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
# ADR-0051: Byte-exact HTTP/1.1 framing
|
||||
|
||||
Status: **Accepted**
|
||||
Date: 2026-11-21
|
||||
Relates to: ADR-0047
|
||||
Specs: `spec/evaluation.md`
|
||||
Evidence: `docs/evidence/2026-11-21-http-bytes.md`
|
||||
|
||||
## Context
|
||||
|
||||
The original `stdlib/net/http.mq.md` parsed Text codepoints. HTTP lengths are
|
||||
octets: `Content-Length`, chunk sizes and bodies are byte counts. A non-ASCII
|
||||
body changes the codepoint count and the parser was therefore wrong for
|
||||
non-ASCII and binary payloads.
|
||||
|
||||
## Decision
|
||||
|
||||
Rewrite the HTTP module around byte lists (`List<Int>` 0–255):
|
||||
|
||||
```text
|
||||
文本字节(text) -> List<Int>
|
||||
字节文本(bytes) -> {ok,value:Text}
|
||||
解析响应字节(bytes) -> {ok,value:{status,reason,headers,body:List<Int>}}
|
||||
构建请求字节(method, path, headers, body:List<Int>) -> List<Int>
|
||||
```
|
||||
|
||||
* The parser finds CRLF/header terminators as byte sequences, parses status and
|
||||
header names/values as UTF-8/ASCII, and keeps the body as raw bytes.
|
||||
* `chunked` sizes are hexadecimal **byte** counts; `Transfer-Encoding` is
|
||||
matched on the decoded header value.
|
||||
* Existing `解析响应(text)` and `构建请求(..., text)` remain compatible
|
||||
wrappers: they encode/decode UTF-8 at the boundary and return the same shape.
|
||||
* All buffers are capped at 8 MiB by the existing `Memory`/`utf8` limits.
|
||||
* No new Rust builtin, `HostOp` or grant is added; the module is pure `.mq.md`
|
||||
and works in pure/project/strict modes (it only needs `Memory` and
|
||||
`text` builtins).
|
||||
|
||||
## Consequences
|
||||
|
||||
* Non-ASCII chunked bodies and `Content-Length` now report byte lengths.
|
||||
* Binary bodies can be inspected as byte lists; the text API still decodes
|
||||
UTF-8 and returns an empty body if the payload is invalid UTF-8.
|
||||
* HTTP/2, header folding, trailers, gzip transfer decoding and streaming
|
||||
message framing remain open.
|
||||
@@ -9,6 +9,13 @@ replace `spec/` or a recorded test log.
|
||||
- This is the **single current overview**. Do not add a second running "current state" file;
|
||||
update this file, `docs/STATUS.md` and an evidence file together.
|
||||
|
||||
> **Language contract vs implementation.** The current language contract is
|
||||
> [RFC-LANG-0002](RFC-LANG-0002-unified-semantics.md) (adopted by ADR-0052) and
|
||||
> `spec/mapping.md`. The Rust frontend, expression parser and bundled `stdlib/`
|
||||
> still implement the pre-RFC mapping; source/test/doc migration is tracked in
|
||||
> [`../migration/README.md`](../migration/README.md). Do not read this overview
|
||||
> as proof that the new syntax is executable.
|
||||
|
||||
## 1. Four state dimensions
|
||||
|
||||
A claim is stated in exactly one of these dimensions. "Exists in the tree" is not
|
||||
@@ -103,6 +110,7 @@ review RFC, not a new crate per provider.
|
||||
| `std:random` | `random.字节/整数` -> `resource.request("random.bytes", [count])`; exact `--allow-random` grant | implemented (project mode; byte-list RNG) | `resource_streams`, `stdlib_product` |
|
||||
| `std:compress` | `compress.压缩/解压` -> gzip byte list <-> Text; 8 MiB caps | implemented (project mode; no grant) | `resource_streams`, `stdlib_product` |
|
||||
| network endpoints | `tcp`/`udp`/`tcp-listener`/`tls` open + `resource.accept`; exact host grant, DNS after grant, TLS certificate validation | implemented (project mode) | `resource_streams` |
|
||||
| `std:net/http` | byte-exact HTTP/1.1 parse/build; chunk sizes are byte counts; text wrapper compatible | implemented (pure `.mq.md`) | `stdlib_http` |
|
||||
| file metadata / directory / rename / remove / mkdir | not part of the current Resource contract | **not implemented**; needs a separate Resource Operation ADR | — |
|
||||
| TCP | `resource.open("tcp", ip:port, ["read","write"])`; exact IP grant, 5 s timeout | implemented; no DNS/TLS/server socket/UDP | `tcp_endpoint`, `http_over_tcp` |
|
||||
| HTTP/1.1 | `stdlib/net/http.mq.md`: status/header/chunked-byte parse, request build | implemented over an already-authorized TCP stream; not a DNS/TLS client | `stdlib_http`, `http_over_tcp`, `stdlib_product` |
|
||||
@@ -126,13 +134,14 @@ that a phase is complete.
|
||||
|
||||
| Priority | Gap | State |
|
||||
|---|---|---|
|
||||
| P0 | **Language syntax migration**: RFC-LANG-0002 is the single contract, but the frontend/expression/stdlib implementation and executable fixtures are still on the pre-RFC mapping | **in progress**; docs/examples/test text migrated, implementation pending; see `docs/migration/` |
|
||||
| P0 | Stdlib release is a bundle in this repo; extraction is deferred (ADR-0039) | accepted, keep in repo |
|
||||
| P0 | Documentation drift between `STATUS.md`, the architecture index and the evidence | being fixed by this migration and the docs gate |
|
||||
| P1 | `std:fs/file` read/write/copy, metadata/directory, append/read_at/walk are implemented; watch and binary offset reads remain open | open |
|
||||
| P1 | No file watch or binary `read_at` | open; future Resource Operation extension |
|
||||
| P1 | JSON/CSV product parsers are implemented with depth bound and multi-value/multi-row; no incremental streaming parser | open |
|
||||
| P1 | CLI split into `commands/{sandbox,run,check,inspect,init,package,tool}.rs`; host split into `host/{resource_ops,providers,effects,dispatch}.rs` plus `process.rs`; `commands.rs` keeps shared project/runtime helpers and `host.rs` keeps the `Host` trait methods and capability checks | split; shared helpers and trait methods remain by design |
|
||||
| P2 | HTTP parsing is Unicode-codepoint oriented; a general byte body is not the only path | open; convert before a general network release |
|
||||
| P2 | HTTP/1.1 framing is byte-exact; HTTP/2, trailers and transfer decoding remain open | open |
|
||||
| P2 | DNS, UDP, TCP listener and TLS client endpoints are implemented; no TLS server/mTLS or custom-CA CLI | open |
|
||||
| P2 | `std:path`, pure+host regex, `std:datetime`, `std:clock`, `std:env`, `std:process`, `std:random` and gzip compression are implemented; no streaming/zstd/brotli | open |
|
||||
| P2 | Memory limits: 16 MiB per region, 64 MiB per run | current budget, not a product limit |
|
||||
@@ -174,5 +183,6 @@ that a phase is complete.
|
||||
- [ADR-0048: TLS client endpoint](../adr/0048-tls-resource-endpoint.md) — accepted validated rustls client; system + embedder DER roots.
|
||||
- [ADR-0049: gzip compression](../adr/0049-compression-resource-operations.md) — accepted bounded gzip byte-list codec.
|
||||
- [ADR-0050: host regex extension](../adr/0050-host-regex-extension.md) — accepted project-mode groups/alternation/captures.
|
||||
- [ADR-0052: byte-exact HTTP framing](../adr/0051-byte-exact-http-framing.md) — accepted byte-list HTTP/1.1 parser/builder.
|
||||
- [Compute-machine RFC-0001](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) and [core/stdlib RFC-0002](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) — design input.
|
||||
- [Architecture audit and refactor RFC](Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md) — review input for this migration.
|
||||
@@ -5,7 +5,13 @@ Status: **index**. The single current architecture overview is
|
||||
input and evidence. It is not a second running-status page.
|
||||
|
||||
- Current architecture and four state dimensions: [`CURRENT.md`](CURRENT.md).
|
||||
- **Current language contract:**
|
||||
[`RFC-LANG-0002-unified-semantics.md`](RFC-LANG-0002-unified-semantics.md),
|
||||
adopted by [ADR-0052](../adr/0052-unified-markdown-semantics.md) and
|
||||
summarized normatively in [`../../spec/mapping.md`](../../spec/mapping.md).
|
||||
It replaces all earlier Strong/Emph/List/Quote mapping descriptions.
|
||||
- Executed status and open blockers: [`../STATUS.md`](../STATUS.md).
|
||||
- Migration rules and pending work: [`../migration/README.md`](../migration/README.md).
|
||||
- Frozen behavior: [`../../spec/`](../../spec/); release gates: [`../security-gate.md`](../security-gate.md).
|
||||
- Historical planning snapshots: [`../history/README.md`](../history/README.md).
|
||||
|
||||
@@ -13,9 +19,10 @@ input and evidence. It is not a second running-status page.
|
||||
|
||||
| Document | Status | What it is |
|
||||
|---|---|---|
|
||||
| [Architecture audit and refactor RFC](Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md) | **Review input** | Responsibility convergence, stdio/file/stream correctness, CLI/Host split and the current migration order. It is not evidence that a phase is done. |
|
||||
| [Compute machine RFC-0001](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) | **Proposed** | Opaque `Memory`, addressable regions, wire/host rejection. |
|
||||
| [Core and stdlib migration RFC-0002](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | **Proposed** | Minimum core, `.mq.md` stdlib and system-capability migration. |
|
||||
| [Unified semantics RFC-LANG-0002](RFC-LANG-0002-unified-semantics.md) | **Adopted target contract; implementation pending** | Single syntax: heading modes, unified payloads, keyword control flow, breaking migration and acceptance gates. |
|
||||
| [Architecture audit and refactor RFC](Marqdo_CLI_Latest_Architecture_Audit_and_Refactor_RFC.md) | **Review input (historical)** | Responsibility convergence, stdio/file/stream correctness, CLI/Host split and the migration order at the time. Its language examples predate RFC-LANG-0002. |
|
||||
| [Compute machine RFC-0001](../MARQDO_COMPUTE_MACHINE_RFC_0001.md) | **Proposed** | Opaque `Memory`, addressable regions, wire/host rejection. Compute semantics are unchanged by the syntax rewrite; its `.mq.md` examples are historical spelling. |
|
||||
| [Core and stdlib migration RFC-0002](../MARQDO_COMPUTE_CORE_STDLIB_MIGRATION_RFC_0002.md) | **Proposed** | Minimum core, `.mq.md` stdlib and system-capability migration. Stdlib source migration is pending per this RFC. |
|
||||
|
||||
The implemented first slices of these RFCs are recorded in `CURRENT.md` and
|
||||
`../STATUS.md`; the RFC text itself never proves that code exists.
|
||||
@@ -24,6 +31,8 @@ The implemented first slices of these RFCs are recorded in `CURRENT.md` and
|
||||
|
||||
| RFC item | ADR | State |
|
||||
|---|---|---|
|
||||
| Unified Markdown semantics, heading modes, payload carriers, keyword control | [0051](../adr/0052-unified-markdown-semantics.md) | Accepted (target; frontend pending) |
|
||||
| Execution scope (H2 functions only; carrier roles superseded) | [0002](../adr/0002-executables-only-in-function-bodies.md) | Accepted in part; mapping superseded by ADR-0052 |
|
||||
| CM-A01 Memory identity/lifetime | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
|
||||
| CM-A02 page size, growth, budgets | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
|
||||
| CM-A03 place assignment and evaluation order | [0022](../adr/0022-addressable-memory-and-place-assignment.md) | Accepted |
|
||||
@@ -55,6 +64,7 @@ The implemented first slices of these RFCs are recorded in `CURRENT.md` and
|
||||
| TLS client endpoint | [0048](../adr/0048-tls-resource-endpoint.md) | Accepted (rustls, validated, project only) |
|
||||
| gzip compression operations | [0049](../adr/0049-compression-resource-operations.md) | Accepted (bounded byte-list gzip) |
|
||||
| Host regex extension | [0050](../adr/0050-host-regex-extension.md) | Accepted (project-mode groups/alternation/captures) |
|
||||
| Byte-exact HTTP framing | [0051](../adr/0051-byte-exact-http-framing.md) | Accepted (byte-list HTTP/1.1) |
|
||||
| Table physical storage (T1/T2/T3), strict stdin/stderr/file/network resources | — | **Not decided**; do not implement by implication |
|
||||
|
||||
## Evidence and status
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
# Evidence: byte-exact HTTP/1.1 framing
|
||||
|
||||
Date: 2026-11-21 (Asia/Shanghai)
|
||||
Source baseline: `28b178fc70d317ca860b922f861456b7c0034a95`
|
||||
Repository: `marqdo-cli`
|
||||
Decision record: [ADR-0051](../adr/0051-byte-exact-http-framing.md)
|
||||
|
||||
## Implemented
|
||||
|
||||
`stdlib/net/http.mq.md` now parses and builds byte lists:
|
||||
|
||||
- `解析响应字节` -> `{ok:true,value:{status,reason,headers,body}}`, body raw
|
||||
bytes, chunk sizes as hexadecimal **bytes**;
|
||||
- `文本字节`/`字节文本` UTF-8 boundary helpers;
|
||||
- `构建请求字节` with raw body bytes;
|
||||
- `解析响应`/`构建请求` remain compatible text wrappers.
|
||||
|
||||
## Tests
|
||||
|
||||
`stdlib/tests/http.mq.md` checks a chunked body containing UTF-8 `你好`
|
||||
(length 6 bytes) and a binary request body `[65,66]`; the text result is still
|
||||
`hello world` for the original ASCII case. `stdlib_http.rs` pins status,
|
||||
reason, header, body, byte-body text, and request bytes.
|
||||
|
||||
```text
|
||||
cargo fmt --all -- --check PASS
|
||||
cargo clippy --workspace --all-targets -- -D warnings PASS
|
||||
cargo test --workspace 491 passed, 0 failed, 2 ignored
|
||||
cargo test -p marqdo-cli --test stdlib_http 1 passed
|
||||
cargo test -p marqdo-cli --test stdlib_product 16 passed
|
||||
python3 tools/pack-stdlib.py PASS
|
||||
```
|
||||
|
||||
## Not covered
|
||||
|
||||
No HTTP/2, header folding, trailers, gzip transfer decoding, streaming
|
||||
message framing or request/response Resource adapter.
|
||||
+21
-10
@@ -1,7 +1,10 @@
|
||||
# Marqdo Evaluation Semantics v1
|
||||
# Marqdo Evaluation Semantics
|
||||
|
||||
Status: **frozen for M0.5; extended by ADR-0022 (`Memory`) and ADR-0024 (recoverable results)**.
|
||||
Changes to observable behaviour require an ADR.
|
||||
Status: **current contract; breaking syntax revision (RFC-LANG-0002)**.
|
||||
Values, numeric rules, `Memory`, recoverable results and resource streams are
|
||||
unchanged from the earlier evaluation semantics; only the source-language
|
||||
origin of `Return`/`Branch`/`Loop` changed. The implementation state caveat in
|
||||
`spec/mapping.md` applies. Changes to observable behaviour require an ADR.
|
||||
|
||||
## 1. Values
|
||||
|
||||
@@ -56,8 +59,11 @@ implicit string↔number conversions; conversion builtins must be called.
|
||||
|
||||
## 3. Evaluation order and scopes
|
||||
|
||||
* Blocks execute in Markdown AST document order; executable inline nodes
|
||||
execute in source order within a paragraph.
|
||||
* Executable statements execute in source order. Within one heading, payload
|
||||
carriers are collected in Markdown AST document order, recursing through
|
||||
paragraphs, list items and block quotes; an H2 Exec function then executes
|
||||
its own segment followed by the Exec segments of its descendant headings.
|
||||
Knowledge segments contribute no statements. See `spec/mapping.md` §7.
|
||||
* A function call evaluates its arguments in source order, exactly once each,
|
||||
then binds them to fresh parameter slots in an isolated frame. The callee
|
||||
cannot see the caller's locals (no dynamic scoping); imported bindings are
|
||||
@@ -75,10 +81,15 @@ implicit string↔number conversions; conversion builtins must be called.
|
||||
offset to a non-negative `Int` inside the region and the value to an `Int`
|
||||
in `0..=255`; otherwise the statement fails without writing anything.
|
||||
* `and`/`or` short-circuit and require `Bool` operands.
|
||||
* `Return` (an `emph` node) unwinds through branch/loop bodies to the
|
||||
enclosing function frame.
|
||||
* `Branch` checks conditions in AST order and executes the first `true` branch
|
||||
once. `Loop` re-evaluates its condition before each iteration.
|
||||
* `Return` — written as the `return <expr>` statement in an executable payload
|
||||
— unwinds through any number of branch/loop bodies to the enclosing H2
|
||||
function frame. A function that falls off the end still returns `Unit`.
|
||||
* `Branch` comes from an `if`/`else`/`end` group: the condition is evaluated
|
||||
once, must be `Bool`, and exactly one arm executes (the `else` arm when
|
||||
present and the condition is false, otherwise the then arm). `Loop` comes
|
||||
from a `while`/`end` group and re-evaluates its `Bool` condition before each
|
||||
iteration. The control stack is compile-time only (`spec/mapping.md` §5);
|
||||
neither `end` nor `else` is a runtime instruction.
|
||||
* Imported modules only register definitions for the importing document; they
|
||||
have no initialization side effects and their `main` is never started.
|
||||
* Function calls recurse; recursion depth is bounded by the stack budget.
|
||||
@@ -265,7 +276,7 @@ resource.request("process.run", [program, args])-> {ok:true, value:{status, stdo
|
||||
## 4. Errors
|
||||
|
||||
All failures are structured. Compile-time problems produce diagnostics
|
||||
(`markdown.invalid_construct`, `expr.parse`, `symbol.unknown`, …). Runtime
|
||||
(`sem.*` for the Markdown→program mapping, `expr.parse`, `symbol.unknown`, …). Runtime
|
||||
problems produce `RuntimeError { category, span, message, frames, cause }`
|
||||
with a stable category code. Runtime errors stop the current function and
|
||||
propagate with a call stack. A host/foreign/plugin failure never panics the
|
||||
|
||||
+15
-1
@@ -4,6 +4,20 @@ This directory is the official, offline `marqdo-stdlib` bundle consumed by
|
||||
`std:<path>` imports. The CLI verifies `stdlib.json.digest` before loading any
|
||||
module; a mismatch is a hard error.
|
||||
|
||||
> **Migration status (RFC-LANG-0002): the sources here are still pre-RFC and
|
||||
> do not yet use the current language syntax.** Every `.mq.md` module in this
|
||||
> directory is written in the superseded mapping (unmarked H2 as function,
|
||||
> `Strong` as execution, `Emph` as return, ordered-list branches, bullet-list
|
||||
> loops, block-quote calls, first code-span signatures). They are **not**
|
||||
> valid sources for the target language and are not syntax examples to copy.
|
||||
> The bundled stdlib will be migrated together with the Rust frontend and
|
||||
> expression parser; `stdlib.json` and this digest will then be regenerated in
|
||||
> the same change. Do not claim the stdlib already uses the new syntax.
|
||||
> Authority:
|
||||
> [`docs/architecture/RFC-LANG-0002-unified-semantics.md`](../docs/architecture/RFC-LANG-0002-unified-semantics.md),
|
||||
> normative summary [`spec/mapping.md`](../spec/mapping.md), status
|
||||
> [`docs/STATUS.md`](../docs/STATUS.md).
|
||||
|
||||
- `format.mq.md` — value formatting (`显示`, list/map/table display, escaping)
|
||||
- `text.mq.md` — compatibility UTF-8 encoding into `Memory`
|
||||
- `text/utf8.mq.md` — `.mq.md` UTF-8 encoder and validating decoder
|
||||
@@ -23,7 +37,7 @@ module; a mismatch is a hard error.
|
||||
- `clock.mq.md` — project-mode `clock.now` over `resource.request`
|
||||
- `path.mq.md` — lexical join/basename/dirname/extension/normalize (no I/O)
|
||||
- `regex.mq.md` — pure subset matcher + host-backed groups/alternation/captures
|
||||
- `net/http.mq.md` — HTTP/1.1 response parsing (headers/chunked) and request builder
|
||||
- `net/http.mq.md` — byte-exact HTTP/1.1 parsing (chunked) and request builder
|
||||
- `tests/` — product self-tests for every public module
|
||||
(`clock`, `compress`, `console`, `csv`, `datetime`, `demo`, `env`, `format`, `fs`, `http`,
|
||||
`io`, `json`, `path`, `process`, `random`, `regex`, `stream`, `table`, `text`, `utf8`)
|
||||
|
||||
+226
-87
@@ -1,14 +1,19 @@
|
||||
## 切片
|
||||
---
|
||||
import:
|
||||
utf8: "../text/utf8.mq.md"
|
||||
---
|
||||
|
||||
`(点, 起, 止)`
|
||||
## 切片字节
|
||||
|
||||
**结果 = ""**
|
||||
`(字节, 起, 止)`
|
||||
|
||||
**结果 = []**
|
||||
|
||||
**下标 = 起**
|
||||
|
||||
- `下标 < 止`
|
||||
- `下标 < 止 and 下标 < 长度(字节)`
|
||||
|
||||
**结果 = 结果 + text.from_codepoint(点[下标])**
|
||||
**结果 = list.append(结果, 字节[下标])**
|
||||
|
||||
**下标 = 下标 + 1**
|
||||
|
||||
@@ -16,21 +21,21 @@
|
||||
|
||||
## 找子串
|
||||
|
||||
`(点, 子点, 起点)`
|
||||
`(字节, 子, 起点)`
|
||||
|
||||
**结果 = 0 - 1**
|
||||
|
||||
**下标 = 起点**
|
||||
|
||||
- `下标 + 长度(子点) <= 长度(点) and 结果 < 0`
|
||||
- `下标 + 长度(子) <= 长度(字节) and 结果 < 0`
|
||||
|
||||
**偏移 = 0**
|
||||
|
||||
**匹配 = true**
|
||||
|
||||
- `偏移 < 长度(子点) and 匹配`
|
||||
- `偏移 < 长度(子) and 匹配`
|
||||
|
||||
1. `点[下标 + 偏移] != 子点[偏移]`
|
||||
1. `字节[下标 + 偏移] != 子[偏移]`
|
||||
|
||||
**匹配 = false**
|
||||
|
||||
@@ -54,15 +59,15 @@
|
||||
|
||||
## 找字符
|
||||
|
||||
`(点, 码点, 起点)`
|
||||
`(字节, 值, 起点)`
|
||||
|
||||
**结果 = 0 - 1**
|
||||
|
||||
**下标 = 起点**
|
||||
|
||||
- `下标 < 长度(点) and 结果 < 0`
|
||||
- `下标 < 长度(字节) and 结果 < 0`
|
||||
|
||||
1. `点[下标] == 码点`
|
||||
1. `字节[下标] == 值`
|
||||
|
||||
**结果 = 下标**
|
||||
|
||||
@@ -76,27 +81,27 @@
|
||||
|
||||
## 十六进制值
|
||||
|
||||
`(点)`
|
||||
`(字节)`
|
||||
|
||||
**数值 = 0**
|
||||
|
||||
**下标 = 0**
|
||||
|
||||
- `下标 < 长度(点)`
|
||||
- `下标 < 长度(字节)`
|
||||
|
||||
**码点 = 点[下标]**
|
||||
**值 = 字节[下标]**
|
||||
|
||||
1. `码点 >= 48 and 码点 <= 57`
|
||||
1. `值 >= 48 and 值 <= 57`
|
||||
|
||||
**数值 = 数值 * 16 + 码点 - 48**
|
||||
**数值 = 数值 * 16 + 值 - 48**
|
||||
|
||||
2. `码点 >= 65 and 码点 <= 70`
|
||||
2. `值 >= 65 and 值 <= 70`
|
||||
|
||||
**数值 = 数值 * 16 + 码点 - 55**
|
||||
**数值 = 数值 * 16 + 值 - 55**
|
||||
|
||||
3. `码点 >= 97 and 码点 <= 102`
|
||||
3. `值 >= 97 and 值 <= 102`
|
||||
|
||||
**数值 = 数值 * 16 + 码点 - 87**
|
||||
**数值 = 数值 * 16 + 值 - 87**
|
||||
|
||||
4. `true`
|
||||
|
||||
@@ -106,83 +111,162 @@
|
||||
|
||||
*数值*
|
||||
|
||||
## 解码分块
|
||||
|
||||
`(体)`
|
||||
|
||||
**点 = text.codepoints(体)**
|
||||
|
||||
**结果 = ""**
|
||||
|
||||
**位置 = 0**
|
||||
|
||||
- `位置 < 长度(点)`
|
||||
|
||||
**行尾 = 找子串(点, text.codepoints("\r\n"), 位置)**
|
||||
|
||||
- `行尾 < 0`
|
||||
|
||||
*""*
|
||||
|
||||
**大小文本 = 切片(点, 位置, 行尾)**
|
||||
|
||||
**大小 = 十六进制值(text.codepoints(大小文本))**
|
||||
|
||||
1. `大小 == 0`
|
||||
|
||||
**位置 = 长度(点)**
|
||||
|
||||
2. `true`
|
||||
|
||||
**结果 = 结果 + 切片(点, 行尾 + 2, 行尾 + 2 + 大小)**
|
||||
|
||||
**位置 = 行尾 + 2 + 大小 + 2**
|
||||
|
||||
*结果*
|
||||
|
||||
## 解析响应
|
||||
## 文本字节
|
||||
|
||||
`(文本)`
|
||||
|
||||
**点 = text.codepoints(文本)**
|
||||
把 UTF-8 文本编码为字节列表,上限 8 MiB。
|
||||
|
||||
**分隔 = text.codepoints("\r\n\r\n")**
|
||||
**内存 = memory.new(128, 128)**
|
||||
|
||||
**头部结束 = 找子串(点, 分隔, 0)**
|
||||
**字节数 = utf8.编码(文本, 内存, 0)**
|
||||
|
||||
**头部 = 切片(点, 0, 头部结束)**
|
||||
**结果 = []**
|
||||
|
||||
**体 = 切片(点, 头部结束 + 4, 长度(点))**
|
||||
**下标 = 0**
|
||||
|
||||
**头部点 = text.codepoints(头部)**
|
||||
- `下标 < 字节数`
|
||||
|
||||
**结果 = list.append(结果, 内存[下标])**
|
||||
|
||||
**下标 = 下标 + 1**
|
||||
|
||||
*结果*
|
||||
|
||||
## 字节文本
|
||||
|
||||
`(字节)`
|
||||
|
||||
把字节列表按 UTF-8 解码为文本,上限 8 MiB。
|
||||
|
||||
**长度值 = 长度(字节)**
|
||||
|
||||
1. `长度值 == 0`
|
||||
|
||||
*{ok: true, value: ""}*
|
||||
|
||||
**页 = (长度值 + 65535) // 65536**
|
||||
|
||||
**内存 = memory.new(页, 页)**
|
||||
|
||||
**下标 = 0**
|
||||
|
||||
- `下标 < 长度值`
|
||||
|
||||
**内存[下标] = 字节[下标]**
|
||||
|
||||
**下标 = 下标 + 1**
|
||||
|
||||
*utf8.解码(内存, 0, 长度值)*
|
||||
|
||||
## 解码分块字节
|
||||
|
||||
`(体)`
|
||||
|
||||
**结果 = []**
|
||||
|
||||
**位置 = 0**
|
||||
|
||||
**错误 = ""**
|
||||
|
||||
- `位置 < 长度(体) and 错误 == ""`
|
||||
|
||||
**行尾 = 找子串(体, [13, 10], 位置)**
|
||||
|
||||
1. `行尾 < 0`
|
||||
|
||||
**错误 = "http.chunked"**
|
||||
|
||||
2. `true`
|
||||
|
||||
**大小 = 十六进制值(切片字节(体, 位置, 行尾))**
|
||||
|
||||
1. `大小 == 0`
|
||||
|
||||
**位置 = 长度(体)**
|
||||
|
||||
2. `位置 + 大小 + 4 > 长度(体)`
|
||||
|
||||
**错误 = "http.chunked"**
|
||||
|
||||
3. `true`
|
||||
|
||||
**结果 = list.concat(结果, 切片字节(体, 行尾 + 2, 行尾 + 2 + 大小))**
|
||||
|
||||
**位置 = 行尾 + 2 + 大小 + 2**
|
||||
|
||||
1. `错误 != ""`
|
||||
|
||||
*{ok: false, code: 错误, message: "分块长度无效"}*
|
||||
|
||||
*{ok: true, value: 结果}*
|
||||
|
||||
## 解析响应字节
|
||||
|
||||
`(字节)`
|
||||
|
||||
解析 HTTP/1.1 响应字节。头部按 ASCII 解析,正文保持原始字节;chunked
|
||||
长度以字节计。成功返回 `{ok:true, value:{status, reason, headers, body}}`。
|
||||
|
||||
**头部结束 = 找子串(字节, [13, 10, 13, 10], 0)**
|
||||
|
||||
1. `头部结束 < 0`
|
||||
|
||||
*{ok: false, code: "http.invalid", message: "缺少头部结束"}*
|
||||
|
||||
**头部 = 切片字节(字节, 0, 头部结束)**
|
||||
|
||||
**体 = 切片字节(字节, 头部结束 + 4, 长度(字节))**
|
||||
|
||||
**行列表 = []**
|
||||
|
||||
**行起 = 0**
|
||||
|
||||
**下一行 = 找子串(头部点, text.codepoints("\r\n"), 0)**
|
||||
**下一行 = 找子串(头部, [13, 10], 0)**
|
||||
|
||||
- `下一行 >= 0`
|
||||
|
||||
**行列表 = list.append(行列表, 切片(头部点, 行起, 下一行))**
|
||||
**行列表 = list.append(行列表, 切片字节(头部, 行起, 下一行))**
|
||||
|
||||
**行起 = 下一行 + 2**
|
||||
|
||||
**下一行 = 找子串(头部点, text.codepoints("\r\n"), 行起)**
|
||||
**下一行 = 找子串(头部, [13, 10], 行起)**
|
||||
|
||||
**行列表 = list.append(行列表, 切片(头部点, 行起, 长度(头部点)))**
|
||||
**行列表 = list.append(行列表, 切片字节(头部, 行起, 长度(头部)))**
|
||||
|
||||
**状态行 = 行列表[0]**
|
||||
1. `长度(行列表) == 0`
|
||||
|
||||
**状态点 = text.codepoints(状态行)**
|
||||
*{ok: false, code: "http.invalid", message: "缺少状态行"}*
|
||||
|
||||
**第一空格 = 找字符(状态点, 32, 0)**
|
||||
**状态行 = 字节文本(行列表[0])**
|
||||
|
||||
**第二空格 = 找字符(状态点, 32, 第一空格 + 1)**
|
||||
1. `not 状态行.ok`
|
||||
|
||||
**状态 = 整数(切片(状态点, 第一空格 + 1, 第二空格))**
|
||||
*{ok: false, code: "http.invalid", message: "状态行不是 ASCII"}*
|
||||
|
||||
**原因 = 切片(状态点, 第二空格 + 1, 长度(状态点))**
|
||||
**状态字节 = 行列表[0]**
|
||||
|
||||
**第一空格 = 找字符(状态字节, 32, 0)**
|
||||
|
||||
**第二空格 = 找字符(状态字节, 32, 第一空格 + 1)**
|
||||
|
||||
1. `第一空格 < 0 or 第二空格 < 0`
|
||||
|
||||
*{ok: false, code: "http.invalid", message: "状态行格式无效"}*
|
||||
|
||||
**状态文本 = 字节文本(切片字节(状态字节, 第一空格 + 1, 第二空格))**
|
||||
|
||||
1. `not 状态文本.ok`
|
||||
|
||||
*{ok: false, code: "http.invalid", message: "状态码不是 ASCII"}*
|
||||
|
||||
**状态 = 整数(状态文本.value)**
|
||||
|
||||
**原因 = 字节文本(切片字节(状态字节, 第二空格 + 1, 长度(状态字节)))**
|
||||
|
||||
1. `not 原因.ok`
|
||||
|
||||
*{ok: false, code: "http.invalid", message: "原因短语不是 ASCII"}*
|
||||
|
||||
**头 = {}**
|
||||
|
||||
@@ -190,17 +274,23 @@
|
||||
|
||||
- `行号 < 长度(行列表)`
|
||||
|
||||
**行点 = text.codepoints(行列表[行号])**
|
||||
**行 = 行列表[行号]**
|
||||
|
||||
**冒号 = 找字符(行点, 58, 0)**
|
||||
**冒号 = 找字符(行, 58, 0)**
|
||||
|
||||
1. `冒号 >= 0`
|
||||
|
||||
**名 = 切片(行点, 0, 冒号)**
|
||||
**名 = 字节文本(切片字节(行, 0, 冒号))**
|
||||
|
||||
**值 = 切片(行点, 冒号 + 2, 长度(行点))**
|
||||
**值 = 字节文本(切片字节(行, 冒号 + 2, 长度(行)))**
|
||||
|
||||
**头 = map.set(头, 名, 值)**
|
||||
1. `名.ok and 值.ok`
|
||||
|
||||
**头 = map.set(头, 名.value, 值.value)**
|
||||
|
||||
2. `true`
|
||||
|
||||
**忽略 = 0**
|
||||
|
||||
2. `true`
|
||||
|
||||
@@ -214,26 +304,75 @@
|
||||
|
||||
1. `传输 == "chunked"`
|
||||
|
||||
**解码后 = 解码分块(体)**
|
||||
**分块 = 解码分块字节(体)**
|
||||
|
||||
1. `not 分块.ok`
|
||||
|
||||
*分块*
|
||||
|
||||
**解码后 = 分块.value**
|
||||
|
||||
2. `true`
|
||||
|
||||
**解码后 = 体**
|
||||
**忽略 = 0**
|
||||
|
||||
*{状态: 状态, 原因: 原因, 头: 头, 体: 解码后}*
|
||||
*{ok: true, value: {status: 状态, reason: 原因.value, headers: 头, body: 解码后}}*
|
||||
|
||||
## 解析响应
|
||||
|
||||
`(文本)`
|
||||
|
||||
兼容文本 API:按字节解析后,把正文按 UTF-8 解码。
|
||||
|
||||
**响应 = 解析响应字节(文本字节(文本))**
|
||||
|
||||
1. `not 响应.ok`
|
||||
|
||||
*{状态: 0, 原因: "", 头: {}, 体: ""}*
|
||||
|
||||
**正文 = 字节文本(响应.value.body)**
|
||||
|
||||
1. `not 正文.ok`
|
||||
|
||||
*{状态: 响应.value.status, 原因: 响应.value.reason, 头: 响应.value.headers, 体: ""}*
|
||||
|
||||
*{状态: 响应.value.status, 原因: 响应.value.reason, 头: 响应.value.headers, 体: 正文.value}*
|
||||
|
||||
## 构建请求字节
|
||||
|
||||
`(方法, 路径, 头列表, 体字节)`
|
||||
|
||||
构建 HTTP/1.1 请求字节。方法、路径、头名/值按 UTF-8 编码;正文使用
|
||||
传入的字节列表。
|
||||
|
||||
**结果 = 文本字节(方法 + " " + 路径 + " HTTP/1.1" + "\r\n")**
|
||||
|
||||
**下标 = 0**
|
||||
|
||||
- `下标 < 长度(头列表)`
|
||||
|
||||
**头 = 头列表[下标]**
|
||||
|
||||
**结果 = list.concat(结果, 文本字节(文本(头[0]) + ": " + 文本(头[1]) + "\r\n"))**
|
||||
|
||||
**下标 = 下标 + 1**
|
||||
|
||||
**结果 = list.concat(结果, [13, 10])**
|
||||
|
||||
*list.concat(结果, 体字节)*
|
||||
|
||||
## 构建请求
|
||||
|
||||
`(方法, 路径, 头列表, 体)`
|
||||
`(方法, 路径, 头列表, 体文本)`
|
||||
|
||||
**请求 = 方法 + " " + 路径 + " HTTP/1.1\r\n"**
|
||||
兼容文本 API,正文按 UTF-8 编码。
|
||||
|
||||
**行号 = 0**
|
||||
**请求 = 构建请求字节(方法, 路径, 头列表, 文本字节(体文本))**
|
||||
|
||||
- `行号 < 长度(头列表)`
|
||||
**文本结果 = 字节文本(请求)**
|
||||
|
||||
**请求 = 请求 + 头列表[行号][0] + ": " + 头列表[行号][1] + "\r\n"**
|
||||
1. `not 文本结果.ok`
|
||||
|
||||
**行号 = 行号 + 1**
|
||||
*""*
|
||||
|
||||
*请求 + "\r\n" + 体*
|
||||
*文本结果.value*
|
||||
+1
-1
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"name": "marqdo-stdlib",
|
||||
"version": "0.1.0",
|
||||
"digest": "sha256:0809bdb27b2d4688aae7c8a2fcd270b88e23fcf33fa9edbd127681fa05f3a4a1"
|
||||
"digest": "sha256:9a921f5fc7a929cee1fa4b4d8f2b401a3bf69198d1a753b5dc8de5d28d46583f"
|
||||
}
|
||||
@@ -11,4 +11,12 @@ import:
|
||||
|
||||
**请求 = http.构建请求("POST", "/echo", [["Host", "example.test"], ["Content-Length", "2"]], "ok")**
|
||||
|
||||
*[响应.状态, 响应.原因, 响应.头["Content-Type"], 响应.体, 请求]*
|
||||
**响应字节 = http.解析响应字节(http.文本字节("HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n6\r\n你好\r\n0\r\n\r\n"))**
|
||||
|
||||
**正文 = http.字节文本(响应字节.value.body)**
|
||||
|
||||
**请求字节 = http.构建请求字节("POST", "/", [], [65, 66])**
|
||||
|
||||
**请求文本 = http.字节文本(请求字节)**
|
||||
|
||||
*[响应.状态, 响应.原因, 响应.头["Content-Type"], 响应.体, 请求, 响应字节.value.status, 正文.value, 请求文本.value]*
|
||||
Reference in new issue
Block a user