Files
marqdo-cli/spec/mapping.md
T
cflmy b38b769932 fix(frontend): enforce function-wide params position
The concurrent mapper tracked the first params declaration in the Mapper (so
duplicates across heading segments were caught) but checked 'first payload'
with a per-segment vector index. A params(...) in a later ### @exec segment
was therefore accepted even after an earlier segment had executed statements,
and a duplicate could even be reported as a position error.

- track whether any statement has been lowered for the function and require
  params(...) to come before it, across all heading segments
- check duplicates first, then execution order, then the in-segment index
- add negative fixtures 48 (after an earlier segment) and 49 (duplicate
  across segments); update the corpus count and spec wording

Verified: cargo test --workspace --locked green (62 targets).
2026-11-21 15:36:30 +08:00

404 lines
19 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Marqdo AST → Program Mapping
Status: **current language contract, breaking rewrite (RFC-LANG-0002)**. This
file replaces the earlier Strong/Emph/List/Quote mapping in place; there is no
second active mapping and no per-file syntax selector.
> **Implementation state:** implemented by `marqdo-frontend` and
> `marqdo-expression`; the workspace gate is green at `ffd1cc9` (see
> `docs/STATUS.md` and `docs/evidence/2026-11-21-semantics-v2-green.md`).
Authority: `docs/architecture/RFC-LANG-0002-unified-semantics.md`.
Markdown parsing remains `marqdo.md.v1` (`spec/markdown-profile.md`); only the
Markdown→program mapping changes.
## 0. The mapping in one page
1. **Headings declare the semantic mode.** `@exec` and `@knowledge` may prefix
any H1–H6 heading. Unmarked headings inherit the nearest ancestor heading's
mode. The document root is Knowledge.
2. **Only an Exec H2 is a callable function.** H1 is a namespace/top-level
unit, H2 Exec is a function, H2 Knowledge is an inert knowledge unit, and
H3–H6 are sections that never create callable functions.
3. **Closed carriers carry one statement each.** In an Exec scope, `Strong`,
`Emph` and inline `Code` are interchangeable carriers of one payload. They
no longer mean execute/return/lazy respectively.
4. **Keywords inside the payload decide the operation.** `if`, `else`, `while`,
`end`, `return`, `params(...)`, assignment and expression statements. Lists,
block quotes and paragraphs are document containers only.
5. **Errors never fall back to prose.** Once a payload is extracted from an
Exec scope it must parse fully; a failure is a diagnostic.
6. **Control blocks never cross a heading segment.** Every `if`/`while` must be
closed by `end` inside the same heading's direct body. A `return` unwinds
the whole H2 function.
Two rules override everything else:
* **Node type and structural position decide whether a node is a payload
candidate**, never the success or failure of parsing and never natural
language.
* **Invalid payloads are hard errors**, never "ordinary Markdown".
## 1. Document regions and heading modes
### 1.1 Declaring a mode
A heading mode declaration is recognized only when it is the exact prefix of
the heading's **first plain Text child**, followed by whitespace and a non-empty
heading name:
```ebnf
heading_mode_prefix = "@exec" | "@knowledge" ;
heading_decl = heading_mode_prefix , whitespace , nonempty_title ;
```
The two legal words are case-sensitive and complete. The rules:
| Source | Result |
|---|---|
| `## @exec main` | Exec heading, semantic name `main` |
| `## @knowledge 原理` | Knowledge heading, semantic name `原理` |
| `## main` | no declaration; inherit |
| `## 关于 @exec 的讨论` | no declaration; the `@exec` is ordinary title text |
| `## @Exec main` / `## @knowledg main` | `sem.invalid_mode_marker` (no case/spelling repair) |
| `## @exec` | `sem.empty_unit_name` |
| `## @exec @knowledge main` | conflicting declarations; hard error |
| `## **@exec** main` | not a declaration; `@exec` is not the first plain Text child |
| prose containing `@exec` | ordinary text; never changes mode |
The heading's **display text** is unchanged for Markdown rendering. Its
**semantic name** has the declaration (and the whitespace after it) removed,
then applies the existing NFC/trim/whitespace-collapse normalization. All
identity, duplicate checks, function registration and `ref` lookup use the
semantic name; `@exec`/`@knowledge` are never part of a callable name.
ATX headings are the only accepted declaration form. A Setext heading has no
prefix declaration in the first version and simply inherits its mode; do not
invent an implicit declaration for it.
### 1.2 Inheritance
Using the Markdown heading stack (nearest lower-level ancestor heading, by
source order):
```text
Mode(document root) = Knowledge
Mode(H) = Explicit(H) if H has a valid declaration
Mode(H) = Mode(nearest ancestor) if H has no declaration and has an ancestor
Mode(H) = Knowledge if H has no declaration and no ancestor
```
Inheritance is a structural calculation over heading levels, not inference from
prose. A heading's scope ends at the next heading of the same or higher level
(same or smaller number); deeper headings are inside it. Siblings never share a
mode: an unmarked heading after a sibling that explicitly switched to Knowledge
still inherits from the nearest **ancestor**, not from the sibling.
Skipped levels (H1 directly followed by H3) still use the nearest valid
ancestor heading.
### 1.3 Heading roles
| Heading | Mode | Language object | Callable? | Effect |
|---|---|---|---|---|
| H1 | Knowledge/Exec | Namespace or top-level unit | No | Sets the default mode for descendants |
| H2 | Exec | Function + Unit | **Yes** | Opens the function body |
| H2 | Knowledge | Knowledge unit | No | Document section; no function |
| H3–H6 | Exec | Section of the owning H2 Exec function | No | Appends an execution segment to that function |
| H3–H6 | Knowledge | Section (knowledge or under a function) | No | Retained as document; no statements |
H3–H6 never create functions, never export callables and never create a new
call frame. An Exec payload under an H1 without an H2 Exec function, or under a
Knowledge H2, is `sem.exec_without_function`; it is never promoted to a function
and never executed as module-level code.
### 1.4 Version selection
There is no syntax-version switch. Every `.mq.md` file, imported module,
dependency, fixture and stdlib module uses this mapping. Introducing
`marqdo_semantics`, `syntax: v1`, `--legacy`, `--syntax` or any per-file mode
selection is an error (`sem.unsupported_version_directive` where the old
directive is seen); no implicit downgrade to an earlier mapper is permitted.
## 2. Inert regions
The following are never searched for payloads, even inside an Exec scope:
| Region | Treatment |
|---|---|
| every `@knowledge` subtree | zero executable statements |
| heading text (including a mode declaration) | unit identity only |
| link labels and URLs, image alt text | document/link data |
| GFM table cells | table data |
| math / math blocks | inert `Formula` resources |
| HTML blocks/inline HTML | document data |
| fenced and indented code blocks | inert `Code` resources |
| front matter | module configuration |
| strikethrough and other non-MVP extensions | inert document data |
Inline `Code` (a code span) is a payload carrier in Exec scope. A fenced code
block is a different AST node and stays inert; the two must not be conflated.
## 3. Executable payload carriers
### 3.1 Accepted carriers
Only these Comrak AST nodes are payload carriers in the first version:
| AST kind | Payload |
|---|---|
| `Strong` | concatenation of its **direct** Text and inline Code children, in source order |
| `Emph` | same as `Strong` |
| inline `Code` | the code span literal |
They are **interchangeable carriers of one statement**. For the same Exec
scope, owner function and payload text:
```text
Parse(Strong("return n")) == Parse(Emph("return n")) == Parse(Code("return n"))
```
The original AST kind is still recorded for rendering and source spans. Format
does not select the operation.
### 3.2 Extraction rules
1. Traverse an Exec heading's direct body in Markdown AST document order, and
recurse only through the allowed containers `Paragraph`, `List`, `ListItem`
and `BlockQuote`.
2. Extract only complete outer carrier nodes. A nested `Code` inside a
`Strong`/`Emph` is consumed by the outer carrier and must not execute again.
3. For `Strong`/`Emph`, only direct Text and inline Code children are allowed.
Any nested `Strong`, `Emph`, `Link`, `Image`, math, line break or other
complex child is `sem.payload_nested`. Never extract a partial statement or
silently concatenate multiple fragments.
4. A payload must be fully consumed by exactly one statement. `x = 1; return x`
in one carrier is `sem.syntax`; write two carriers instead.
5. Empty carriers in Exec scope are `sem.empty_payload`; they are never ignored.
6. Every payload keeps its original UTF-8 file span. For `Strong(Code(...))`
record the segment mapping so diagnostics point into the inner text, not at
the emphasis delimiters.
7. Two adjacent carriers produce two statements in source order. One carrier is
never split into multiple statements by newlines or semicolons.
### 3.3 Containers
Containers provide order and layout, not operation type:
| Container | Compiler behavior | Not allowed |
|---|---|---|
| `Paragraph` | extract permitted carriers in source order | prose does not execute |
| ordered `List` | recurse through items in source order | no implicit `Branch` |
| bullet `List` | recurse through items in source order | no implicit `Loop` |
| `BlockQuote` | recurse through nested blocks in source order | no implicit `Call` |
| `Heading` | switches/extends mode and segments | not a statement |
A list's `ordered`/`start`/`tight` attributes, an item's checkbox, and quote
nesting depth never affect control flow. A plain ordered or bullet list with no
carriers has an empty execution projection (zero statements), which is not the
same as producing a `List([])` value.
### 3.4 The deliberate emphasis cost
In an Exec scope, `**重要**` is a payload and is parsed as the expression
`重要`. If no such symbol exists, that is `symbol.unknown`. The language does
not guess that the author wanted emphasis. Put purely visual emphasis in a
`@knowledge` section (or another inert region). This is a deliberate
determinism trade-off, not an implementation gap.
## 4. Statements
Each extracted payload is exactly one statement, defined by
`spec/expression.ebnf`:
```text
if <expr> else while <expr> end
return <expr> params(...) assignment expression
```
Lexical details:
* Keywords are recognized only as complete first tokens: `ifdef(...)` and
`returning` are ordinary identifiers; a bare `if`/`else` used as an
identifier is a reserved-keyword error.
* `if`, `while` and `return` require a non-empty expression that consumes the
whole payload (to EOF). Trailing characters are errors.
* `else` and `end` accept no additional tokens; `else x` and `end if` are
`sem.syntax`.
* No trailing colon: `if x > 0:` is `sem.syntax`. No `elif`, `for`, `break`,
`continue`, `try`, `end if` or semicolon-separated statements in this
version.
* `return` always requires an expression; falling off the end of a function
still returns `Unit`. Explicit `return None` is available when `None` is the
intended value.
* `params(...)` is a compile-time signature declaration, not a runtime
callable; see §6.
* Existing expression precedence, literals, calls, indexing, assignment and
`Memory` place writes are unchanged. Conditions must evaluate to `Bool` and
keep the existing purity rule.
## 5. Control-stack lowering
Within one execution segment, payloads form a token stream. Lower it with one
explicit stack:
```text
open_if(cond) -> push IfFrame(cond, then=[], else=None)
statement -> append to the innermost active body
else -> require top is IfFrame with else=None; switch to else body
open_while(cond)-> push WhileFrame(cond, body=[])
end -> require a non-empty stack; pop the top frame and lower it
at_segment_end -> require the stack to be empty
```
Rules:
* `else` pairs with the top-of-stack `IfFrame` only. It may not skip an open
`while` or an inner `if`.
* At most one `else` per `if`; `else` with no open `if` is
`sem.unmatched_else`; a second `else` is `sem.duplicate_else`.
* `end` closes exactly the most recent open `if`/`while`; no match is
`sem.unmatched_end`.
* Reaching a segment boundary with an open frame is `sem.unclosed_control`,
with a related span on the opener. The compiler must not synthesize an `end`.
* `if` lower to the existing `Branch` IR; the optional `else` may lower to a
final unconditional arm while preserving left-to-right short-circuit
evaluation and executing at most one arm. `while` lowers to `Loop`; `return`
lowers to `Return` and unwinds the whole H2 function. `else`/`end` generate no
runtime instruction of their own.
## 6. Functions, signatures and calls
* Only an Exec H2 creates a callable function. Its body is the concatenation,
in source order, of its direct Exec body segment and the Exec segments of its
H3–H6 descendants that are part of the same function. Knowledge segments
contribute nothing.
* H3–H6 Exec sections share the H2 function's frame; locals assigned in one
segment are visible in later segments. They do not create new functions.
* `params(name: Type, ...)` is legal only as the **first direct execution
payload of an Exec H2 function**, at most once, and not inside a list or
block quote. The position is function-wide: it may be preceded by prose or a
Knowledge subsection, but not by an executing statement in **any** heading
segment of that function (a later `### @exec` section may not add a
signature after earlier code ran). Violations are `sem.signature_position`;
a duplicate declaration anywhere in the function is also rejected.
* A function without `params` keeps the existing free-variable inference.
As before, an inferred signature accepts **named arguments only**; declare
`params(...)` to accept positional arguments. Public functions should declare
`params`; the optional `sem.inferred_signature` lint is emitted for an
inferred signature that actually has parameters.
* Calls are ordinary expressions. `print(1)`, `f(n=5)` and `> **print(1)**`
all call because the payload contains a call; the block quote itself does
not create a call. A plain `> print(1)` is inert prose.
* `@knowledge` content is never scanned for identifiers, free variables or
calls.
* `main` must be an Exec H2 to be the default `run` entry; if none exists, `run`
reports `sem.no_entry_function`.
## 7. Segments and the heading boundary
An **execution segment** is one heading's direct body, excluding descendant
headings. Segments are ordered by source position. For each Exec H2 function,
process its own segment followed by the Exec segments of its descendant
headings, in document order.
The control stack is reset and required to be empty at every heading boundary.
A control block opened in one heading's direct body must be closed in that same
body; `end` in a later H3/H4 or on the other side of a Knowledge heading is
`sem.unmatched_end` there, while the opener's segment reports
`sem.unclosed_control`. The two errors are reported independently and are never
"repaired" across the boundary.
Complete segments are appended to the enclosing function body in source order;
earlier segments' assignments remain visible to later ones.
## 8. Diagnostics
Stable machine-readable codes (the human `message`/`hint` text may change):
| Code | Trigger |
|---|---|
| `sem.invalid_mode_marker` | unknown/case-variant/conflicting `@...` declaration at heading start |
| `sem.empty_unit_name` | a declaration with no non-empty heading name |
| `sem.duplicate_unit_name` | duplicate canonical heading/unit name in one scope |
| `sem.exec_without_function` | an executable payload with no owning H2 Exec function |
| `sem.payload_nested` | a `Strong`/`Emph` with a disallowed nested node |
| `sem.empty_payload` | an empty carrier in an Exec scope |
| `sem.syntax` | malformed statement, missing condition/value, trailing tokens, colon, unsupported keyword |
| `sem.unmatched_else` | `else` with no open `if` at the top of the stack |
| `sem.duplicate_else` | a second `else` for the same `if` |
| `sem.unmatched_end` | `end` with no open control frame |
| `sem.unclosed_control` | an open `if`/`while` at a heading/function/file boundary |
| `sem.signature_position` | `params` not the first function-wide direct payload, duplicated, or in a container |
| `sem.unsupported_version_directive` | legacy `marqdo_semantics`/syntax-selection front matter |
| `sem.no_entry_function` | `run` with no Exec H2 `main` |
| `sem.inferred_signature` | lint: public function without explicit `params` |
Existing `expr.*`, `symbol.unknown`, `runtime.*`, `resource.*` and capability
diagnostics keep their meaning. Diagnostics are ordered by
`(source_file, start_byte, code)`, and each carries an exact UTF-8 span plus a
related span when an opener or conflicting declaration is involved. A file with
any hard error is never executed.
## 9. Security and the runtime boundary
`@exec` is a compile-time projection, not a capability grant. Resource, file,
network, stdin/stdout, memory and budget checks are unchanged. A payload in
`@knowledge` produces zero IR, so it can never cause a host effect, even if its
text looks like `resource.open(...)`. Permission flags never change the
AST→IR result; they are checked at runtime by the existing host boundary.
The mapping introduces no `HostOp` for `if`/`else`/`while`/`end`/`return`, no
second interpreter and no runtime knowledge of Markdown emphasis. It lowers to
the existing `Assign`/`Eval`/`Branch`/`Loop`/`Return` IR.
## 10. Migration table (old mapping → current mapping)
| Requirement | Old source | Current source |
|---|---|---|
| Namespace | `# 算法` | `# @knowledge 算法`, or `# @exec 算法` for an Exec default |
| Function | `## main` | `## @exec main` (or an unmarked H2 under an Exec H1) |
| Assignment | `**x = 1**` | `**x = 1**`, `*x = 1*` or `` `x = 1` `` |
| Return | `*x*` | `**return x**`, `*return x*` or `` `return x` `` |
| Branch | ordered list with a first code-span condition | `if cond ... else ... end`; containers are presentation only |
| Loop | bullet list with a first code-span condition | `while cond ... end` |
| Call | block quote containing a call, or a Strong expression | any carrier whose expression is a call |
| Signature | first standalone code span `(n: Int)` after the H2 | `params(n: Int)` as the first function-wide direct payload |
| Ordinary list | error inside a function | ordinary document structure, zero statements |
| Ordinary quote | error inside a function | ordinary document structure, zero statements |
| Knowledge | implied by position/inertness only | explicit `@knowledge` scope |
Old files are not compatible. Migration must be done by AST role, not by a
global regular expression, and a migrated program must preserve function names,
argument meaning, evaluation order, early-return behavior and side effects.
## 11. Canonical example
```markdown
# @knowledge 数学演示
这是普通文档;**加粗**只显示为加粗。
## @exec main
`x = 5`
`y = x * x`
`return y`
```
Normalized IR: `Assign(x, 5)`, `Assign(y, Mul(x, x))`, `Return(y)`. The H1 is
Knowledge; the H2 overrides to Exec and is the callable `main`.
## 12. Non-negotiable implementation constraints
Do not:
* try to parse a carrier and fall back to prose on failure;
* infer a list is a branch because an item looks like a condition;
* use emphasis type to choose execute vs return;
* use block-quote/list type to choose call/branch/loop;
* scan raw Markdown text for `@exec`/`if`/`return` instead of using the AST;
* recover a misspelled `@knowledge` by inheriting the ancestor mode;
* expose a per-file syntax switch, legacy mapper, `--legacy` flag or dual
active specification.