Parser API
The parser converts SevenMark source into a SyntaxDocument. It does not fetch documents, check access, or send notifications.
Rust usage
The Rust crates are not published to crates.io. A crate in this workspace can use the workspace dependency:
[dependencies]
sevenmark_parser = { workspace = true }For a crate outside this workspace, use a path dependency to crates/sevenmark_parser in a local checkout.
Enable include_locations to include source positions in serialized AST output:
[dependencies]
sevenmark_parser = { workspace = true, features = ["include_locations"] }Parse a document
use sevenmark_parser::parse_document;
fn main() {
let input = "# Hello **World**\n\nThis is *SevenMark*.";
let document = parse_document(input);
assert_eq!(document.source_len(), input.len());
for element in document.as_slice() {
println!("{element:?}");
}
}parse_document
pub fn parse_document(input: &str) -> sevenmark_ast::SyntaxDocumentinput contains the source text. The returned SyntaxDocument retains the original source byte length and the parsed elements.
| Method | Result |
|---|---|
as_slice() | Borrow the Element slice |
into_elements() | Consume the document and return Vec<Element> |
source_len() | Read the original UTF-8 source byte length |
into_parts() | Consume the document and return its elements and source byte length |
SyntaxDocument also dereferences to [Element]. Serde serializes it as an element array; it does not serialize the stored source length.
The parser returns a document, not a Result. Recovery can preserve malformed syntax as ordinary text. If parsing fails or leaves source unconsumed, the parser stores that source in Element::Error. The internal parsing recursion limit is 16 levels. This bounds recursive parser work, including alternatives; it is not a count of visible containers. A rejected depth returns one Error containing the whole original source. It cannot become a successful partial AST. Each document has its own depth state, including when documents share a storage budget. Storage exhaustion takes precedence and returns no source echo from the budgeted APIs.
Reference ranges
The public document_references module uses the same syntax parser for editor completion and decoration:
reference_spans(document, source)collects parsed reference ranges.reference_at(source, cursor)finds a reference at a UTF-8 byte offset.reference_query(source, cursor, reference)extracts the query at that offset.
Pass the original source for the parsed document. These functions do not resolve document identities or authorize targets. See the document reference LSP contract for UTF-16 editor ranges and host-supplied candidates.
Parser context, combinators, and recovery state are private. AST types belong to the sevenmark_ast crate.
WebAssembly usage
The sevenmark_wasm crate enables include_locations. Its two parse functions return JSON strings:
| Function | Position unit |
|---|---|
parse_sevenmark(input) | UTF-8 byte offsets |
parse_sevenmark_to_codemirror(input) | Absolute UTF-16 code unit offsets |
These exports use 16 MiB source and 128 MiB logical syntax-storage defaults. A host can pass explicit validated admission to parse_sevenmark_with_limits(input, max_source_bytes, max_ast_bytes) or parse_sevenmark_to_codemirror_with_limits(input, max_source_bytes, max_ast_bytes). Limits use finite positive whole JavaScript numbers. Source uses UTF-8 bytes and has a portable ceiling of 268,435,455 bytes. Syntax storage has a portable ceiling of 2,147,483,647 logical bytes. Invalid or exceeded limits return an error without a partial AST. Logical storage does not measure process resident memory.
Both offset forms start at zero. CodeMirror 6 uses the UTF-16 form.
Build an npm package
From the repository root, prepare the bundler package:
cargo xtask wasm-npm-pack --crate sevenmark_wasm --scope your-scopeThe command writes crates/sevenmark_wasm/pkg-npm/. It does not publish the package. To build and publish it, use:
cargo xtask wasm-npm-publish --crate sevenmark_wasm --scope your-scopeAfter publication, install the package in a frontend app:
pnpm add @your-scope/sevenmarkA bundler must support WebAssembly modules:
import {
parse_sevenmark,
parse_sevenmark_to_codemirror,
} from "@your-scope/sevenmark";
const source = "# Hello **World**";
const byteAst = JSON.parse(parse_sevenmark(source));
const editorAst = JSON.parse(parse_sevenmark_to_codemirror(source));Build another target
Run one of these commands from crates/sevenmark_wasm:
| Consumer | Build command |
|---|---|
| Browser with explicit initialization | wasm-pack build --target web |
| Bundler | wasm-pack build --target bundler |
| Node.js or a VS Code extension | wasm-pack build --target nodejs |
The web target requires initialization before parsing:
import init, { parse_sevenmark } from "./pkg/sevenmark_wasm.js";
await init();
const ast = JSON.parse(parse_sevenmark("# Hello **World**"));The Node.js target uses CommonJS:
const { parse_sevenmark } = require("./pkg/sevenmark_wasm.js");
const ast = JSON.parse(parse_sevenmark("**Bold** and *italic*"));Source positions
include_locations controls position fields in serialized AST output. Without this feature, JSON omits those fields. With it, JSON includes span and the element's other source position fields.
The following examples show selected fields for # Hello.
Without include_locations:
[{ "Header": { "level": 1, "children": [{ "Text": { "value": "Hello" } }] } }]With include_locations:
[
{
"Header": {
"span": { "start": 0, "end": 7 },
"level": 1,
"children": [
{ "Text": { "span": { "start": 2, "end": 7 }, "value": "Hello" } }
]
}
}
]Native AST spans use UTF-8 bytes. The CodeMirror WASM function converts them to UTF-16 code units. Use the position unit required by the consumer; do not mix the two forms.