Skip to content

Parser API ​

The parser converts SevenMark source into a SyntaxDocument. It does not fetch documents, check access, or send notifications.

Rust usage ​

The Rust crates are not published to crates.io. A crate in this workspace can use the workspace dependency:

toml
[dependencies]
sevenmark_parser = { workspace = true }

For a crate outside this workspace, use a path dependency to crates/sevenmark_parser in a local checkout.

Enable include_locations to include source positions in serialized AST output:

toml
[dependencies]
sevenmark_parser = { workspace = true, features = ["include_locations"] }

Parse a document ​

rust
use sevenmark_parser::parse_document;

fn main() {
    let input = "# Hello **World**\n\nThis is *SevenMark*.";
    let document = parse_document(input);

    assert_eq!(document.source_len(), input.len());
    for element in document.as_slice() {
        println!("{element:?}");
    }
}

parse_document ​

rust
pub fn parse_document(input: &str) -> sevenmark_ast::SyntaxDocument

input contains the source text. The returned SyntaxDocument retains the original source byte length and the parsed elements.

MethodResult
as_slice()Borrow the Element slice
into_elements()Consume the document and return Vec<Element>
source_len()Read the original UTF-8 source byte length
into_parts()Consume the document and return its elements and source byte length

SyntaxDocument also dereferences to [Element]. Serde serializes it as an element array; it does not serialize the stored source length.

The parser returns a document, not a Result. Recovery can preserve malformed syntax as ordinary text. If parsing fails or leaves source unconsumed, the parser stores that source in Element::Error. The internal parsing recursion limit is 16 levels. This bounds recursive parser work, including alternatives; it is not a count of visible containers. A rejected depth returns one Error containing the whole original source. It cannot become a successful partial AST. Each document has its own depth state, including when documents share a storage budget. Storage exhaustion takes precedence and returns no source echo from the budgeted APIs.

Reference ranges ​

The public document_references module uses the same syntax parser for editor completion and decoration:

  • reference_spans(document, source) collects parsed reference ranges.
  • reference_at(source, cursor) finds a reference at a UTF-8 byte offset.
  • reference_query(source, cursor, reference) extracts the query at that offset.

Pass the original source for the parsed document. These functions do not resolve document identities or authorize targets. See the document reference LSP contract for UTF-16 editor ranges and host-supplied candidates.

Parser context, combinators, and recovery state are private. AST types belong to the sevenmark_ast crate.

WebAssembly usage ​

The sevenmark_wasm crate enables include_locations. Its two parse functions return JSON strings:

FunctionPosition unit
parse_sevenmark(input)UTF-8 byte offsets
parse_sevenmark_to_codemirror(input)Absolute UTF-16 code unit offsets

These exports use 16 MiB source and 128 MiB logical syntax-storage defaults. A host can pass explicit validated admission to parse_sevenmark_with_limits(input, max_source_bytes, max_ast_bytes) or parse_sevenmark_to_codemirror_with_limits(input, max_source_bytes, max_ast_bytes). Limits use finite positive whole JavaScript numbers. Source uses UTF-8 bytes and has a portable ceiling of 268,435,455 bytes. Syntax storage has a portable ceiling of 2,147,483,647 logical bytes. Invalid or exceeded limits return an error without a partial AST. Logical storage does not measure process resident memory.

Both offset forms start at zero. CodeMirror 6 uses the UTF-16 form.

Build an npm package ​

From the repository root, prepare the bundler package:

bash
cargo xtask wasm-npm-pack --crate sevenmark_wasm --scope your-scope

The command writes crates/sevenmark_wasm/pkg-npm/. It does not publish the package. To build and publish it, use:

bash
cargo xtask wasm-npm-publish --crate sevenmark_wasm --scope your-scope

After publication, install the package in a frontend app:

bash
pnpm add @your-scope/sevenmark

A bundler must support WebAssembly modules:

ts
import {
  parse_sevenmark,
  parse_sevenmark_to_codemirror,
} from "@your-scope/sevenmark";

const source = "# Hello **World**";
const byteAst = JSON.parse(parse_sevenmark(source));
const editorAst = JSON.parse(parse_sevenmark_to_codemirror(source));

Build another target ​

Run one of these commands from crates/sevenmark_wasm:

ConsumerBuild command
Browser with explicit initializationwasm-pack build --target web
Bundlerwasm-pack build --target bundler
Node.js or a VS Code extensionwasm-pack build --target nodejs

The web target requires initialization before parsing:

ts
import init, { parse_sevenmark } from "./pkg/sevenmark_wasm.js";

await init();
const ast = JSON.parse(parse_sevenmark("# Hello **World**"));

The Node.js target uses CommonJS:

js
const { parse_sevenmark } = require("./pkg/sevenmark_wasm.js");

const ast = JSON.parse(parse_sevenmark("**Bold** and *italic*"));

Source positions ​

include_locations controls position fields in serialized AST output. Without this feature, JSON omits those fields. With it, JSON includes span and the element's other source position fields.

The following examples show selected fields for # Hello.

Without include_locations:

json
[{ "Header": { "level": 1, "children": [{ "Text": { "value": "Hello" } }] } }]

With include_locations:

json
[
  {
    "Header": {
      "span": { "start": 0, "end": 7 },
      "level": 1,
      "children": [
        { "Text": { "span": { "start": 2, "end": 7 }, "value": "Hello" } }
      ]
    }
  }
]

Native AST spans use UTF-8 bytes. The CodeMirror WASM function converts them to UTF-16 code units. Use the position unit required by the consumer; do not mix the two forms.