feat(pdf): own Typst wasm compiler (@blocknote/xl-typst-compiler), replacing typst.ts - #3020
Draft
YousefED wants to merge 8 commits into
Draft
feat(pdf): own Typst wasm compiler (@blocknote/xl-typst-compiler), replacing typst.ts#3020YousefED wants to merge 8 commits into
YousefED wants to merge 8 commits into
Conversation
…placing typst.ts Compiles the official typst crates to wasm behind a minimal TypeScript API and moves the whole PDF pipeline onto it: native, validated PDF/UA-1 (tryDeclarePdfUA with typed violations), compile failures as values (compileErrors/compileWarnings), no CDN or network access anywhere, no page-level compiler singleton, and no @myriaddreamin/* or @cantoo/pdf-lib runtime dependencies. The wasm builds from rust/ via a self-provisioning build step (scripts/ensure-wasm.mjs) that covers CI and Vercel.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
GitHub's ubuntu-24.04 images no longer ship Rust, so ensure-wasm now bootstraps rustup on CI like it does on Vercel (dev machines still get instructions instead of an install). PDFExporter.toPDF replaces toBytes/toBlob - one method, one result union, with the success branch carrying the PDF as both bytes and a Blob - mirroring toTypst and the other exporters' to<Format> naming.
…DF options Fonts (bytes, value-or-promise) and the compiler wasm move to the PDFExporter constructor - names and bytes side by side, matching how the other exporters put all configuration in the constructor - while toPDF takes a single per-export bag of document facts (plus tryDeclarePdfUA, assets, creationTimestamp), like every other exporter's export method. Also: exported spreadable default-font loaders (replacing the short-lived extraFonts), lazy result Blob, a font-name/file sync invariant test, the lang requirement failing fast, PdfExporterOptions as the full options shape with Partial at the constructor, and a leaner e2e docker context (rust/target excluded).
…tside the e2e container wasm-pack now runs with the crate directory as cwd so rustup discovers rust-toolchain.toml (a bootstrapped rustup with no default toolchain found nothing to run - masked locally by an existing default). The e2e shards can't build the wasm at all (the Playwright container has no C toolchain for proc-macros), so a bare-runner job builds it once and shares pkg/dist/types as an artifact.
Simple-first flow matching the DOCX template: PDF/UA confined to its own section, CDN/offline said once, internals cut, fonts split into short sections, custom mappings owned by the Typst page (the base layer never points up; the PDF page keeps its own option surfaces explicit). Typst page gains its own Customizing/assets/math-diagram/options sections; math's export section renamed Typst / PDF. DEFAULT_FONT_FAMILY / DEFAULT_MONO_FONT_FAMILY are exported so fallback lists need no hardcoded names.
… and prose-style rules from the PDF page review
…builds Adds the four typst-pdf-page chromium baselines (visually verified, pixel-exact via pdf.js rasterization) and gives the visual test cold-start headroom on shared runners. The e2e image install no longer re-verifies minimumReleaseAge (2000+ live registry checks - the frozen lockfile was already policy-verified by the install that produced it and by CI) and caches the pnpm store/metadata across rebuilds, taking the step from 70+ flaky minutes to ~4 (sub-minute warm). The pkg-pr-new soft release excludes xl-typst-compiler: its ~25MB wasm exceeds the service's non-whitelisted upload limit.
@blocknote/ariakit
@blocknote/code-block
@blocknote/core
@blocknote/diagram-block
@blocknote/mantine
@blocknote/math-block
@blocknote/react
@blocknote/server-util
@blocknote/shadcn
@blocknote/xl-ai
@blocknote/xl-docx-exporter
@blocknote/xl-email-exporter
@blocknote/xl-multi-column
@blocknote/xl-odt-exporter
@blocknote/xl-pdf-exporter
@blocknote/xl-typst-exporter
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces
@myriaddreamin/typst.tswith our own wasm binding over the official typst crates (0.15.1, crates.io — no fork):@blocknote/xl-typst-compiler, ~350 lines of Rust + TypeScript. Motivation: typst.ts is a single-maintainer project compiled against a fork of typst, and its release freeze blocked native PDF-standards support in the browser (the wasm-side feature merged upstream four days after the last published rc).What the new package is
World(main source + asset map,.typassets importable), fonts supplied as bytes per instance — no page-level singleton (instance creation incl. parsing the 8 default fonts: ~6ms; typst's caches are module-global, so fresh instances compile warm at ~7-9ms; full document cold compile: ~2min wasm build, 242ms first compile, 7ms warm).pdfStandard: "ua-1"makes typst/krilla validate conformance during the compile and refuse to emit a nonconforming PDF — a successful compile is the conformance statement.API changes (xl-pdf-exporter)
toPDF(blocks, options?)replacestoBytes/toBlob— named like its siblings (toTypst,toODTDocument, …) and taking the family-consistent single per-export bag: document facts (title,lang,paper,header, …) plustryDeclarePdfUA, extraassets, andcreationTimestamp. The result union is{ bytes, blob, pdfUA, compileWarnings }or{ error: "compile-failed", compileErrors, compileWarnings }— theblobis a memoized lazy getter (constructing a Blob copies the bytes, so results that only usebytesnever pay for it).PdfExporterOptions = TypstExporterOptions & { wasm?, fonts, emojiFont }(full shape; the constructor takes aPartialand fills defaults, mirroring the base). Family names and font bytes now sit side by side;fonts/emojiFontaccept promises so lazy loading fits the sync constructor. The bundled defaults are exported (loadDefaultBodyFonts/loadDefaultEmojiFont) so extending is a spread, e.g. CJK:fonts: loadDefaultBodyFonts().then((f) => [...f, notoSansSC])withfontFamily: ["Inter 18pt", "Noto Sans SC"].tryDeclarePdfUA(defaulttrue— safe because the claim is compile-validated and can never be false): conforming documents get thepdfuaidclaim, nonconforming ones re-export tagged-but-unclaimed with Typst's violations inpdfUA. Declaring requireslangand throws without it (a caller-args error, failing fast): Typst always writes/Lang, defaulting to English — verified, it cannot be omitted upstream — and a wrong language declaration is an accessibility defect no validator can catch.TypstExporterno longer fabricates title/lang/author defaults.compileErrors/compileWarnings) is the severity; the wasm returns one uniform never-thrown payload with an explicit output-xor-errors invariant check.compileWarnings, so renaming a default family or swapping a bundled font file fails CI regardless of which side drifted.Build & infra
pkg/(wasm + glue) is a gitignored build output. The package's build task self-provisions viascripts/ensure-wasm.mjs: content-hash skip when fresh (~0s), builds via the lockfile-pinnedwasm-packdevDep otherwise; toolchain pinned inrust/rust-toolchain.toml. On CI and Vercel it bootstraps rustup automatically (GitHub's ubuntu-24.04 images ship no Rust); on Vercel the cargo caches live undernode_modules/.cache(persisted), so only the first build on a fresh cache pays the ~2-15min compile. Local cold compile: 1m52s; warm/unchanged: ~2s..dockerignoreexcludesrust/target(1.3GB) from the e2e image context.shared/util/typstTestUtil.ts) run the same wasm in Node —@myriaddreamin/typst-ts-node-compileris gone, and test fonts are now the repo's own (no more system-font dependence).Test changes
install-pdf-tooling.sh(veraPDF + pinned poppler container) is deleted: conformance is validated by the compile itself (the declared output was verified against veraPDF--flavour ua1: 0 failed checks, and was pixel-identical to the previous engine's visual baselines), and the visual regression moved into the browser e2e suite, which rasterizes the produced PDF with pdf.js and screenshots each page (typst-pdf-page-N, chromium, deterministic via a fixedcreationTimestamp)..typimports, byte-reproducibility, font dedup/introspection, invalid-options throw), exporter suite (14: declare flow, lang gating, warnings forwarding, font-sync invariant, lazy blob, spread-extended defaults), slimmed pdfua structural test (pdf-lib as devDep for compressed-object assertions).Still pending on this branch
typst-pdf-page-Ne2e baselines (docker image rebuild in progress; until committed, the screenshot matcher seeds them per-run)Numbers vs typst.ts: 24.7MB wasm (vs 28.8MB, pre-binaryen — a modern
wasm-optpass is a known future size win), ~350 LOC owned (vs ~28k in the replaced stack), zero runtime deps beyond the two workspace packages.🤖 Generated with Claude Code