Benchmark bun check against tsc in every workspace

Part of #649 (ruling 3: this is the first benchmark). Related: #554, and #554 Q7: a swap to bun check reopens the Next type-check skip.

Benchmark

Field	Value
Metric	(1) Type-check wall time for all 15 workspaces, in seconds, and peak RSS in MiB. (2) Gate: differing diagnostic lines per workspace, split into "TS 7 change" and "bun check only". Exact count.
Before	Laptop: not measured yet (agent in the repo, Step 0, before any other step). CI reference, from the 2026-11-11 milestone review read of the Run Report: `📝 Type Check All` step median 22 s (18–32 s, n = 7, 2026-11-07 to 2026-11-11).
After	Not measured yet.
Condition	Named laptop (model, cores, RAM, OS). Bun 1.4.4, `tsc` 5.9.3 and 6.0.3, the TS 7 control version, pinned to one exact `typescript@7.x.y` and recorded here before Step 1. HEAD SHA. Fixed `.tsbuildinfo` and `.next/types` state. No other load.
How to run	Step 0: `bun run typecheck`, n = 5 after one discarded warm-up. Step 4: the same command shape with `bun check` in each workspace, n = 5 after one discarded warm-up. Time with `/usr/bin/time -l` on macOS or `/usr/bin/time -v` on Linux. Diagnostics: Steps 1 to 3. CI: read `📝 Type Check All` from 5 runs where the job ran.
Pass rule	Adopt only if "bun check only" diffs are 0 in every workspace and the whole run is at least 2x faster (median against median). Any "bun check only" diff: do not adopt; post the diff. Under 2x: close as not worth the change. Timing runs in every case.
Guard	Seeded-error control: both checkers fail in every workspace with the same TS code. This gate is the only type check before a webapp deploy, because #560 skips the one in the webapp Docker build.
Owner	Agent in the repo: Steps 0 to 6 and the Measured comment. Maintainer: rules the kill rule and the swap.
Claim	Adopt: "Type-checking all 15 docs.plus workspaces fell from s to s on (median of 5)." Kill: "bun check reported diagnostics that tsc did not, so docs.plus did not adopt it."

Why

Bun 1.4.3 added bun check, a port of typescript-go. Vendor numbers: 3.0–6.4x faster than tsc 7.0.2, 2.2–4.9x less memory, and 51,210 of 51,210 TypeScript conformance tests passed.

Our baseline is the JavaScript tsc 5.9.3 and 6.0.3, not tsc 7, so the vendor ratio does not predict our gain. The real gate is whether the diagnostics match.

Our typecheck runs tsc --noEmit in 15 workspaces (HEAD 0b302b222) through bun run --filter '*' typecheck. Three extensions also run tsc -p tsconfig.test.json. It is a check:ci gate (scripts/check-ci.ts:244), so it runs on every pre-push. The CI Type Check job is path-gated (the 📝 Type Check job and its 📝 Type Check All step in prod.docs.plus.yml).

CI before, 2026-11-11: 📝 Type Check All median 22 s (18–32 s, n = 7). The step is off the deploy critical path, so a faster check cannot move M1 (#569). The gain is the local pre-push loop and PR feedback.

What `bun check` must match

It follows TypeScript 7.1. 11 workspaces use tsc 5.9.3 through the catalog. 4 use 6.0.3: webapp, hocuspocus.server, email-templates and pad-url. Most workspaces therefore jump two majors.
The root tsconfig.json is not in the gate. No workspace extends it.
apps/admin-dashboard/tsconfig.json:5 sets "baseUrl": ".", and its paths entries (:19-27) have no ./ prefix. The Bun docs say TypeScript 7 reports TS5102 for baseUrl.
tsconfig.base.json:15 sets "ignoreDeprecations": "5.0", and 13 configs inherit it. Its TypeScript 7 behaviour is not verified.
apps/webapp and apps/admin-dashboard set incremental: true (tsconfig.json:17). A warm tsc run there reuses .tsbuildinfo. CI starts with none, and bun check writes none.
next-env.d.ts in both apps reads .next/types/. Record whether those files exist, and keep that state fixed for every run. No tsconfig sets plugins, so the Next plugin is not a factor.

Steps

Run on Bun 1.4.4, on the real tree. Never use a branch or a worktree. Pass --incremental false to every tsc control run, so the repo's .tsbuildinfo is not overwritten.

0. Before. On the named laptop, time bun run typecheck 5 times after one discarded run, before any other step. Record median and min–max wall time and peak RSS.
1. Diagnostics first, one run per workspace. Run tsc --noEmit --pretty false --incremental false and bun check --no-script --no-pretty. In the three extensions, also run both with -p tsconfig.test.json. Normalise to file:line:col TScode and diff.
- TS 7 control. Also run bunx --package typescript@<7.x.y> tsc --noEmit --pretty false per workspace. Use the one exact version recorded in Condition for every run, so the control does not float. Sort each diff line as "TS 7 change" (tsc 7 shows it too) or "bun check only".
2. Same files. Compare tsc --listFilesOnly with bun check --no-script --listFiles, without lib files.
- Pre-check: bun check --no-script --listFiles | head must print paths. If not, use --explainFiles. If neither works, drop Step 2 and list it under "Not measured".
3. Non-empty control. Run both with --noUncheckedIndexedAccess --incremental false. Both must report the same errors.
- Seeded-error control. In a git archive copy, put a wrong argument type, a missing property and an import of a missing export in each workspace. Both checkers must fail in every workspace with the same TS code.
4. Timing, in every case. Label it "not adoptable" when "bun check only" diffs exist. Use n = 5 per command after one discarded warm-up, and report median and min–max wall time and peak RSS.
- Use /usr/bin/time -l on macOS or /usr/bin/time -v on Linux for peak RSS. Name the model, cores, RAM, OS, Bun and tsc versions, and the .tsbuildinfo and .next/types state.
- In apps/webapp and apps/admin-dashboard, time tsc twice: as the script runs (buildinfo present, like pre-push), and with --incremental false (like CI).
- Whole run: run both checkers the same way. Either run both through bun run --filter '*' with a temporary script, or run both in a sequential loop.
5. CI. Read 📝 Type Check All from 5 recent runs where the job ran: median and min–max. A CI bun check number needs a CI run. That is out of scope until the swap issue.
6. Post a Measured comment in the [Tracking] Bun 1.4.x: release-age guard, bunfig cleanup, and measured bun check, bun fmt and bun test numbers #649 format, with every table.

Kill rules

Any "bun check only" diagnostics difference in any workspace: do not adopt. Post the diff, still post the Step 4 timing, and raise the TypeScript 7 question as a new issue. A "TS 7 change" diff alone does not kill adoption; list it for the TypeScript 7 question.
No "bun check only" diffs, but the whole run falls by less than 2x: close as not worth the change.
Otherwise: open the swap as a child of [Tracking] Faster, change-aware CI/CD pipeline #554, listing every script to change. The swap child keeps tsc as a one-line rollback.

Decisions (only if the diagnostics match and the gain is at least 2x)

Swap only the check:ci gate, or every workspace typecheck script? Recommended: every typecheck script, so one checker runs everywhere.
Keep the typescript package? Recommended: yes. Editors, next build, the tsup dts builds and typescript-eslint still need it.

Measured, 2026-11-12

Question: Does bun check report the same diagnostics as our tsc in all 15 workspaces, and is the whole type-check run at least 2x faster?

Setup

Field	Value
Commit	`cfac48d87` (the issue names `0b302b222`; HEAD moved before the run)
Machine	Laptop, Mac17,6, 18 cores, 48 GiB, macOS 27.0.1
Bun	1.4.4
`tsc`	5.9.3 in 11 workspaces, 6.0.3 in webapp, hocuspocus.server, email-templates and pad-url (from `bun.lock`)
TS 7 control	`typescript@7.0.2`, the `latest` tag, released 2026-07-08. One version for every control run.
Tree	A `git archive cfac48d87` copy, outside the repo. Then `bun install --frozen-lockfile --ignore-scripts`, `bun x prisma generate` in hocuspocus.server, and `bash scripts/build-extensions.sh`. No env files.
Next types	No `next-env.d.ts` and no `.next/types` in either app, the same as a CI checkout.
`.tsbuildinfo`	None at the start. The Step 0 warm-up wrote one in webapp and one in admin-dashboard. Every later "buildinfo present" run reused them. The CI-like copy had none and ran `--incremental false`.
Load	Other work shared the laptop. A run counts only if the 1-minute load average was under 6 at its start. Counted runs started at 4.7 to 5.99. Runs above 6 stay in the raw file, marked not counted.
Date	2026-11-12, 11:19 to 11:51 UTC

The issue says "run on the real tree". Other changes were open in the working tree, so every run used the copy. The copy also matches CI.

Method

Step 0 (before): /usr/bin/time -l bun run typecheck, 1 discarded warm-up, then timed runs until 5 counted.
Step 1 (diagnostics): one run per config, 18 configs (15 workspaces plus -p tsconfig.test.json in indent, hyperlink and mermaid).
- tsc --noEmit --pretty false --incremental false
- bun check --no-script --no-pretty --all
- bunx --package typescript@7.0.2 tsc --noEmit --pretty false --incremental false
- Each line was normalised to file:line:col TScode and sorted. A tsc/bun difference that TS 7.0.2 shares is a "TS 7 change". Any other difference is "bun check only".
Step 2 (same files): tsc --listFilesOnly against bun check --no-script --listFiles, lib files removed, paths resolved.
Step 3 (controls): Step 1 again with --noUncheckedIndexedAccess. Seeded errors ran in a second copy, with two new files in each workspace's src/. They hold a wrong argument type (TS2345), a missing property (TS2741) and an import of a missing export (TS2305).
Step 4 (timing): each workspace typecheck script pointed at bun check --no-script, the three extensions also with -p tsconfig.test.json. Then bun run typecheck, so the shape matches Step 0. The four shapes ran in turn, round by round, 1 discarded warm-up, then runs until 5 counted each.
- The two apps were also timed alone: tsc --noEmit with buildinfo, tsc --noEmit --incremental false, and bun check --no-script.
Step 5 (CI): 📝 Type Check All step start and end, from gh api .../actions/runs/<id>/jobs, for the 5 latest prod.docs.plus.yml runs where the job ran.
Peak RSS is from /usr/bin/time -l. For bun run --filter, it is the largest single process, not the sum.

Result: diagnostics (Steps 1 to 3)

Check	Configs	Diff lines	TS 7 change	bun check only
Step 1, as configured	18	12	12	0
Step 2, file lists, source files	18	0	n/a	n/a
Step 3, `--noUncheckedIndexedAccess`	18	26	26	0
Step 3, seeded errors	18	15	15	0

The 12 Step 1 lines, all also reported by TS 7.0.2:

Workspace	Lines	Cause
apps/admin-dashboard	`tsconfig.json:5:5 TS5102`, `tsconfig.json:20-27 TS5090` (8 lines)	`baseUrl` is removed in TS 7, and the 8 `paths` entries lack `./`
apps/hocuspocus.server	`src/modules/document-conversion/domain/markdownExport.ts:34:3 TS2322`, `:35:20 TS2322`	Tiptap extension types no longer assign
extensions/extension-hyperlink, `-p tsconfig.test.json`	`src/link-icons/__tests__/setupDom.ts:38:1 TS2322`	`Element.matches` must now be a type predicate

admin-dashboard checks 0 files under bun check. It stops on the 9 config errors: "Stopped before type checking 74 files". So its Step 1 "0 bun check only" says nothing about its code.
To close that gap, a scratch copy dropped baseUrl and added ./ to the 8 paths. Then all three checkers gave 0 lines in admin-dashboard, and 5 equal lines each under --noUncheckedIndexedAccess. tsc 5.9.3 also passes with that config.
Step 3, --noUncheckedIndexedAccess: tsc 698 lines, bun check 708, TS 7.0.2 708. The 26 diff lines are admin-dashboard (9 config lines, plus 5 tsc lines that bun check never reached), hocuspocus.server (11) and the hyperlink test config (1). In hocuspocus.server, 9 of its 11 lines are TS2769 overload errors in one test file. TS 7 puts them at other columns: 6 lines appear and 3 go.
Step 3, seeded errors: both checkers fail in all 18 configs. In 17, both give TS2305, TS2345 and TS2741 at the same positions. In admin-dashboard, bun check fails with TS5102 and TS5090 only. With the scratch config fix, it gives the 3 seeded codes too.
Step 2: the source files match in all 18 configs. Only dependency files differ: hocuspocus.server (1 file, tsc only) and webapp (4 tsc only, 1 bun check only), all under node_modules.

Result: timing (Steps 0, 4 and 5)

Whole run, bun run typecheck, all 15 workspaces, n = 5 each, 1 warm-up discarded:

Shape	Exit	Median (s)	Min–max (s)	Largest single process RSS, median (MiB)
Step 0, `tsc`, buildinfo present	0	5.97	5.26–7.48	1,079
Step 4, `tsc`, buildinfo present	0	5.53	5.27–6.74	1,008
Step 4, `tsc`, apps with `--incremental false` (CI-like)	0	6.59	6.19–8.09	1,088
Step 4, `bun check` (not adoptable as is: exit 1)	1	0.56	0.44–0.59	318
Step 4, `bun check`, admin-dashboard config fixed in the copy	1	0.52	0.42–0.59	322

Ratios, median against median: 5.97 / 0.56 = 10.7x from the Step 0 before. 5.53 / 0.56 = 9.9x for the interleaved pair. 6.59 / 0.56 = 11.8x against the CI-like run. Median CPU time was 19.9 s user for Step 0 and 2.2 s user for bun check.

bun check exits 1 because of the 12 "TS 7 change" lines above. All 15 workspaces still ran to the end in every run.

Apps alone, n = 5 each, 1 warm-up discarded:

Workspace	Command	Exit	Median (s)	Min–max (s)	Peak RSS, median (MiB)
apps/webapp	`tsc --noEmit`, buildinfo present	0	1.06	1.03–1.16	598
apps/webapp	`tsc --noEmit --incremental false`	0	3.93	3.88–4.26	909
apps/webapp	`bun check --no-script`	0	0.15	0.13–0.16	322
apps/admin-dashboard	`tsc --noEmit`, buildinfo present	0	0.60	0.60–0.62	401
apps/admin-dashboard	`tsc --noEmit --incremental false`	0	1.37	1.36–1.43	478
apps/admin-dashboard	`bun check --no-script` (checks 0 files)	1	0.05	0.05–0.06	157
apps/admin-dashboard	`bun check --no-script`, config fixed in the copy	0	0.07	0.07–0.07	202

CI, 📝 Type Check All step, tsc, 5 latest prod.docs.plus.yml runs where the job ran:

Runs	n	Median (s)	Min–max (s)
43008200172, 42788282072, 42781445956, 42778681880, 42689997917	5	22	19–31

Step times come from whole-second timestamps. A CI bun check number needs a CI run, so it waits for the swap issue.

Raw data

Attached: 654-step0-runs.jsonl, 654-step4-whole.jsonl, 654-step4-apps.jsonl, 654-step1-summary.json, 654-step2-listfiles.json, 654-step3-nuia-summary.json, 654-step3-seeded-summary.json, 654-step5-ci.tsv. Each timing line holds the load average and free memory at its start.

Not measured

A cold run with no buildinfo through the gate's own tsc script. The CI-like row uses --incremental false instead.
bun check in CI.
Fixes for the hocuspocus.server and hyperlink test lines. Only the admin-dashboard config fix was tried, in a scratch copy.
TS 7.1.0-beta as a second control. Every TS 7 change line matched 7.0.2, so no line needed it.
Editor, next build, tsup dts and typescript-eslint behaviour. They keep the typescript package.

Verdict

Adopt rule met: 0 "bun check only" lines in 18 configs, and the whole run is 9.9x faster. The swap child of #554 must first clear the 12 "TS 7 change" lines, because bun check exits 1 at cfac48d87. The maintainer rules the swap.

Ruling, 2026-11-14: adopt

The maintainer ruled the swap at the v2.3.6 release audit. The adopt rule passed: 0 "bun check only" lines in 18 configs, and the whole run is 9.9x faster (Measured).

Decision 1: swap every workspace typecheck script.
Decision 2: keep the typescript package.

The swap is #794, a child of #554. It clears the 12 "TS 7 change" lines first, because bun check exits 1 on them today. It keeps tsc as a one-line rollback.

Closing as completed.

Navigation Menu

Repository navigation

Benchmark bun check against tsc in every workspace #654

Benchmark

Why

What `bun check` must match

Steps

Kill rules

Decisions (only if the diagnostics match and the gain is at least 2x)

Measured, 2026-11-12

Setup

Method

Result: diagnostics (Steps 1 to 3)

Result: timing (Steps 0, 4 and 5)

Raw data

Not measured

Verdict

Ruling, 2026-11-14: adopt

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Benchmark bun check against tsc in every workspace #654

Description

Benchmark

Why

What bun check must match

Steps

Kill rules

Decisions (only if the diagnostics match and the gain is at least 2x)

Activity

HMarzban commented on Nov 12, 2026

Measured, 2026-11-12

Setup

Method

Result: diagnostics (Steps 1 to 3)

Result: timing (Steps 0, 4 and 5)

Raw data

Not measured

Verdict

HMarzban commented on Nov 14, 2026

Ruling, 2026-11-14: adopt

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

What `bun check` must match