Skip to content

test: reduce e2e timing flakes#2649

Draft
NathanColosimo wants to merge 17 commits into
mainfrom
nathanc/fix-sleep-skew
Draft

test: reduce e2e timing flakes#2649
NathanColosimo wants to merge 17 commits into
mainfrom
nathanc/fix-sleep-skew

Conversation

@NathanColosimo

@NathanColosimo NathanColosimo commented Jun 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Draft: experimenting with less flaky PR runs.

Changes:

  • Add skew-aware elapsed-time assertions for distributed Vercel e2e timestamp comparisons.
  • Loosen race timing checks so they still prove the 10s losing branch did not win.
  • Replace fixed sleeps in hook/cancel tests with event polling.
  • Give readable stream e2e more room for delayed final chunks/close on Vercel.
  • Add environment-aware e2e timeout budgets so local runs stay tight while Vercel prod runs have enough room for slow hook/event/run API calls under load.
  • Remove one redundant scheduler-speed assertion and one dead wait.

Validation so far: install, Biome check, dependency builds, core typecheck, and CI monitoring. I am rerunning full e2e/check workflows after passes to verify this is consistently green, not just lucky.

@changeset-bot

changeset-bot Bot commented Jun 25, 2026 •
edited
Loading

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: b682cf4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
Name Type
@workflow/core Patch
@workflow/builders Patch
@workflow/cli Patch
@workflow/next Patch
@workflow/nitro Patch
@workflow/vitest Patch
@workflow/web-shared Patch
@workflow/web Patch
workflow Patch
@workflow/world-testing Patch
@workflow/astro Patch
@workflow/nest Patch
@workflow/rollup Patch
@workflow/sveltekit Patch
@workflow/vite Patch
@workflow/nuxt Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Jun 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
example-nextjs-workflow-turbopack Ready Ready Preview, Comment Jun 26, 2026 12:26pm
example-nextjs-workflow-webpack Ready Ready Preview, Comment Jun 26, 2026 12:26pm
example-workflow Ready Ready Preview, Comment Jun 26, 2026 12:26pm
workbench-astro-workflow Ready Ready Preview, Comment Jun 26, 2026 12:26pm
workbench-express-workflow Ready Ready Preview, Comment Jun 26, 2026 12:26pm
workbench-fastify-workflow Ready Ready Preview, Comment Jun 26, 2026 12:26pm
workbench-hono-workflow Ready Ready Preview, Comment Jun 26, 2026 12:26pm
workbench-nitro-workflow Ready Ready Preview, Comment Jun 26, 2026 12:26pm
workbench-nuxt-workflow Ready Ready Preview, Comment Jun 26, 2026 12:26pm
workbench-sveltekit-workflow Ready Ready Preview, Comment Jun 26, 2026 12:26pm
workbench-tanstack-start-workflow Ready Ready Preview, Comment Jun 26, 2026 12:26pm
workbench-vite-workflow Ready Ready Preview, Comment Jun 26, 2026 12:26pm
workflow-docs Ready Ready Preview, Comment, Open in v0 Jun 26, 2026 12:26pm
workflow-swc-playground Ready Ready Preview, Comment Jun 26, 2026 12:26pm
workflow-tarballs Ready Ready Preview, Comment Jun 26, 2026 12:26pm
workflow-web Ready Ready Preview, Comment Jun 26, 2026 12:26pm

@github-actions

github-actions Bot commented Jun 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Express 0.046s (-5.0%) 1.006s (~) 0.960s 10 1.00x
💻 Local Nitro 0.048s (-17.4% 🟢) 1.006s (~) 0.958s 10 1.05x
💻 Local Next.js (Turbopack) 0.052s (+2.8%) 1.006s (~) 0.954s 10 1.13x
🐘 Postgres Next.js (Turbopack) 0.061s (+5.1% 🔺) 1.014s (~) 0.952s 10 1.34x
🐘 Postgres Express 0.064s (-2.6%) 1.012s (~) 0.948s 10 1.40x
🐘 Postgres Nitro 0.097s (+49.1% 🔺) 1.027s (+0.9%) 0.930s 10 2.11x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 0.220s (-4.1%) 1.819s (-10.2% 🟢) 1.599s 10 1.00x
▲ Vercel Nitro 0.232s (-8.1% 🟢) 1.723s (-13.8% 🟢) 1.491s 10 1.06x
▲ Vercel Next.js (Turbopack) 0.648s (-26.0% 🟢) 2.513s (-6.3% 🟢) 1.865s 10 2.95x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

workflow with 1 step

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Express 1.077s (~) 2.006s (~) 0.929s 10 1.00x
💻 Local Nitro 1.083s (-2.1%) 2.006s (~) 0.923s 10 1.01x
💻 Local Next.js (Turbopack) 1.091s (+0.8%) 2.007s (~) 0.915s 10 1.01x
🐘 Postgres Express 1.095s (-0.6%) 2.009s (~) 0.915s 10 1.02x
🐘 Postgres Next.js (Turbopack) 1.096s (~) 2.010s (~) 0.913s 10 1.02x
🐘 Postgres Nitro 1.097s (+0.8%) 2.041s (+1.5%) 0.944s 10 1.02x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 1.432s (+1.9%) 3.117s (+3.8%) 1.685s 10 1.00x
▲ Vercel Nitro 1.452s (+4.5%) 3.181s (+5.5% 🔺) 1.730s 10 1.01x
▲ Vercel Next.js (Turbopack) 2.472s (+16.9% 🔺) 3.887s (+11.4% 🔺) 1.416s 10 1.73x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

workflow with 10 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 10.333s (-1.3%) 11.015s (~) 0.682s 3 1.00x
💻 Local Express 10.422s (~) 11.024s (~) 0.602s 3 1.01x
🐘 Postgres Express 10.438s (~) 11.013s (~) 0.575s 3 1.01x
💻 Local Nitro 10.452s (-1.3%) 11.023s (~) 0.571s 3 1.01x
💻 Local Next.js (Turbopack) 10.530s (~) 11.023s (~) 0.494s 3 1.02x
🐘 Postgres Next.js (Turbopack) 10.547s (+0.6%) 11.024s (~) 0.477s 3 1.02x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 11.721s (-5.1% 🟢) 13.210s (-9.1% 🟢) 1.488s 3 1.00x
▲ Vercel Express 11.810s (-3.4%) 13.776s (+0.7%) 1.966s 3 1.01x
▲ Vercel Next.js (Turbopack) 12.362s (-6.9% 🟢) 14.182s (-6.2% 🟢) 1.820s 3 1.05x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

workflow with 25 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 13.354s (-1.8%) 14.016s (~) 0.662s 5 1.00x
💻 Local Express 13.539s (-0.7%) 14.028s (~) 0.489s 5 1.01x
🐘 Postgres Express 13.617s (~) 14.022s (~) 0.405s 5 1.02x
💻 Local Nitro 13.623s (-1.8%) 14.028s (~) 0.404s 5 1.02x
🐘 Postgres Next.js (Turbopack) 13.696s (+0.7%) 14.021s (~) 0.325s 5 1.03x
💻 Local Next.js (Turbopack) 13.825s (+1.2%) 14.028s (~) 0.202s 5 1.04x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 17.140s (~) 18.922s (-3.6%) 1.782s 4 1.00x
▲ Vercel Express 17.215s (-13.1% 🟢) 19.470s (-8.7% 🟢) 2.255s 4 1.00x
▲ Vercel Next.js (Turbopack) 18.890s (-23.1% 🟢) 21.095s (-19.7% 🟢) 2.205s 3 1.10x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

workflow with 50 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 11.807s (-3.4%) 12.273s (-5.7% 🟢) 0.466s 8 1.00x
🐘 Postgres Express 12.039s (-1.4%) 12.768s (-1.9%) 0.729s 8 1.02x
💻 Local Express 12.058s (-0.8%) 12.882s (-1.1%) 0.824s 7 1.02x
💻 Local Nitro 12.189s (-4.3%) 13.027s (~) 0.838s 7 1.03x
💻 Local Next.js (Turbopack) 12.236s (~) 13.025s (~) 0.789s 7 1.04x
🐘 Postgres Next.js (Turbopack) 12.256s (~) 13.014s (~) 0.758s 7 1.04x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 19.396s (+9.5% 🔺) 21.808s (+6.5% 🔺) 2.412s 5 1.00x
▲ Vercel Express 19.874s (-9.5% 🟢) 23.338s (-0.6%) 3.464s 4 1.02x
▲ Vercel Next.js (Turbopack) 22.665s (+8.4% 🔺) 25.356s (+12.2% 🔺) 2.691s 4 1.17x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

Promise.all with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 1.175s (-1.2%) 2.007s (~) 0.832s 15 1.00x
🐘 Postgres Next.js (Turbopack) 1.194s (+0.7%) 2.008s (~) 0.814s 15 1.02x
🐘 Postgres Nitro 1.200s (+3.1%) 2.012s (~) 0.812s 15 1.02x
💻 Local Express 1.395s (~) 2.007s (~) 0.612s 15 1.19x
💻 Local Nitro 1.396s (-9.0% 🟢) 2.007s (~) 0.611s 15 1.19x
💻 Local Next.js (Turbopack) 1.481s (+4.2%) 2.073s (+3.3%) 0.593s 15 1.26x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 2.237s (+10.7% 🔺) 3.857s (+5.9% 🔺) 1.620s 8 1.00x
▲ Vercel Nitro 2.548s (+15.3% 🔺) 4.288s (+1.3%) 1.741s 7 1.14x
▲ Vercel Next.js (Turbopack) 4.050s (+8.2% 🔺) 6.049s (+10.5% 🔺) 1.999s 5 1.81x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Promise.all with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 1.247s (-5.3% 🟢) 2.016s (-22.3% 🟢) 0.769s 15 1.00x
🐘 Postgres Express 1.283s (-10.6% 🟢) 2.316s (-10.6% 🟢) 1.033s 13 1.03x
🐘 Postgres Next.js (Turbopack) 1.319s (-1.9%) 3.010s (~) 1.691s 10 1.06x
💻 Local Express 2.220s (-15.5% 🟢) 2.826s (-6.1% 🟢) 0.606s 11 1.78x
💻 Local Nitro 2.563s (+5.5% 🔺) 2.919s (-12.7% 🟢) 0.356s 11 2.06x
💻 Local Next.js (Turbopack) 2.689s (+4.3%) 3.008s (~) 0.319s 10 2.16x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 3.309s (+40.2% 🔺) 5.023s (+23.4% 🔺) 1.714s 6 1.00x
▲ Vercel Express 3.694s (+51.0% 🔺) 5.833s (+59.0% 🔺) 2.139s 6 1.12x
▲ Vercel Next.js (Turbopack) 5.039s (+31.5% 🔺) 7.174s (+35.0% 🔺) 2.135s 5 1.52x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

Promise.all with 50 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 1.444s (-6.9% 🟢) 3.210s (-20.0% 🟢) 1.766s 10 1.00x
🐘 Postgres Express 1.558s (-3.9%) 4.013s (-3.0%) 2.456s 8 1.08x
🐘 Postgres Next.js (Turbopack) 2.919s (+12.3% 🔺) 6.217s (+3.4%) 3.298s 5 2.02x
💻 Local Nitro 5.889s (-14.8% 🟢) 6.613s (-12.1% 🟢) 0.723s 5 4.08x
💻 Local Express 5.986s (-0.6%) 6.615s (~) 0.630s 5 4.15x
💻 Local Next.js (Turbopack) 6.637s (-4.8%) 8.269s (+10.0% 🔺) 1.632s 4 4.60x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 3.892s (+23.2% 🔺) 6.164s (+29.3% 🔺) 2.271s 5 1.00x
▲ Vercel Nitro 4.085s (+29.6% 🔺) 6.376s (+26.7% 🔺) 2.291s 5 1.05x
▲ Vercel Next.js (Turbopack) 4.625s (-6.8% 🟢) 6.833s (+7.1% 🔺) 2.208s 5 1.19x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Promise.race with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 1.164s (-2.0%) 2.006s (~) 0.842s 15 1.00x
🐘 Postgres Express 1.174s (-2.5%) 2.006s (~) 0.832s 15 1.01x
🐘 Postgres Next.js (Turbopack) 1.189s (+0.6%) 2.007s (~) 0.818s 15 1.02x
💻 Local Express 1.370s (-4.4%) 2.007s (~) 0.637s 15 1.18x
💻 Local Nitro 1.431s (-8.6% 🟢) 2.007s (~) 0.576s 15 1.23x
💻 Local Next.js (Turbopack) 1.499s (-1.6%) 2.007s (~) 0.508s 15 1.29x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 2.303s (-5.9% 🟢) 4.657s (+14.8% 🔺) 2.355s 7 1.00x
▲ Vercel Nitro 2.569s (+4.7%) 4.294s (-3.7%) 1.724s 8 1.12x
▲ Vercel Next.js (Turbopack) 3.433s (+1.7%) 5.142s (+0.6%) 1.709s 6 1.49x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Promise.race with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 1.303s (-0.6%) 3.010s (~) 1.707s 10 1.00x
🐘 Postgres Nitro 1.312s (+1.7%) 2.150s (-10.2% 🟢) 0.838s 14 1.01x
🐘 Postgres Express 1.312s (-1.0%) 2.222s (-7.2% 🟢) 0.910s 14 1.01x
💻 Local Express 2.260s (-12.0% 🟢) 2.826s (-6.1% 🟢) 0.566s 11 1.73x
💻 Local Nitro 2.469s (-9.1% 🟢) 2.735s (-20.8% 🟢) 0.266s 11 1.89x
💻 Local Next.js (Turbopack) 2.705s (~) 3.210s (+3.2%) 0.505s 10 2.08x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 2.456s (-1.6%) 4.165s (+8.3% 🔺) 1.709s 8 1.00x
▲ Vercel Nitro 2.765s (+3.9%) 4.357s (-2.6%) 1.592s 7 1.13x
▲ Vercel Next.js (Turbopack) 4.377s (+32.1% 🔺) 6.127s (+17.9% 🔺) 1.750s 5 1.78x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Promise.race with 50 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 1.577s (-2.8%) 4.010s (-9.7% 🟢) 2.433s 8 1.00x
🐘 Postgres Nitro 1.719s (+9.7% 🔺) 3.113s (-22.3% 🟢) 1.394s 10 1.09x
🐘 Postgres Next.js (Turbopack) 2.694s (-9.4% 🟢) 5.682s (-8.6% 🟢) 2.988s 6 1.71x
💻 Local Nitro 6.179s (-21.0% 🟢) 7.014s (-17.7% 🟢) 0.835s 5 3.92x
💻 Local Express 6.324s (-8.3% 🟢) 7.021s (-2.7%) 0.696s 5 4.01x
💻 Local Next.js (Turbopack) 7.152s (+30.4% 🔺) 7.816s (+29.9% 🔺) 0.664s 5 4.53x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 4.489s (-18.3% 🟢) 7.224s (+0.6%) 2.735s 5 1.00x
▲ Vercel Nitro 4.763s (+24.9% 🔺) 7.112s (+29.5% 🔺) 2.349s 5 1.06x
▲ Vercel Express 4.837s (+37.6% 🔺) 6.843s (+31.0% 🔺) 2.006s 5 1.08x

🔍 Observability: Next.js (Turbopack) | Nitro | Express

workflow with 10 sequential data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 0.470s (-9.0% 🟢) 1.013s (-1.0%) 0.543s 60 1.00x
🐘 Postgres Express 0.509s (-18.5% 🟢) 1.006s (-5.0% 🟢) 0.497s 60 1.08x
💻 Local Nitro 0.554s (-22.5% 🟢) 1.005s (~) 0.452s 60 1.18x
🐘 Postgres Next.js (Turbopack) 0.565s (-2.9%) 1.024s (~) 0.459s 59 1.20x
💻 Local Express 0.576s (-0.9%) 1.022s (+1.7%) 0.446s 59 1.23x
💻 Local Next.js (Turbopack) 0.628s (+3.3%) 1.005s (-1.6%) 0.377s 60 1.34x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 3.087s (+5.4% 🔺) 4.759s (-1.7%) 1.672s 13 1.00x
▲ Vercel Express 3.395s (+21.7% 🔺) 5.332s (+16.9% 🔺) 1.937s 12 1.10x
▲ Vercel Next.js (Turbopack) 4.670s (+40.1% 🔺) 6.678s (+37.8% 🔺) 2.008s 9 1.51x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

workflow with 25 sequential data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 0.993s (-20.7% 🟢) 1.413s (-29.6% 🟢) 0.420s 64 1.00x
🐘 Postgres Express 1.219s (-9.7% 🟢) 2.007s (-1.1%) 0.788s 45 1.23x
🐘 Postgres Next.js (Turbopack) 1.298s (-8.6% 🟢) 2.008s (-2.2%) 0.710s 45 1.31x
💻 Local Nitro 1.413s (-21.7% 🟢) 2.006s (~) 0.594s 45 1.42x
💻 Local Express 1.454s (-3.8%) 2.006s (~) 0.551s 45 1.46x
💻 Local Next.js (Turbopack) 1.554s (+3.9%) 2.006s (~) 0.452s 45 1.56x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 7.146s (+8.5% 🔺) 9.083s (+10.0% 🔺) 1.937s 10 1.00x
▲ Vercel Express 7.369s (+12.0% 🔺) 9.453s (+16.3% 🔺) 2.084s 10 1.03x
▲ Vercel Next.js (Turbopack) 10.824s (+28.9% 🔺) 13.101s (+31.0% 🔺) 2.277s 7 1.51x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

workflow with 50 sequential data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 2.043s (-20.5% 🟢) 2.509s (-20.8% 🟢) 0.467s 48 1.00x
🐘 Postgres Express 2.399s (-8.8% 🟢) 3.085s (+0.9%) 0.685s 39 1.17x
🐘 Postgres Next.js (Turbopack) 2.601s (-1.1%) 3.008s (~) 0.408s 40 1.27x
💻 Local Nitro 3.157s (-18.3% 🟢) 3.760s (-10.1% 🟢) 0.603s 32 1.55x
💻 Local Express 3.243s (+0.6%) 3.912s (-1.6%) 0.669s 31 1.59x
💻 Local Next.js (Turbopack) 3.438s (+4.9%) 4.076s (+0.8%) 0.638s 30 1.68x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 14.800s (+12.1% 🔺) 17.526s (+17.4% 🔺) 2.727s 7 1.00x
▲ Vercel Express 15.154s (+17.2% 🔺) 17.976s (+23.8% 🔺) 2.821s 7 1.02x
▲ Vercel Next.js (Turbopack) 21.078s (+19.5% 🔺) 23.674s (+23.0% 🔺) 2.596s 6 1.42x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

workflow with 10 concurrent data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 0.193s (+11.6% 🔺) 1.006s (~) 0.813s 60 1.00x
🐘 Postgres Express 0.199s (-7.1% 🟢) 1.006s (~) 0.806s 60 1.03x
🐘 Postgres Nitro 0.207s (+5.2% 🔺) 1.015s (+0.8%) 0.808s 60 1.07x
💻 Local Nitro 0.466s (-20.1% 🟢) 1.005s (~) 0.539s 60 2.41x
💻 Local Express 0.582s (+22.0% 🔺) 1.040s (+3.4%) 0.457s 58 3.02x
💻 Local Next.js (Turbopack) 0.642s (+8.7% 🔺) 1.022s (+1.7%) 0.381s 59 3.32x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 1.293s (~) 2.795s (+4.4%) 1.503s 22 1.00x
▲ Vercel Nitro 1.444s (+8.1% 🔺) 2.825s (-4.1%) 1.381s 22 1.12x
▲ Vercel Next.js (Turbopack) 2.727s (+32.0% 🔺) 4.577s (+27.7% 🔺) 1.850s 14 2.11x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

workflow with 25 concurrent data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Next.js (Turbopack) 0.298s (+2.7%) 1.018s (+1.1%) 0.720s 89 1.00x
🐘 Postgres Nitro 0.311s (+2.0%) 1.032s (+1.4%) 0.721s 88 1.05x
🐘 Postgres Express 0.317s (-2.3%) 1.006s (-1.1%) 0.689s 90 1.07x
💻 Local Nitro 2.122s (-24.2% 🟢) 2.685s (-25.7% 🟢) 0.563s 34 7.13x
💻 Local Express 2.135s (-5.5% 🟢) 2.766s (-6.1% 🟢) 0.631s 33 7.17x
💻 Local Next.js (Turbopack) 2.778s (-1.0%) 3.417s (+8.6% 🔺) 0.639s 27 9.33x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 1.941s (+20.8% 🔺) 3.697s (+18.9% 🔺) 1.756s 25 1.00x
▲ Vercel Nitro 2.657s (+74.3% 🔺) 4.369s (+42.7% 🔺) 1.712s 22 1.37x
▲ Vercel Next.js (Turbopack) 4.155s (-17.3% 🟢) 6.192s (-6.9% 🟢) 2.037s 15 2.14x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

workflow with 50 concurrent data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 0.396s (-15.3% 🟢) 1.014s (-5.1% 🟢) 0.618s 119 1.00x
🐘 Postgres Next.js (Turbopack) 0.514s (-1.3%) 3.035s (+0.8%) 2.521s 40 1.30x
🐘 Postgres Express 0.516s (~) 1.149s (+8.5% 🔺) 0.633s 105 1.30x
💻 Local Nitro 9.012s (-20.7% 🟢) 10.192s (-18.7% 🟢) 1.179s 12 22.77x
💻 Local Express 9.487s (-5.7% 🟢) 10.279s (-7.6% 🟢) 0.793s 12 23.97x
💻 Local Next.js (Turbopack) 10.718s (-4.2%) 11.576s (-3.0%) 0.858s 11 27.09x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 3.031s (-14.2% 🟢) 5.075s (-1.3%) 2.045s 24 1.00x
▲ Vercel Nitro 3.369s (-17.8% 🟢) 5.636s (-3.7%) 2.267s 22 1.11x
▲ Vercel Next.js (Turbopack) 4.962s (+5.0%) 7.454s (+16.1% 🔺) 2.492s 17 1.64x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Stream Benchmarks (includes TTFB metrics)
workflow with stream

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 1.134s (~) 1.996s (~) 0.001s (-57.1% 🟢) 2.019s (+0.5%) 0.886s 10 1.00x
💻 Local Next.js (Turbopack) 1.144s (+0.5%) 1.967s (~) 0.012s (+7.1% 🔺) 2.020s (~) 0.876s 10 1.01x
💻 Local Nitro 1.146s (-4.5%) 2.004s (~) 0.010s (-22.0% 🟢) 2.018s (~) 0.872s 10 1.01x
🐘 Postgres Express 1.148s (-1.4%) 1.995s (~) 0.001s (+27.3% 🔺) 2.009s (~) 0.861s 10 1.01x
🐘 Postgres Next.js (Turbopack) 1.151s (~) 2.000s (~) 0.001s (+27.3% 🔺) 2.011s (~) 0.860s 10 1.01x
💻 Local Express 1.158s (~) 2.005s (~) 0.010s (-19.0% 🟢) 2.018s (~) 0.860s 10 1.02x

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.012s (+1.5%) 3.044s (-11.9% 🟢) 1.748s (+23.9% 🔺) 5.308s (-1.0%) 3.296s 10 1.00x
▲ Vercel Express 2.180s (+4.2%) 3.470s (+7.2% 🔺) 1.664s (-24.8% 🟢) 5.671s (-4.7%) 3.491s 10 1.08x
▲ Vercel Next.js (Turbopack) 3.693s (+5.8% 🔺) 3.553s (-14.2% 🟢) 1.691s (+18.1% 🔺) 7.052s (+5.7% 🔺) 3.359s 10 1.84x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

stream pipeline with 5 transform steps (1MB)

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 1.598s (+1.9%) 1.993s (-0.5%) 0.000s (-100.0% 🟢) 2.008s (-0.9%) 0.410s 30 1.00x
💻 Local Nitro 1.622s (-5.4% 🟢) 2.008s (~) 0.000s (-98.3% 🟢) 2.011s (-1.0%) 0.389s 30 1.01x
💻 Local Express 1.643s (+5.3% 🔺) 2.008s (~) 0.000s (-96.0% 🟢) 2.011s (-0.6%) 0.368s 30 1.03x
🐘 Postgres Next.js (Turbopack) 1.646s (+1.9%) 2.000s (~) 0.000s (-100.0% 🟢) 2.008s (-1.0%) 0.362s 30 1.03x
🐘 Postgres Nitro 1.648s (+7.2% 🔺) 2.028s (+1.0%) 0.000s (-98.6% 🟢) 2.043s (+0.9%) 0.395s 30 1.03x
💻 Local Next.js (Turbopack) 1.684s (+5.6% 🔺) 1.969s (~) 0.000s (-96.8% 🟢) 2.011s (-0.7%) 0.326s 30 1.05x

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 7.687s (+5.6% 🔺) 9.142s (+9.0% 🔺) 0.000s (-100.0% 🟢) 9.716s (+7.6% 🔺) 2.029s 7 1.00x
▲ Vercel Nitro 7.837s (+36.8% 🔺) 9.097s (+23.0% 🔺) 0.000s (-100.0% 🟢) 9.521s (+14.0% 🔺) 1.684s 7 1.02x
▲ Vercel Next.js (Turbopack) 12.968s (+33.2% 🔺) 13.037s (+26.2% 🔺) 0.000s (-100.0% 🟢) 14.812s (+27.7% 🔺) 1.844s 5 1.69x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

10 parallel streams (1MB each)

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 0.743s (+1.0%) 1.174s (+18.1% 🔺) 0.000s (-100.0% 🟢) 1.202s (+16.8% 🔺) 0.459s 50 1.00x
🐘 Postgres Express 0.761s (-2.1%) 1.062s (~) 0.000s (+Infinity% 🔺) 1.078s (-1.5%) 0.317s 56 1.02x
🐘 Postgres Next.js (Turbopack) 0.980s (-4.8%) 1.428s (+2.4%) 0.000s (+Infinity% 🔺) 1.438s (+2.4%) 0.458s 42 1.32x
💻 Local Nitro 1.242s (-15.1% 🟢) 2.014s (~) 0.001s (+30.0% 🔺) 2.018s (~) 0.775s 30 1.67x
💻 Local Express 1.278s (+1.1%) 2.014s (~) 0.000s (-54.5% 🟢) 2.017s (~) 0.739s 30 1.72x
💻 Local Next.js (Turbopack) 1.431s (+4.3%) 1.980s (~) 0.001s (-21.7% 🟢) 2.018s (~) 0.587s 30 1.93x

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 3.315s (-8.2% 🟢) 4.481s (-10.8% 🟢) 0.000s (-100.0% 🟢) 4.972s (-9.9% 🟢) 1.657s 13 1.00x
▲ Vercel Express 3.453s (-19.1% 🟢) 4.897s (-9.2% 🟢) 0.000s (NaN%) 5.414s (-7.8% 🟢) 1.961s 12 1.04x
▲ Vercel Next.js (Turbopack) 5.891s (+6.0% 🔺) 6.814s (+10.0% 🔺) 0.000s (+12.5% 🔺) 7.835s (+9.3% 🔺) 1.944s 8 1.78x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

fan-out fan-in 10 streams (1MB each)

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 1.378s (-17.9% 🟢) 1.990s (-15.2% 🟢) 0.000s (+Infinity% 🔺) 2.044s (-13.2% 🟢) 0.667s 30 1.00x
🐘 Postgres Express 1.563s (~) 2.172s (~) 0.000s (+Infinity% 🔺) 2.193s (~) 0.630s 28 1.13x
🐘 Postgres Next.js (Turbopack) 2.586s (+0.8%) 3.096s (~) 0.000s (+Infinity% 🔺) 3.115s (~) 0.529s 20 1.88x
💻 Local Express 3.215s (-14.4% 🟢) 3.841s (-11.0% 🟢) 0.001s (+96.9% 🔺) 3.845s (-10.9% 🟢) 0.630s 16 2.33x
💻 Local Nitro 3.409s (-16.6% 🟢) 3.900s (-15.2% 🟢) 0.001s (+9.4% 🔺) 3.905s (-15.3% 🟢) 0.496s 16 2.47x
💻 Local Next.js (Turbopack) 3.872s (+5.1% 🔺) 4.277s (+7.1% 🔺) 0.001s (+69.6% 🔺) 4.317s (+7.0% 🔺) 0.445s 14 2.81x

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 5.124s (-4.8%) 6.386s (-8.1% 🟢) 0.000s (NaN%) 6.872s (-7.7% 🟢) 1.748s 9 1.00x
▲ Vercel Express 5.713s (-8.9% 🟢) 7.269s (+0.6%) 0.000s (~) 7.807s (+2.1%) 2.094s 8 1.12x
▲ Vercel Next.js (Turbopack) 8.008s (-5.7% 🟢) 9.261s (+5.1% 🔺) 0.000s (NaN%) 10.474s (+6.0% 🔺) 2.465s 6 1.56x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

Summary

Fastest Framework by World

Winner determined by most benchmark wins

World 🥇 Fastest Framework Wins
💻 Local Express 10/21
🐘 Postgres Nitro 13/21
▲ Vercel Express 10/21
Fastest World by Framework

Winner determined by most benchmark wins

Framework 🥇 Fastest World Wins
Express 🐘 Postgres 17/21
Next.js (Turbopack) 🐘 Postgres 16/21
Nitro 🐘 Postgres 18/21
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Slurp: Time from first byte to complete stream consumption (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • Samples: Number of benchmark iterations run
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world (local development)
  • 🐘 Postgres: PostgreSQL database world (local development)
  • ▲ Vercel: Vercel production/preview deployment
  • 🌐 Turso: Community world (local development)
  • 🌐 MongoDB: Community world (local development)
  • 🌐 Redis: Community world (local development)
  • 🌐 Jazz: Community world (local development)
  • 🌐 Redis: Community world (local development)
  • 🌐 Redis + BullMQ: Community world (local development)
  • 🌐 Cloudflare: Community world (local development)
  • 🌐 MySQL: Community world (local development)
  • 🌐 Azure: Community world (local development)
  • 🌐 NATS JetStream: Community world (local development)
  • 🌐 Upstash: Community world (local development)
  • 🌐 Platformatic: Community world (local development)

📋 View full workflow run

@github-actions

github-actions Bot commented Jun 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

✅ All tests passed

Summary

Passed Failed Skipped Total
✅ ▲ Vercel Production 1442 0 230 1672
✅ 💻 Local Development 1605 0 219 1824
✅ 📦 Local Production 1605 0 219 1824
✅ 🐘 Local Postgres 1593 0 231 1824
✅ 🪟 Windows 152 0 0 152
✅ 📋 Other 885 0 179 1064
Total 7282 0 1078 8360

Details by Category

✅ ▲ Vercel Production
App Passed Failed Skipped
✅ astro 125 0 27
✅ example 125 0 27
✅ express 125 0 27
✅ fastify 125 0 27
✅ hono 125 0 27
✅ nextjs-turbopack 149 0 3
✅ nextjs-webpack 149 0 3
✅ nitro 125 0 27
✅ nuxt 125 0 27
✅ sveltekit 144 0 8
✅ vite 125 0 27
✅ 💻 Local Development
App Passed Failed Skipped
✅ astro-stable 127 0 25
✅ express-stable 127 0 25
✅ fastify-stable 127 0 25
✅ hono-stable 127 0 25
✅ nextjs-turbopack-canary 133 0 19
✅ nextjs-turbopack-stable 152 0 0
✅ nextjs-webpack-canary 133 0 19
✅ nextjs-webpack-stable 152 0 0
✅ nitro-stable 127 0 25
✅ nuxt-stable 127 0 25
✅ sveltekit-stable 146 0 6
✅ vite-stable 127 0 25
✅ 📦 Local Production
App Passed Failed Skipped
✅ astro-stable 127 0 25
✅ express-stable 127 0 25
✅ fastify-stable 127 0 25
✅ hono-stable 127 0 25
✅ nextjs-turbopack-canary 133 0 19
✅ nextjs-turbopack-stable 152 0 0
✅ nextjs-webpack-canary 133 0 19
✅ nextjs-webpack-stable 152 0 0
✅ nitro-stable 127 0 25
✅ nuxt-stable 127 0 25
✅ sveltekit-stable 146 0 6
✅ vite-stable 127 0 25
✅ 🐘 Local Postgres
App Passed Failed Skipped
✅ astro-stable 126 0 26
✅ express-stable 126 0 26
✅ fastify-stable 126 0 26
✅ hono-stable 126 0 26
✅ nextjs-turbopack-canary 132 0 20
✅ nextjs-turbopack-stable 151 0 1
✅ nextjs-webpack-canary 132 0 20
✅ nextjs-webpack-stable 151 0 1
✅ nitro-stable 126 0 26
✅ nuxt-stable 126 0 26
✅ sveltekit-stable 145 0 7
✅ vite-stable 126 0 26
✅ 🪟 Windows
App Passed Failed Skipped
✅ nextjs-turbopack 152 0 0
✅ 📋 Other
App Passed Failed Skipped
✅ e2e-local-dev-nest-stable 127 0 25
✅ e2e-local-dev-tanstack-start- 127 0 25
✅ e2e-local-postgres-nest-stable 126 0 26
✅ e2e-local-postgres-tanstack-start- 126 0 26
✅ e2e-local-prod-nest-stable 127 0 25
✅ e2e-local-prod-tanstack-start- 127 0 25
✅ e2e-vercel-prod-tanstack-start 125 0 27

📋 View full workflow run

This was referenced Jun 26, 2026
Draft
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ijjk ijjk Awaiting requested review from ijjk ijjk will be requested when the pull request is marked ready for review ijjk is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@NathanColosimo