Add workflow analytics world APIs#2234
Merged
Merged
Conversation
🦋 Changeset detectedLatest commit: 7e9fc67 The changes in this PR will be included in the next version bump. This PR includes changesets to release 20 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Contributor
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 1 step💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 25 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 50 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.all with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.race with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 10 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 25 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 50 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 10 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 25 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 50 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Stream Benchmarks (includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) stream pipeline with 5 transform steps (1MB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) 10 parallel streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) fan-out fan-in 10 streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
|
Contributor
🧪 E2E Test Results✅ All tests passed Summary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
|
karthikscale3
changed the title
karthikscale3
force-pushed
the
codex/workflow-analytics-apis
branch
from
066d69c to
b733f17
Compare
This was referenced Jun 26, 2026
| workflowEncryptionEnabled: NullableBooleanSchema, | ||
| }); | ||
|
|
||
| export const AnalyticsEventSchema = z.object({ |
Member
There was a problem hiding this comment.
This duplicates a lot of schemas from the world otherwise, when I'd expect we want the analytics schemas to be keeping parity with the regular storage schemas. So maybe I'd define those as the storage schema +- some attributes, instead of copying entirely
Contributor
Author
There was a problem hiding this comment.
I considered deriving these from the storage schemas, but I think keeping analytics schemas standalone is the safer contract here.
The analytics namespace is intentionally a metadata-only read model, not a storage object with fields removed. Several shapes diverge from runtime storage: events are flattened instead of nested under eventData, hook rows omit secret/runtime ownership fields, and the read model carries fields like workflowCoreVersion, workflowEncryptionEnabled, region, vercelId, and requestId.
I’m also wary of deriving via omit, because the failure mode for future storage fields is bad: payload/secret fields could become accepted by analytics unless we remember to exclude them. With standalone schemas, every analytics field is explicitly opted in and reviewable.
I did reuse the shared enum schemas where parity matters (WorkflowRunStatusSchema, StepStatusSchema, EventTypeSchema, WaitStatusSchema). Happy to add a short comment documenting that these schemas are intentionally standalone metadata-only read contracts, if that would help make the boundary clearer.
TooTallNate
approved these changes
Jun 30, 2026
TooTallNate
left a comment
TooTallNate
left a comment
Member
There was a problem hiding this comment.
Approve — clean, well-isolated metadata-only read namespace
Solid foundation for the analytics surface. The design choices are right where it matters:
analytics?: Analyticsis optional onWorld— backends without a metadata read path leave it unset and consumers fall back to runtime storage APIs. Clean capability detection, no breaking change for local/postgres.- Genuinely metadata-only. I grepped the entire analytics layer (schema + vercel impl): zero
tokenreferences and no payload-bearing fields (input/output/result/error/payload). Hooks carrystatus/timestamps/flags but notoken, runs/steps/events/waits carry ids/statuses/timestamps/names/error codes only. This is exactly the separation you want — the secret and payloads structurally cannot leak through this surface. - Injection-safe client. Every path parameter goes through
encodeURIComponent; the only un-encoded interpolations are list endpoints with no path params (query strings built viaURLSearchParams). Every response is Zod-validated against the analytics schema. createVercelWorldwiring is correct (analytics: createAnalytics(config)), the new schemas are exported from@workflow/world, and the changeset (minorfor world + world-vercel) is right.
Built both packages; tsc --noEmit clean on @workflow/world and @workflow/world-vercel.
On @VaguelySerious's schema-duplication point
It's a fair maintainability concern, but I'd lean toward keeping these as standalone schemas rather than deriving them from the storage schemas (StorageSchema.omit(...)), for two reasons:
- They're not a strict subset. The analytics rows add fields the storage schemas don't have (
region,vercelId,requestId,runCreatedAton events; the flattened top-level shape generally), and omit many. A derivation would beomit(...).extend(...)with enough divergence that it wouldn't actually reduce surface area much. - Coupling has a security downside here. The whole point of this namespace is that payload/secret fields can never appear in it. If analytics derived from the storage schema via
omit, a future field added to the storage schema would land in analytics by default unless someone remembers to omit it — the failure mode is "secret leaks into the metadata surface," which is exactly what we don't want to make the default. Independent schemas make adding an analytics field an explicit, reviewable act.
That said, a middle ground worth considering (non-blocking): a couple of shared enum/primitive building blocks (the status enums are already imported from the storage modules — good) and a comment on each analytics schema noting it intentionally mirrors a subset of the storage shape, so the parity expectation is documented even though the schemas stay separate. I'd ship as-is.
…ics-apis # Conflicts: # packages/world/src/interfaces.ts
VaguelySerious
approved these changes
Jul 1, 2026
VaguelySerious
left a comment
VaguelySerious
left a comment
Member
There was a problem hiding this comment.
LGTM, I understand the reasoning behind the zod duplication. Maybe we should add a note in the agents file to ensure we don't accidentally cause drift here, but not required
Contributor
|
Backport PR opened against |
This was referenced Jul 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Adds a new optional, metadata-only
world.analytics.*namespace for workflow observability reads.@workflow/worldanalytics?: Analyticsto theWorldinterfacecreateAnalytics()in@workflow/world-vercelcreateVercelWorld()to exposeworld.analytics@workflow/worldand@workflow/world-vercelWhy
This gives observability surfaces (CLI, dashboards) an explicit, read-only
metadata surface for listing and trace views, separate from the canonical
runtime storage APIs. Payloads, RemoteRef resolution, inputs, outputs, and
error hydration remain on the existing runtime APIs (
runs,steps,events,hooks).analyticsis optional on theWorldinterface: backends that do notprovide a metadata read path simply leave it unset, and consumers fall
back to the runtime storage APIs.
API surface
world.analyticsexposes read-onlylist/getfor:runs— list (filter byworkflowName,status) and get by idsteps— list per run and get by idevents— list per run, list bycorrelationId, and get by idhooks— list (optionally per run) and get by idwaits— list per run (filter bystatus) and get by idAll return metadata only (ids, status, timestamps, names, error codes,
flags) — no payload-bearing fields.
Validation
pnpm --filter @workflow/world exec tsc --noEmit --pretty falsepnpm --filter @workflow/world-vercel exec tsc --noEmit --pretty falsepnpm exec biome check packages/world/src/analytics.ts packages/world/src/interfaces.ts packages/world/src/index.ts packages/world-vercel/src/analytics.ts packages/world-vercel/src/index.tsMerge order
This work lands as a stack of four PRs on the
world.analyticsread-path:world.analyticsnamespace (basemain) — merge firstworld.analytics(stacked on Add workflow analytics world APIs #2234)world.analytics(stacked on Add workflow analytics world APIs #2234)#2648 and #2647 both depend only on #2234 and are independent of each other. #2652 builds on #2647. After #2234 merges to
main, the stacked PRs auto-retarget.