test: add the toolkit join parity harness

[teallarson]

Why

The generator is being split into upstream data, enrichment, and curation layers. This adds a deterministic parity gate that answers one question: does joining those layers still reproduce the toolkit JSON we ship today?

It turns output preservation from a manual review task into a mechanical check, protecting the pipeline as its sources are extracted and reorganized.

How it fits

catalog snapshot + enrichment + curation
                    │
                    v
              real merger
                    │
                    v
          parity check vs shipped JSON

The harness sits alongside generation rather than replacing it: it uses the production merger, reports the first structural difference by JSON path, and makes future pipeline slices independently verifiable.

Changes

• Added a paginated catalog snapshot script that records total_count and rejects truncated fetches.
• Added a join verifier using the real merger, with committed toolkit data as the current baseline until extracted layers land.
• Added tests for the zero-difference case, enrichment changes, missing catalog tools, and the catalog response shape.

Verification

• Re-joining all 117 committed toolkits produces zero differences.
• Injected enrichment changes fail with a precise JSON path.
• Optional-field equivalence and volatile generatedAt are handled structurally to avoid false positives.

Independent of #1112 at merge time, but intended to validate it and the subsequent enrichment/source-extraction slices.

---

Note

Low Risk
New scripts and tests only; no changes to the production generator merge or shipped data paths.

Overview
Adds a join parity gate for the upcoming catalog / enrichment / curation split: mechanically check that those layers still produce the same shipped toolkit JSON.

capture-catalog-snapshot.ts paginates Engine /v1/tool_metadata (aligned with EngineApiSource), records total_count, and refuses to write truncated snapshots. .gitignore keeps the ~10 MB local snapshot out of git.

verify-toolkit-join.ts loads a snapshot through parseToolMetadataResponse, runs the real mergeToolkit path (design-system metadata + provider resolver), overlays frozen enrichment (codeExample, secretsInfo, summary), and compares to data/toolkits with a structural diff (ignores generatedAt, treats optional null/absent keys as equivalent). Non-zero exit on mismatch or missing catalog tools. Until separate layer dirs exist, enrichment/curation default to the committed toolkit files.

verify-toolkit-join.test.ts covers the diff helpers, catalog reshape, full round-trip over all committed toolkits, and failure cases (enrichment drift, missing catalog entries).

^{Reviewed by Cursor Bugbot for commit f18ce96. Bugbot is set up for automated code reviews on this repo. Configure here.}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview	Aug 12, 2026 7:16pm

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit e9a38de. Configure here.}