Skip to content

feat(web): add public leaderboard provider race endpoint#4787

Open
pedroheyerdahl wants to merge 3 commits into
mainfrom
leaderboard-provider-race-endpoint
Open

feat(web): add public leaderboard provider race endpoint#4787
pedroheyerdahl wants to merge 3 commits into
mainfrom
leaderboard-provider-race-endpoint

Conversation

@pedroheyerdahl

@pedroheyerdahl pedroheyerdahl commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a public Snowflake-backed endpoint GET /api/public/leaderboard-provider-race that serves weekly token volume per model lab for the kilo.ai/leaderboard/race visualization. The race page currently hardcodes 53 weeks of data in the landing repo; this endpoint lets it load dynamically.

The query groups kilo_dw.dbt_prod.usage_daily at week × model_provider_company × is_open_weights (fixed start 2025-07-01 → most recent complete day). One payload drives both the per-lab race and an open-weight vs proprietary toggle. The lab mapping (model_provider_company, is_open_weights) comes from the dbt model_dim seed rather than being re-derived in SQL — this is the same gap the Omni dashboard's CASE workaround was papering over.

Follows the existing createPublicSnowflakeReport pattern (Redis-cached 1h, CORS, Sentry) used by leaderboard-model-usage and leaderboard-model-provider-usage.

Verification

  • oxlint clean on both changed files (0 errors)
  • Route is a structural copy of the adjacent leaderboard-model-usage/route.ts; tsgo reports no errors on the new files (full-repo typecheck blocked locally by unbuilt workspace packages — pre-existing, unrelated)
  • Manual: hit the endpoint on a preview deployment once the dbt backfill has landed (see dependency below)

Visual Changes

N/A

Reviewer Notes

  • Deploy dependency: the query reads usage_daily.is_open_weights, added by kilocode-dbt PR feat(kiloclaw): controller, device pairing, and dashboard redesign #480 + its backfill (2026-07-25-backfill-is-open-weights.sql). That backfill must run before this endpoint returns data; until then the query errors and the endpoint returns 502 (cached null).
  • Response shape: [{ weekStart: "YYYY-MM-DD", provider: string, isOpenWeights: boolean | null, tokens: number }]. isOpenWeights is null for unmapped models (is_open_weights NULL in model_dim) so the client can distinguish "unknown" from a confirmed closed-weight lab; only explicit 'true'/'false' become a boolean. is_open_weights is per-model in dbt, so a lab like Alibaba appears in both open and closed rows; the client sums across isOpenWeights for the lab race and across provider for the open/closed view.
  • model_provider_company = 'other' (unmapped models) is included so the client can bucket it; drop here if we'd rather exclude upstream.

Comment thread apps/web/src/app/api/public/leaderboard-provider-race/route.ts Outdated
@kilo-code-bot

kilo-code-bot Bot commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Full review of the current head (the branch's commit SHAs were rewritten since the prior review, so the previous incremental-diff base commit no longer exists in history); the public leaderboard provider race endpoint correctly maps unmapped/NULL is_open_weights values to null rather than coercing to false, and no new issues were found.

Files Reviewed (2 files)
  • apps/web/src/app/api/public/leaderboard-provider-race/route.ts - previously flagged NULL-coercion issue remains resolved; no new issues
  • apps/web/src/lib/redis-keys.ts - no issues
Previous Review Summaries (3 snapshots, latest commit aa9aee5)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit aa9aee5)

Status: No Issues Found | Recommendation: Merge

Executive Summary

The previously flagged NULL-coercion issue in the provider race endpoint is now resolved by mapping unmapped is_open_weights values to null instead of false.

Files Reviewed (1 file changed since last review)
  • apps/web/src/app/api/public/leaderboard-provider-race/route.ts - previous WARNING resolved (parseOpenWeights now returns null for unmapped/NULL values instead of coercing to false); no new issues

Previous review (commit 3431d73)

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/src/app/api/public/leaderboard-provider-race/route.ts 54 is_open_weights NULL (e.g. unmapped model_provider_company = 'other' rows) is silently coerced to false instead of being validated or flagged
Files Reviewed (1 file changed since last review)
  • apps/web/src/app/api/public/leaderboard-provider-race/route.ts - query updated to exclude the partial current week; no new issues

Fix these issues in Kilo Cloud

Previous review (commit e1a74e2)

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/src/app/api/public/leaderboard-provider-race/route.ts 53 is_open_weights NULL (e.g. unmapped model_provider_company = 'other' rows) is silently coerced to false instead of being validated or flagged
Files Reviewed (2 files)
  • apps/web/src/app/api/public/leaderboard-provider-race/route.ts - 1 issue
  • apps/web/src/lib/redis-keys.ts - 0 issues

Fix these issues in Kilo Cloud


Reviewed by claude-sonnet-5 · Input: 32 · Output: 8K · Cached: 735.4K

Review guidance: REVIEW.md from base branch main

Weekly token volume per model lab for the kilo.ai/leaderboard/race
visualization. Grouped at week x model_provider_company x is_open_weights
so one payload drives both the per-lab race and an open-weight vs
proprietary toggle. Lab mapping comes from the dbt model_dim seed rather
than being re-derived in SQL.

Depends on the kilocode-dbt usage_daily backfill that adds is_open_weights.
Match the prior '52 complete weeks' behavior so the latest returned week
is always complete (keeps the insights 'latest complete week' stat correct).
Map only explicit 'true'/'false' strings to a boolean and return null for
NULL/unmapped models so the open/closed split isn't silently corrupted by
model_dim mapping gaps. Also removes the string===boolean comparison that
broke the typecheck (TS2367).
@pedroheyerdahl
pedroheyerdahl force-pushed the leaderboard-provider-race-endpoint branch from aa9aee5 to c217b0a Compare July 27, 2026 13:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant