Add leaderboard.querygym.com Astro app + workspace#5
Merged
radinhamidi merged 7 commits intomainfrom Apr 29, 2026
Merged
Conversation
Creates a Cloudflare Pages-deployable leaderboard at
reproducibility/site/, plus the pnpm workspace and shared design
system that PR2 (marketing site) will also build on. The querygym
Python package is unaffected — pyproject.toml's
[tool.setuptools.packages.find] include rule and MANIFEST.in's
prune directives keep the wheel and sdist identical to today.
What ships:
- pnpm workspace at the repo root (package.json, pnpm-workspace.yaml,
pnpm-lock.yaml). engines pinned to node>=20, pnpm>=9.
- web/shared/: design tokens, Tailwind preset, shared Header/Footer,
shared logo asset. Imports as @qg/shared.
- reproducibility/site/: Astro 5 + Tailwind + Solid integrations.
- scripts/build-data.ts: walks reproducibility/data/results.csv +
manifest.json + runs/**/*.json + dataset_registry.yaml; emits
per-view JSON shards (overview, datasets, methods, models, runs,
views/dataset-{id}, views/method-{id}, views/model-{id}). Emits
valid empty shards when results.csv has 0 rows so the build still
succeeds.
- Pages: /, /datasets/, /datasets/[id], /methods/, /models/,
/runs/[run_id], /about. Per-dataset shards are loaded via Astro's
import.meta.glob (statically analyzable). Empty state is rendered
for datasets with no runs and on /.
- Visual language matches the existing Jekyll site (purple→pink
gradient, Inter, dark-mode toggle ported from default.html).
- CI workflow (.github/workflows/leaderboard-build.yml) runs on PRs
touching reproducibility/site/**, reproducibility/data/**,
web/shared/**, package.json, pnpm-workspace.yaml,
dataset_registry.yaml. Does pnpm install + build:data + build,
uploads dist as an artifact. No Python steps.
- Defensive .gitignore entries for node_modules, .astro, dist, and
the generated reproducibility/site/src/data/. Anchored existing
/datasets/ /scratch/ etc. rules to the repo root so they no longer
collide with the leaderboard's pages/datasets/ folder.
- Defensive MANIFEST.in excludes for package.json, pnpm-* lockfiles,
node_modules, .astro — belt-and-suspenders against future MANIFEST
sweeps.
Verified locally:
- pnpm install + pnpm -F @qg/leaderboard build:data + build:
37 pages with empty results.csv (32 datasets via registry + index +
methods/models/about + 2 dataset index pages).
- Round-trip with the existing test fixture: 38 pages, dataset page
shows the row with metric values, run-detail page renders.
- python -m build: sdist contains zero web/repro-site/node leaks;
wheel is 36 files, all under querygym/.
Cloudflare Pages project (configured manually in CF dashboard):
- Project: querygym-leaderboard
- Repo: ls3-lab/QueryGym, branch: main
- Root dir: reproducibility/site
- Build cmd: cd ../.. && pnpm install --frozen-lockfile &&
pnpm -F @qg/leaderboard build:data &&
pnpm -F @qg/leaderboard build
- Output dir: reproducibility/site/dist
- Custom domain: leaderboard.querygym.com
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI runs `pytest reproducibility/tests` directly (not `python -m pytest`), which doesn't add the repo root to sys.path. Without the explicit pythonpath, `from reproducibility.lib import ...` fails with ModuleNotFoundError. Pin it in [tool.pytest.ini_options] so the test setup works regardless of invocation style. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
pnpm/action-setup@v4 errored because both the workflow's `version: 10` and package.json's `packageManager: "pnpm@10.14.0"` were set, which triggers ERR_PNPM_BAD_PM_VERSION. Removing `version` from the workflow; the action picks up the canonical version from packageManager. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The workflow's path filter previously skipped any change that didn't touch reproducibility/**, the example pipeline, or dataset_registry.yaml. That meant the pyproject.toml pytest config fix (previous commit) didn't actually re-run CI. Add pyproject.toml and the workflow file itself to the path filter, plus workflow_dispatch for manual reruns. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Even with pythonpath = ["."] in pytest config, CI's Python 3.9 + pytest 8.4 + cov plugin combination doesn't make `reproducibility` importable as a top-level package when invoked as `pytest`. `python -m pytest` always prepends cwd to sys.path, which fixes it. This is the canonical workaround for "from <repo-root-pkg> import" in CI without making the package pip-installable. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The standard Python .gitignore line `lib/` (no leading slash) was matching reproducibility/lib/ everywhere in the tree, which silently hid emit.py, validate.py, and __init__.py from the previous commit. CI couldn't import reproducibility.lib because the directory itself didn't exist on the remote — explaining the persistent ModuleNotFoundError despite the pytest config + python -m pytest fixes. Anchor the Python distribution directories (build/, dist/, lib/, lib64/, eggs/, etc.) to the repo root with a leading slash so they match only top-level dirs, not nested project directories that legitimately use the same names. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributors generate manifest.json on machines that may or may not have querygym installed (the lib reads __version__ via lazy import and falls back to "unknown" on ImportError). CI always has querygym installed via `pip install -e ".[repro,dev]"`, so the regenerated manifest's querygym_version differed from the committed one and --check failed on a purely informational field. content_hash still pins the actual aggregate data byte-for-byte. schema_version, run_count, and row_count remain in the comparison — those are real correctness signals. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Builds the leaderboard site at
reproducibility/site/(Astro + Tailwind + Solid) and the pnpm workspace + shared design system (web/shared/) that PR2 (marketing-site redesign) will also build on. Designed to deploy as a Cloudflare Pages project atleaderboard.querygym.com.Depends on #4 — the data pipeline. Reads
reproducibility/data/results.csv+runs/**/*.json+manifest.json+dataset_registry.yamlat build time. Once #4 merges, this PR's diff will narrow to just the leaderboard files.Highlights
_layouts/default.html).pnpm -F @qg/leaderboard build:data(TypeScript script that walks the canonical artifacts) →pnpm -F @qg/leaderboard build(Astro static output). Reproducible, deterministic, no SSR./,/datasets/,/datasets/[id],/methods/,/models/,/runs/[run_id],/about. Datasets list comes fromdataset_registry.yaml(always non-empty); per-dataset shards loaded viaimport.meta.globso Astro can statically analyze them.leaderboard-build.yml) runspnpm install+build:data+buildon PRs touching the relevant paths. No Python steps. Uploadsdist/as an artifact for review preview.Python package isolation — verified locally
python -m build(reproducibility/(site|lib|scripts|tests|data)|web/|package\.json|pnpm|node_modules)querygym/(unchanged from main)querygym/is untouched.pip install querygymis byte-identical to today.Cloudflare Pages — to configure once
After this PR merges, in the Cloudflare dashboard:
querygym-leaderboardls3-lab/QueryGym, production branchmainreproducibility/sitecd ../.. && pnpm install --frozen-lockfile && pnpm -F @qg/leaderboard build:data && pnpm -F @qg/leaderboard buildreproducibility/site/distleaderboard.querygym.comThen add the CNAME for
leaderboardin thequerygym.comDNS zone (CF auto-creates it for same-account zones).Test plan
pnpm installsucceeds (399 packages, lockfile committed).pnpm -F @qg/leaderboard build:data && pnpm -F @qg/leaderboard buildsucceeds with emptyresults.csv→ 37 static pages.sample_run.jsonintoruns/..., ran aggregator, rebuilt → 38 pages, dataset page shows the row with metric0.3709, run-detail page renders.python -m buildproduces a clean sdist + wheel (Python package isolation verified).leaderboard-build.ymlworkflow.Out of scope (separate PRs)
web/site/) — PR2 of the website rollout.🤖 Generated with Claude Code