feat(master): serve the experiment list as recently-active plus search

[beinan]

⚠️ Stacked on #209 → #210. Both of those commits are included here. Merge order: #209 → #210 → this. Once they land, this diff reduces to the UI commit alone.

Why this exists

#210 removes cold experiments from the stats table — that's what keeps a scan round proportional to active work. But on its own it also removes them from GET /experiments, which is only acceptable once they stay findable another way. This PR is what makes retirement safe to turn on.

	behaviour
default list	the stats table — experiments written recently enough not to have been retired
search	stats table plus the registry for names it no longer holds; cold matches observed on demand
detail	on a stats miss, resolve via registry and observe on demand rather than returning 404

Retirement therefore removes an experiment from the default view and never makes it unfindable.

The default is also the right one at scale on its own merits: a flat list of every experiment ever created is neither renderable nor interesting once there are tens of thousands.

Two deliberate decisions

1. observe_cold does not write a stats row. Reading about a retired experiment must not make it hot again — otherwise browsing the UI would silently undo retirement, and the table would creep back toward holding everything, which is exactly the state retirement exists to prevent.

It also takes no MasterState, so it structurally cannot write one. A refactor that hands it one has to justify itself. There's a test asserting the no-rehydration property.

2. Search bounds its on-demand observations by page size. A query matching thousands of cold experiments must not open thousands of datasets to render one page.

Known gap — stating it rather than hiding it

The two handlers changed here live in routes.rs, whose tests all require etcd and are #[ignore]d, so CI does not cover them. observe_cold is tested in scanner.rs, which CI does run.

So the registry-fallback logic in the handlers is verified locally but not by CI. That's a pre-existing gap in the test structure, worth fixing as its own change rather than as a fourth concern here — and it's the kind of blind spot that only became visible after #202 made CI actually run integration tests.

Testing

Full workspace: 178 core + 30 master + 51 server + 5 concurrency + 1 merge-cleanup, all pass. clippy -D warnings and fmt clean.

The complete picture

This finishes the line of work from the original report (/experiments at 30–43s):

	problem	fix
#207	write amplification — 100 versions/round	one snapshot commit; 84× fewer versions
#209	scan cost — full observe of everything, every round	skip unchanged; cold = one open
#210	table size — held every experiment ever created	retire after 7 days idle, drained first
this	retirement made cold experiments invisible	active-N default + registry-backed search
#208	all of the above failing silently	maintenance failure metrics

🤖 Generated with Claude Code