Repository navigation
Dashboard usability test: six proposals, four implemented in drafts, two need a call #220
Description
A task-based usability test ran on a local merge of #204, #205, #206, #219 and r5-api-scenarios. Seven tasks covered four personas: policy analyst, rates trader, journalist and developer. Playwright started every task at / and counted clicks at 1440 and 375 px. All seven tasks can be completed, and no page scrolls horizontally at 375. The small fixes are already pushed to the owning PRs:
- r5 UI: question router on /models, copy-paste /connect, glossary, home and contact #204 0d9711a: Combine step 1 gets a visible "See the scored 1p example →" link. The validation tab heading changes from "How each model is validated" to "How each model is checked".
- Model pages: one-column methodology, one overview order, names not ids #206 234fdd9: the /pe macro-feedback links now go to
/obr#answers. - Round 5 ui-data: forecast-evaluation scoreboard with full-height rail, flat archive, chart units and sources #205 1973dd2: the Forecasts expander links to the Growth and Inflation tables. The topic tables now say their bands are raw while the track-record fans are re-calibrated. No number changed.
The proposals below need a judgement call or a bigger change, so I have not made them.
- Say whether the macro feedback changes the cost. An analyst asked "does the macro feedback change it?" The site answers in GDP (about 0.34× the revenue by q12, /obr#answers). It never says plainly that the costing itself is not revised: borrowing is not returned by the adapter, and that appears only in the "Cannot answer" row. Proposal: one sentence at the /obr worked example. (obr/index.html, Model pages: one-column methodology, one overview order, names not ids #206)
- Show each model's verdict on its router card. On /models with "Cost a reform" and "UK" selected, the OBR card's only quality signal is the tag "published anchor". The verdict sits inside the collapsed "What it is, strengths and limits". Proposal: show the one-line verdict on the card itself. It is already on the model's own "at a glance" panel, so link to that or reuse it, without adding a second copy of the fact. (models/index.html, r5 UI: question router on /models, copy-paste /connect, glossary, home and contact #204)
- Add a yes/no lead on /models/validation. A journalist asking "is any model validated?" finds four classes and seven verdicts, but no line that answers the question. The verdict column supports one: no model is validated as a forecaster, and the strongest evidence is replication of a published anchor. This needs your sign-off because it is a summary claim. (r5 UI: question router on /models, copy-paste /connect, glossary, home and contact #204)
- Put the CPI and GDP numbers next to the fans on /forecasts. A rates reader needs 3 clicks and 2 expanders to read the next four quarters (Forecasts, then the Inflation tab, then "Quarter by quarter"). The fans have no values and no hover. Decision: table the re-calibrated bands next to the fan, or keep only the raw table on the topic tab (it is now labelled). (forecasts/make_open_fans.py, Round 5 ui-data: forecast-evaluation scoreboard with full-height rail, flat archive, chart units and sources #205)
- Show the answer first on Same question. For +100bp, the benchmark box comes first and the answer table starts below the fold at 1440. Proposal: put the table first and the benchmark directly under it, or collapse the benchmark to one line. (models/compare/build.py, Models: 'Same question, every model' comparison view (/models/compare) #219)
- Fix the Forecasts sub-tab row at desktop width.
economy-subnav-rowis 1599 px wide in a 1440 viewport, so "Tax and benefit reform" is cut under the fade. Options: shorter labels ("Tax and benefits"), or the UK/US switch on its own row. (site_nav.py, r5 UI: question router on /models, copy-paste /connect, glossary, home and contact #204) - Developer example. The /connect prompt "a 25 basis point US rate cut … hank_shock" sits next to
--size -0.0025, which is a 100bp-annualised move. This is already tracked in hank_shock docstring: -0.0025 is a 100bp-annualised move, not '25bp'; productivity 0.01 is 2.1% TFP, not 1% #211, so fix it on /connect when hank_shock docstring: -0.0025 is a 100bp-annualised move, not '25bp'; productivity 0.01 is 2.1% TFP, not 1% #211 lands.
Evidence: the persona table, paths and click counts are in the r5 ux-test notes. The screenshots are local. An MCP check, frbus_shock(rffintay_aerr, 1.0, horizon=12), returned GDP −0.551% at 2027Q4, the same as /models/compare.
Recommendation: do 1, 2 and 5 now; they are wording and order changes within one PR each. Items 3 and 4 need your call.
🤖 Generated with Claude Code
Activity
Router cards show each model's verdict line, read off its page's verd…
Validation tab: one-line answer to 'is any model validated?', read of…
OBR worked example: say the second round changes GDP, not the costing (…
Band legends say which bands they draw: topic and unemployment fans '…
Forecasts sub-tab strip fits at 1440 px: topic tabs drop the repeated…
Forecasts: each GDP/CPI fan gets its quarter-by-quarter numbers insid…
Same question: the answer table comes first, the benchmark box under …
vahid-ahmadi commentedon Nov 6, 2026
All six proposals are now implemented, one commit per item, so any of them can be dropped on its own. No number changed. Item 7 still waits on #211.
rounds/2026-07-29/boe-svar.json, generated byforecasts/score.py.Checks: the merge of base + #204 + #206 + #205 + #219 has no conflicts. On that merge, pytest gives 1241 passed (with
PYTHONPATH=integration/src), site_contract passes, the topics/score/open-fans/compare--checkall pass, and the link check finds 0 broken links. No page scrolls horizontally at 375 px. Screenshots at 1440 and 375 px are local in shots/r5/uxfix.🤖 Generated with Claude Code
[-]Dashboard usability test (r5): seven tasks pass; six proposals that need a call[/-][+]Dashboard usability test: six proposals, four implemented in drafts, two need a call[/+]Forecasts: say when the record was last scored, above the scoreboard …
Compare: the takeaway line names the model behind each verdict
vahid-ahmadi commentedon Nov 6, 2026
Usability test 2: before and after
I re-ran the same 4 personas and 7 tasks, plus 3 new tasks, on a fresh local merge of base + #204 + #206 + #205 + #219. The merge has no conflicts and was not pushed. I tested at 1440 and 375 px.
No number changed. Checks on the merge: pytest 1256 passed, 20 skipped; site_contract OK; generator
--checkOK; 0 broken links; no horizontal scroll at 375 px.