Skip to content

Prepare fixed-grid MEPV scans and share stable response pairs - #93

Merged
frankier merged 17 commits into
mainfrom
codex/grid-preallocation-scratch
Nov 10, 2026
Merged

frankier merged 17 commits into
mainfrom
codex/grid-preallocation-scratch

Conversation

@frankier

@frankier frankier commented Nov 7, 2026 •

Copy link
Copy Markdown
Member

Fixed-grid MEPV prepares the history density and point estimate once per scan, reuses private candidate buffers, and delegates centered moments to PBB. Supported binary linear scans share FIB's stable response pairs; binary log scans share native log-response pairs. Range coordinates are materialized once per scan so range indexing does not obstruct response-loop vectorization.

The workspace uses the registered releases PsychometricsBazaarBase 0.8.11 (including PBB PRs #21 and #22) and FittedItemBanks 0.7.7. The Julia 1.12 CI job checks minimum supported versions.

Prepared contexts incorporate speculative response deltas and belong to one history/scan. The binary linear moment call uses skip_checks=true because preparation owns equally indexed, disjoint buffers and supported banks return two response factors. PBB retains mass and denominator checks. Its merged channel-pair reductions require no additional CAT call-site changes.

The prepared path currently requires scalar Float64 grids, mean ability estimation and a multiplicative density model. Supported linear banks are FIB transfer/CDF-MIRT banks and guess/slip wrappers. Custom banks retain scalar evaluation; unsupported configurations retain generic speculation. Log variances preserve absolute-denominator semantics, including extreme scales.

Validation

The rerun of CI on the current CAT head passes Julia 1.12 and 1.13 tests, coverage, and documentation against merged PBB main. The fresh benchmark compares every candidate score between linear/log Float64 and the Float32 probe up to 100,000 items, and checks the generic from-scratch reference at 1,000 items. All selections match the historical fixture: 21 / 809 / 9424 / 30483. Maximum relative criterion error is 1.82e-15 for log Float64 and 3.78e-7 for the Float32 probe.

Fresh Julia 1.13.3 benchmarks — merged PBB main

Median milliseconds per complete next_item call, except the explicitly labelled Float32 prepared-scan probe:

Items CAT linear Float64 CAT Float32 probe CAT log Float64 SuperFastCAT Float32
32 0.01287 0.01321 0.17036 0.01405
1,000 0.51033 0.32005 7.50172 0.42726
10,000 4.87849 2.80023 70.52945 4.05184
100,000 49.10860 27.72840 725.47476 41.23065

Original 10^8-item RNG prefix, the same first ten responses, Normal(0,1) prior and 33 points on [-4,4]. One thread pinned to CPU 0; 51 randomized interleaved warmed rounds through 10,000 items, 31 at 100,000. All timed compilation/recompilation and GC are zero. At 100,000 the median within-round CAT-linear/SFC ratio is 1.186 and Float32-probe/SFC is 0.674.

Float32 is not a shipped end-to-end path in this PR. A Float32 grid currently takes the generic fallback and fails on SqDev{Float64}(Float32). The Float32 probe explicitly constructs a prepared state, converts the existing Float64 history cache on every scan, and uses a local precision-preserving guess/slip adapter: FIB 0.7.7's 1.0 scale otherwise promotes those responses to Float64. No package methods were replaced. Leaving that FIB promotion intact gives a mixed-precision prepared probe of 34.455 ms at 100,000 items. These probe results show potential, not production Float32 performance.

LLVM shows four-wide Float64 response production and eight-wide Float32 production in the probe. The log-density producer remains scalar. The benchmark still uses the current generic FIB normalization, with no unit-scale specialization.

Driver, raw timings, type/path diagnostics, correctness checks, LLVM and immutable environment provenance are retained in CATExperiments under bench/merged_paths_20261109/ and data/analysis/20261109-merged-paths/.

The pull_request_target benchmark workflow still reads main's Julia 1.12 workflow until merge; that automatic run was cancelled. All direct measurements above use Julia 1.13.3.

@github-actions

github-actions Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

Benchmark Results

main e89273c... main / e89273c...
next_item_rules/4pl/mean_point_mepv_10 1.24 ± 0.079 ms 1.21 ± 0.07 ms 1.02 ± 0.088
next_item_rules/4pl/mean_point_mepv_bare 0.586 ± 0.023 ms 0.516 ± 0.0078 ms 1.14 ± 0.047
next_item_rules/4pl/mode_point_mepv_10 25.6 ± 1.2 ms 25.6 ± 1.1 ms 1 ± 0.065
next_item_rules/4pl/mode_point_mepv_bare 7.49 ± 1.5 ms 7.23 ± 1.8 ms 1.04 ± 0.33
time_to_load 1.6 ± 0.029 s 1.68 ± 0.021 s 0.952 ± 0.021

Benchmark Plots

A plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR.
Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).

@coveralls

coveralls commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Coverage Report for CI Build 42667679524

Coverage increased (+0.8%) to 55.022%

Details

  • Coverage increased (+0.8%) from the base build.
  • Patch coverage: 88 of 88 lines across 7 files are fully covered (100%).
  • 8 coverage regressions across 2 files.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

8 previously-covered lines in 2 files lost coverage.

File Lines Losing Coverage Coverage
src/Aggregators/speculators.jl 6 78.0%
src/Aggregators/tracked.jl 2 82.61%

Coverage Stats

Coverage Status
Relevant Lines: 2459
Covered Lines: 1353
Line Coverage: 55.02%
Coverage Strength: 433618.88 hits per line

💛 - Coveralls

@frankier
frankier force-pushed the codex/grid-preallocation-scratch branch from 66bfb13 to 51781f2 Compare November 8, 2026 17:28
@frankier
frankier marked this pull request as ready for review November 8, 2026 17:48
@frankier
frankier marked this pull request as draft November 9, 2026 10:54
@frankier
frankier marked this pull request as ready for review November 9, 2026 17:26
@frankier
frankier merged commit 5a106e2 into main Nov 10, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants