Skip to content

Make _parse_imro_string self-contained by deriving IMRO field schemas from the catalogue#417

Open
h-mayorquin wants to merge 1 commit intoSpikeInterface:mainfrom
h-mayorquin:single_source_ot_truth_parse_imro
Open

Make _parse_imro_string self-contained by deriving IMRO field schemas from the catalogue#417
h-mayorquin wants to merge 1 commit intoSpikeInterface:mainfrom
h-mayorquin:single_source_ot_truth_parse_imro

Conversation

@h-mayorquin
Copy link
Collaborator

This is another PR in the series of Neuropixels cleanups (see #410, #405) with the goal of making everything come from a single source of data: the Neuropixels probe library from ProbeTable.

The following victim is _parse_imro_string, which required callers to pass a probe_part_number just so it could look up the IMRO field schema, even though the IMRO string itself contains the format type code needed to determine the schema. This forced us to maintain a hand-written dict (probe_part_number_to_probe_type) that duplicated information already present in the ProbeTable catalogue, and read_imro had to pre-parse the header and reverse-lookup through it. The dict had to be manually updated whenever a new probe was added, and it had already drifted from the catalogue (missing types like 1120, 1122, 1123, 1200, 3010, 3020).

I added a post-processing script (resources/postprocess_neuropixels_probe_features.py) that derives two mappings from the catalogue JSON after each weekly ProbeTable sync: z_imro_format_type_to_imro_format (format type code to field schema) and z_imro_format_type_to_part_number (format type code to canonical part number for geometry construction). _parse_imro_string now takes just the IMRO string, extracts the format type from the header, and looks up the schema directly. read_imro uses the part number mapping instead of the hand-maintained dict.

Something to discuss: while working on this, I noticed that some format type codes in the old dict (NHP types 1015, 1021, 1022, 1031, 1032 and NP2.x variants 2004, 2014) are not listed in any ProbeTable val_def entry. My recollection is that the old dict had them because it was originally designed to map part numbers to types (where one entry per part number made sense) and was later repurposed in reverse to resolve types back to part numbers. I have left them in a small fallback dict for backwards compatibility, but I think we should remove them after a release or two if nobody reports a breakage. What do you think?

@h-mayorquin h-mayorquin marked this pull request as ready for review March 24, 2026 23:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant