Skip to content

Add PTB-XL dataset support (ECG) - #1187

Open
AxelNoun wants to merge 4 commits into
sunlabuiuc:masterfrom
AxelNoun:feat/ptbxl-dataset
Open

Add PTB-XL dataset support (ECG)#1187
AxelNoun wants to merge 4 commits into
sunlabuiuc:masterfrom
AxelNoun:feat/ptbxl-dataset

Conversation

@AxelNoun

@AxelNoun AxelNoun commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Contributor: AxelNoun external contributor, no NetID
Contribution Type: New Dataset + Task
Description: Adds PTB-XL (12-lead ECG, PhysioNet open access) as a
PyHealth dataset, with a 5-superclass multi-label diagnosis task and a
split helper using the official stratified folds. Motivated by the CPBench
effort: it gives conformal prediction a multi-label ECG benchmark that needs
no credentialing, with metadata suited to covariate/label-shift splits.
Based on Wagner et al., Scientific Data 2020
(https://www.nature.com/articles/s41597-020-0495-6).

Files to Review:

  • pyhealth/datasets/ptbxl.py dataset implementation
  • pyhealth/datasets/configs/ptbxl.yaml table config (template)
  • pyhealth/tasks/ptbxl.py 5-superclass multi-label task
  • pyhealth/datasets/splitter.py split_by_strat_fold helper
  • tests/core/test_ptbxl.py tests with synthetic fixtures
  • docs/api/datasets/pyhealth.datasets.ptbxl.rst, docs/api/tasks/pyhealth.tasks.ptbxl.rst
  • examples/ecg/ptbxl/ptbxl_superclass_quickstart.py

Implementation notes:

  • wfdb is an optional extra (pip install pyhealth[ptbxl]), lazily imported.
    Happy to make it a hard dependency if you'd prefer.
  • Derived metadata CSV goes to MODULE_CACHE_PATH, not root. Filenames include
    sampling rate and a short hash of the resolved data root so different roots
    never share a cache; reuse also validates signal_file ⊆ data_root.
  • Resolved cache YAML (new vs EEGBCI): BaseDataset only accepts
    config_path, so PTB-XL writes a root+rate-keyed YAML next to the derived CSV
    (ptbxl-config-{rate}hz-{root_hash}.yaml) before super().__init__(), with
    file_path already pointing at the derived metadata CSV. EEGBCI instead
    patches self.config.tables[...].file_path after super(). Open to aligning
    with maintainers' preferred pattern.
  • Unrelated pre-existing # noqa: E402 on the EEGBCI import line in
    pyhealth/datasets/__init__.py left untouched (out of diff).
  • Signals returned as (n_leads, n_samples), matching EEGBCI's convention.
  • Age ≥90 is encoded as 300 per HIPAA; kept raw with age_is_censored,
    distinct from age_is_missing (written as pd.NA).
  • Empty label sets (407/21799) dropped by default via drop_empty_labels.
  • Fixture under test-resources/ptbxl/ is synthetic WFDB (.hea/.dat)
    plus tiny CSVs covering censored age, missing age, multilabel, and empty superclass.

Testing / CI:

  • Local: pytest tests/core/test_ptbxl.py 17 passed (with wfdb installed).
  • Without wfdb / pyhealth[ptbxl], the 2 waveform I/O tests skip; metadata /
    task / e2e event-read tests still run.
  • pixi test feature pins wfdb so make testall under pixi covers waveform I/O;
    CI paths that do not install that extra will not exercise load_ptbxl_record.

Follow-up: 71-SCP-code classification and age regression are structured
for but not implemented here planned as a separate PR.

Test plan

  • pytest tests/core/test_ptbxl.py 17 passed (with wfdb)
  • python tools/check_pr_rules.py --base upstream/master --head HEAD
  • CI make testall / contribution-rules on the PR

Recreates #1186, which was closed automatically on 12 Aug when the fork
hosting its branch was detached during an account cleanup on my side
my mistake, apologies for the noise.

The head commit is byte-identical to the original PR (47aa3a41d7).
All prior context and discussion: #1186

AxelNoun and others added 4 commits August 12, 2026 21:23
Introduce PTBXLDataset (BaseDataset + YAML) with optional wfdb extra,
root-keyed metadata cache, and a resolved cache YAML so file_path is
correct before BaseDataset init.

Co-authored-by: Cursor <cursoragent@cursor.com>
Add PTBXLSuperclassClassification and split_by_strat_fold (folds 1-8 /
9 / 10) in separate modules so they can move to benchmarks later.

Co-authored-by: Cursor <cursoragent@cursor.com>
Cover helpers, metadata cache keyed by root, event reads via
PTBXLDataset, waveform IO (wfdb), and the 5-superclass task path.

Co-authored-by: Cursor <cursoragent@cursor.com>
Document the dataset/task Overview RST pages and a minimal ECG
superclass example for CPBench users.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant