[aditya0by0]

Also merge: sfluegel05/HiMol#3

[aditya0by0]

How Missing Labels Are Handled in HiMol Finetuning

HiMol uses a three-valued label encoding for classification: +1 (positive/active), -1 (negative/inactive), and 0 (missing/unmeasured). The 0 sentinel is established during data loading in finetune/loader.py — DeepChem weights of 0 and NaN values are both mapped to 0, while real 0/1 labels are converted to -1/+1.

Loss Computation

In finetune/finetune.py (train() and eval() functions), the key mechanism is:

is_valid = y**2 > 0                          # 0²=0 → invalid; (±1)²=1 → valid
loss_mat = criterion(pred.double(), (y+1)/2)  # {-1,+1} → {0,1} for BCE
loss_mat = torch.where(is_valid, loss_mat, torch.zeros(...))  # zero out missing
loss = torch.sum(loss_mat) / torch.sum(is_valid)              # normalize by valid count

Missing labels are zeroed out in the loss matrix and excluded from the normalization denominator (torch.sum(is_valid)), so they contribute nothing to the gradient. The same pattern appears in finetune/optimization.py.

Metrics Computation

• ROC-AUC (per-task): Only computed for tasks with both positive and negative examples. Within each task, the is_valid mask filters out missing entries before passing to roc_auc_score. Tasks that can't be computed are skipped, and a "Some target is missing!" warning with the missing ratio is printed.

• Macro F1 (finetune/metrics.py): Classes with zero positive labels are excluded from the macro average via a mask. Classes with positive labels but no positive predictions get NaN precision, which is converted to 0.

• Micro F1: Uses torchmetrics' MultilabelF1Score (includes all labels in averaging).

• Regression: No missing label handling — MSE/MAE/RMSE computed directly on all predictions.

Note: The eval() function has a bug where loss is computed using only the last batch's batch.y rather than the full concatenated dataset. The eval_chebi() function fixes this by using the full y_true array.

[aditya0by0]

How Missing Labels Are Handled in HiMol Finetuning

HiMol uses a three-valued label encoding for classification: +1 (positive/active), -1 (negative/inactive), and 0 (missing/unmeasured). The 0 sentinel is established during data loading in finetune/loader.py — DeepChem weights of 0 and NaN values are both mapped to 0, while real 0/1 labels are converted to -1/+1.

Loss Computation

In finetune/finetune.py (train() and eval() functions), the key mechanism is:

is_valid = y**2 > 0                          # 0²=0 → invalid; (±1)²=1 → valid
loss_mat = criterion(pred.double(), (y+1)/2)  # {-1,+1} → {0,1} for BCE
loss_mat = torch.where(is_valid, loss_mat, torch.zeros(...))  # zero out missing
loss = torch.sum(loss_mat) / torch.sum(is_valid)              # normalize by valid count

Missing labels are zeroed out in the loss matrix and excluded from the normalization denominator (torch.sum(is_valid)), so they contribute nothing to the gradient. The same pattern appears in finetune/optimization.py.

Metrics Computation

• ROC-AUC (per-task): Only computed for tasks with both positive and negative examples. Within each task, the is_valid mask filters out missing entries before passing to roc_auc_score. Tasks that can't be computed are skipped, and a "Some target is missing!" warning with the missing ratio is printed.
• Macro F1 (finetune/metrics.py): Classes with zero positive labels are excluded from the macro average via a mask. Classes with positive labels but no positive predictions get NaN precision, which is converted to 0.
• Micro F1: Uses torchmetrics' MultilabelF1Score (includes all labels in averaging).
• Regression: No missing label handling — MSE/MAE/RMSE computed directly on all predictions.

Note: The eval() function has a bug where loss is computed using only the last batch's batch.y rather than the full concatenated dataset. The eval_chebi() function fixes this by using the full y_true array.

fixed in ChEB-AI/python-chebai#174

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.