Repository navigation
Conversation
|
Hi @tlopex, could you take a look at these Relax/ONNX normalization correctness fixes when you have a chance? Please let me know if any further changes are needed. Thanks! |
|
Could one of you take a look at these Relax/ONNX normalization correctness fixes when you have a chance, or point me to the right reviewer? I am keen to help get these fixes into TVM and happy to address feedback, add further regression tests, rebase, or split the changes if that would make review easier. The GitHub Actions checks also seem to need maintainer help with fork-workflow approval or a restart. I would appreciate any guidance on getting the appropriate checks running. Thank you for your time! |
|
Hi @mshr-h @tlopex, could you review these Relax/ONNX normalization fixes when convenient? They cover BatchNormalization momentum, LayerNormalization statistics outputs, and negative BatchNorm axes, with focused regressions. The five currently reported CI contexts are successful. Please let me know if you would prefer these fixes split for review. Thanks! |
This fixes three normalization correctness issues at their respective contract boundaries:
BatchNormalization.momentumto Relax's update-coefficient convention;LayerNormalizationMean and InvStdDev outputs instead of scalar placeholders;DecomposeOpsForTrainingconstructs reduction axes.The LayerNormalization statistics follow ONNX
stash_type=1and are computed in float32. The BatchNorm momentum conversion is kept in the ONNX frontend so existing Relax/TOPI semantics remain unchanged.Closes #20496
Closes #20503
Closes #20504
Tests:
pytest tests/python/relax/test_frontend_onnx.py -k "layer_norm or batch_norm" -q(8 passed)pytest tests/python/relax/test_transform_decompose_ops.py::test_batch_norm_training_negative_axis -q(1 passed)