Skip to content

fix(gemma4): diagnose malformed per-module quant overrides#695

Merged
inureyes merged 1 commit into
mainfrom
fix/691-gemma4-quant-override-diagnostics
Jul 8, 2026
Merged

fix(gemma4): diagnose malformed per-module quant overrides#695
inureyes merged 1 commit into
mainfrom
fix/691-gemma4-quant-override-diagnostics

Conversation

@inureyes

@inureyes inureyes commented Jul 8, 2026

Copy link
Copy Markdown
Member

Summary

PR #690 added Gemma 4 per-module MLP quantization overrides through a flattened overrides map on QuantizationArgs, but a malformed override value (a non-object value, a wrong-typed/negative/zero group_size, or an unsupported bits width) silently fell back to the root defaults with no indication anything was wrong, which is a debugging hazard for quantization frontier work.

What changed

  • QuantizationArgs::quant_params_for in src/models/gemma4.rs now validates the override entry matched by the queried module prefix before applying it, reporting an actionable tracing::warn! naming the offending key for a non-object override value, a wrong-typed or negative group_size/bits, a zero group size, or an unsupported bit width (mlxcel only supports the affine {2,3,4,5,6,8} set), then falls back to the root defaults for that field exactly as before.
  • Added two small validation helpers, validate_override_group_size and validate_override_bits, that produce the actionable reason strings used in the warnings.
  • Metadata keys that legitimately live alongside per-module overrides in the same flattened map (mode, quant_method, quant_algo, config_groups) are untouched because they never coincide with a real module-prefix lookup, so they are never mistaken for malformed overrides.
  • Added a quant_override_diagnostics_tests module in src/models/gemma4.rs covering valid full and partial overrides, absent overrides, a non-object override value, string-typed and negative/zero/unsupported-width numeric fields, sibling metadata keys, and a typo'd prefix that simply never matches.

Test plan

  • cargo check -p mlxcel --features cuda --lib
  • cargo test -p mlxcel --features cuda --lib models::gemma4::quant (24 tests, all pass, including the 11 new diagnostics tests and the pre-existing quant_scheme_tests)
  • cargo test -p mlxcel --features cuda --lib models::gemma4_tests (9 pass, 8 ignored for serial MLX execution as before, including the existing gemma4_root_quantization_preserves_per_module_overrides regression test, confirming valid configs load unchanged)
  • cargo clippy -p mlxcel --features cuda --lib --tests (no warnings touching gemma4.rs; the crate has one pre-existing unrelated redundant_closure warning in src/models/sanitize.rs that predates this change)
  • cargo fmt -p mlxcel -- --check

Closes #691

PR #690 added Gemma 4 per-module MLP quantization overrides via a flattened `overrides` map on `QuantizationArgs`, but a malformed override value (a non-object value, a wrong-typed/negative/zero `group_size`, or an unsupported `bits` width) silently fell back to the root defaults with no indication anything was wrong, which is a debugging hazard for quantization frontier work (issue #691).

`QuantizationArgs::quant_params_for` now validates the entry matched by the queried module `prefix` before applying it: a non-object value, or a `group_size`/`bits` field with the wrong type, a negative value, a zero group size, or an unsupported bit width (mlxcel only supports the affine `{2,3,4,5,6,8}` set) is reported via `tracing::warn!` naming the offending key, then the loader falls back to the root defaults for that field exactly as before. Metadata keys that legitimately live alongside per-module overrides in the same flattened map, such as `mode`, `quant_method`, `quant_algo`, and `config_groups`, are untouched because they never coincide with a real module-prefix lookup.

Added a `quant_override_diagnostics_tests` module covering valid full and partial overrides, absent overrides, a non-object override value, string-typed and negative/zero/unsupported-width numeric fields, sibling metadata keys, and a typo'd prefix that simply never matches, verifying valid configs keep loading unchanged while malformed ones are diagnosed instead of silently mis-routed.
@inureyes inureyes added type:bug Bug fixes, error corrections, or issue resolutions priority:low Low priority area:models Model architectures, weights, loading, metadata labels Jul 8, 2026
@inureyes

inureyes commented Jul 8, 2026

Copy link
Copy Markdown
Member Author

PR Finalization Complete

Summary

No source changes were needed; working tree is unchanged from the prior commit. Ready for merge.

@inureyes inureyes added the status:done Completed label Jul 8, 2026
@inureyes inureyes merged commit cc19322 into main Jul 8, 2026
5 checks passed
@inureyes inureyes deleted the fix/691-gemma4-quant-override-diagnostics branch July 8, 2026 12:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:models Model architectures, weights, loading, metadata priority:low Low priority status:done Completed type:bug Bug fixes, error corrections, or issue resolutions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(gemma4): diagnose malformed per-module quant overrides

1 participant