[None][test] Waive TestGemma4MoE::test_bf16 (missing bf16 MMMU reference)

[dongfengy]

Summary by CodeRabbit

• Tests
  • Added a temporary waiver for a multi-GPU accuracy test that cannot run until the required reference specifications are available.

Description

accuracy/test_llm_api_autodeploy.py::TestGemma4MoE::test_bf16 evaluates MMMU with the plain bf16 spec, but references/mmmu.yaml only registers a quant_algo: NVFP4 / kv_cache_quant_algo: FP8 entry for google/gemma-4-26B-A4B-it (added in #16108). The accuracy harness therefore raises before scoring:

FAILED accuracy/test_llm_api_autodeploy.py::TestGemma4MoE::test_bf16 - ValueError: Not registered specs: {'dtype': 'auto', 'quant_algo': None, 'kv_cache_quant_algo': None, 'spec_dec_algo': None, 'extra_acc_spec': None}

The failure is deterministic and PR-independent: it failed identically in every recent LLM/main/L0_Test-x86_64-Multi-GPU run (builds 2327–2336), currently blocking pre-merge CI for any PR that triggers the multi-GPU stage (see also the report on #16108).

This PR waives the test until the bf16 MMMU reference is added (or the test is routed to a registered spec). cc @2ez4bz

Test Coverage

Waive-list-only change; no runtime code affected. mergeWaiveList picks this up for all in-flight PRs once merged.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

🤖 Generated with Claude Code

[dongfengy]

/bot run --disable-fail-fast

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d5ae9fcb-0d3b-48dc-baad-149edec37b53

📥 Commits

Reviewing files that changed from the base of the PR and between 5e77882 and cda65cd.

📒 Files selected for processing (1)

• tests/integration/test_lists/waives.txt

---

📝 Walkthrough

Walkthrough

Adds one scoped skip waiver for the Gemma4MoE BF16 integration test on the DGX H100 four-GPU AutoDeploy configuration due to missing MMMU reference specifications.

Changes

Integration test waiver

Layer / File(s)	Summary
Add scoped test waiver tests/integration/test_lists/waives.txt	Adds a temporary skip entry for TestGemma4MoE::test_bf16 on DGX_H100-4_GPUs-AutoDeploy-1.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Possibly related issues

• NVIDIA/TensorRT-LLM issue 16275 — Tracks the same Gemma4MoE BF16 failure on the DGX H100 four-GPU AutoDeploy stage.

Possibly related PRs

• NVIDIA/TensorRT-LLM#6795 — Updates the same integration waiver list with a different test waiver.
• NVIDIA/TensorRT-LLM#15195 — Adds temporary skip entries to the same waiver list.

Suggested reviewers: emmaqiaoch, yuxianq

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly states the waiver of TestGemma4MoE::test_bf16 and the missing bf16 MMMU reference.
Description check	✅ Passed	The description includes the issue, rationale, and test coverage, and it matches the required template sections.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[tensorrt-cicd]

PR_Github #58837 [ run ] triggered by Bot. Commit: cda65cd Link to invocation

[tensorrt-cicd]

PR_Github #58837 [ run ] completed with state SUCCESS. Commit: cda65cd
/LLM/main/L0_MergeRequest_PR pipeline #47390 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[dongfengy]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #58839 [ run ] triggered by Bot. Commit: a5daf1b Link to invocation

[tensorrt-cicd]

PR_Github #58839 [ run ] completed with state SUCCESS. Commit: a5daf1b
/LLM/main/L0_MergeRequest_PR pipeline #47392 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[dongfengy]

/bot reuse-pipeline

[tensorrt-cicd]

PR_Github #58840 [ reuse-pipeline ] triggered by Bot. Commit: a5daf1b Link to invocation

[tensorrt-cicd]

PR_Github #58840 [ reuse-pipeline ] completed with state SUCCESS. Commit: a5daf1b
Can't reuse PR_Github #58839 with status: FAILED

Link to invocation

[dongfengy]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #58863 [ run ] triggered by Bot. Commit: b418e58 Link to invocation

[tensorrt-cicd]

PR_Github #58863 [ run ] completed with state SUCCESS. Commit: b418e58
/LLM/main/L0_MergeRequest_PR pipeline #47411 completed with status: 'SUCCESS'

CI Report

Link to invocation

[dongfengy]

Superseded: TestGemma4MoE::test_bf16 has been waived on main via #16283 (all platforms, https://nvbugs/6445322), which mergeWaiveList picks up for all PR runs. This PR is no longer needed.