Skip to content

[https://nvbugs/6428144][test] Unwaive GB300 DeepSeek-R1 gen-only disagg PerfSanity#16267

Open
chienchunhung wants to merge 1 commit into
NVIDIA:mainfrom
chienchunhung:codex/verify-nvbug-6438586-ci
Open

[https://nvbugs/6428144][test] Unwaive GB300 DeepSeek-R1 gen-only disagg PerfSanity#16267
chienchunhung wants to merge 1 commit into
NVIDIA:mainfrom
chienchunhung:codex/verify-nvbug-6438586-ci

Conversation

@chienchunhung

@chienchunhung chienchunhung commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

Remove the waiver filed under NVBug 6428144 and restore post-merge CI coverage for:

perf/test_perf_sanity.py::test_e2e[disagg_upload-gen-only-gb300_deepseek-r1-fp4_128k8k_con256_ctx1_pp4_gen1_dep8_eplb0_mtp1_ccb-NIXL]

Rationale

PR #15737 normalized the gathered per-rank status before the gen-only disaggregated benchmark fill-gate reduction, addressing the list/int failure reported by NVBug 6438586. This waived workload exercises that gate and now passes on the fixed code.

NVBug 6428144 remains the tracker attached to this test's waiver; NVBug 6438586 and PR #15737 provide the related product-fix context.

Validation

Tested commit: 907c581354f5dc394a01fb5e8739270a281570be

  • Targeted L0 pipeline #47307, launched by PR_Github #58724: the exact GB300 test was selected and passed (1 passed; 3,582.82-second call; pytest exit code 0).
  • Full L0 pipeline #47430, launched by PR_Github #58888: passed on the same commit. The pipeline reused both successful targeted stages from #47307 and separately ran the CBTS-selected coverage (15 passed, 0 failed, 0 skipped).
  • python3 scripts/check_test_list.py --validate --check-duplicate-waives — 2,209 unique entries validated.
  • git diff --check

Signed-off-by: Chien-Chun Hung <2679986+chienchunhung@users.noreply.github.com>

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast --stage-list "DGX_B200-16_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE1-GPU8-Post-Merge-3,GB300-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-3"

@chienchunhung chienchunhung marked this pull request as ready for review July 11, 2026 01:02
@chienchunhung chienchunhung changed the title [https://nvbugs/6438586][test] verify disagg benchmark fill gate [https://nvbugs/6438586][test] verify disagg benchmark fill gate (DO NOT REVIEW / MERGE) Jul 11, 2026
@coderabbitai

coderabbitai Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Removes one waiver entry from the integration test waiver list for the disagg_upload-gen_only-gb300_deepseek-r1-fp4 performance test configuration. No public entities or other waiver entries change.

Changes

Performance waiver update

Layer / File(s) Summary
Remove targeted performance waiver
tests/integration/test_lists/waives.txt
Deletes the perf/test_perf_sanity.py::test_e2e waiver for the specified GB300 DeepSeek configuration.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Possibly related PRs

Suggested reviewers: mzweilz, dpitman-nvda

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title is concise and accurately reflects removing the GB300 DeepSeek-R1 perf sanity waiver.
Description check ✅ Passed The description explains the change, rationale, and validation clearly, with only the checklist section missing.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #58722 [ run ] triggered by Bot. Commit: 907c581 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #58723 [ run ] triggered by Bot. Commit: 907c581 Link to invocation

@chienchunhung chienchunhung marked this pull request as draft July 11, 2026 01:07
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #58724 [ run ] triggered by Bot. Commit: 907c581 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #58722 [ run ] completed with state ABORTED. Commit: 907c581

Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #58723 [ run ] completed with state ABORTED. Commit: 907c581

Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #58724 [ run ] completed with state SUCCESS. Commit: 907c581
/LLM/main/L0_MergeRequest_PR pipeline #47307 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

@chienchunhung

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #58888 [ run ] triggered by Bot. Commit: 907c581 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #58888 [ run ] completed with state SUCCESS. Commit: 907c581
/LLM/main/L0_MergeRequest_PR pipeline #47430 completed with status: 'SUCCESS'

CI Report

Link to invocation

@chienchunhung

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #59006 [ run ] triggered by Bot. Commit: 907c581 Link to invocation

@chienchunhung chienchunhung changed the title [https://nvbugs/6438586][test] verify disagg benchmark fill gate (DO NOT REVIEW / MERGE) [https://nvbugs/6428144][test] Unwaive GB300 DeepSeek-R1 gen-only disagg PerfSanity Jul 13, 2026
@chienchunhung chienchunhung marked this pull request as ready for review July 13, 2026 17:32
@chienchunhung chienchunhung enabled auto-merge (squash) July 13, 2026 17:35
@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #59006 [ run ] completed with state SUCCESS. Commit: 907c581
/LLM/main/L0_MergeRequest_PR pipeline #47535 completed with status: 'SUCCESS'

CI Report

Link to invocation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants