[None][fix] handle empty stream in process_streaming_events

[lonexreb]

Description

process_streaming_events (tensorrt_llm/serve/responses_utils.py, the /v1/responses streaming path) binds final_res only inside the stream loop:

async for res in generator:
    final_res = res
    ...
final_response = await streaming_processor.get_final_response(final_res)  # UnboundLocalError if 0 iterations

If the generator yields nothing (empty or aborted stream), final_res is never bound and get_final_response(final_res) raises UnboundLocalError, escaping the async generator.

Change

Initialize final_res = None before the loop and return early when no output was produced (no final response to build).

Test

New CPU-only unit test in tests/unittest/llmapi/apps/test_responses_empty_stream.py: driving process_streaming_events with an empty async stream (processor mocked) completes without raising and does not call get_final_response.

Summary by CodeRabbit

• Bug Fixes

  • Improved handling of empty or aborted streaming responses.
  • Prevented errors when a stream produces no output events.
  • Ensured final response processing is skipped when no response is available.

• Tests

  • Added coverage for empty streaming scenarios to verify they complete cleanly without producing events or triggering unnecessary final-response processing.

[coderabbitai]

📝 Walkthrough

Walkthrough

process_streaming_events now safely handles async streams that produce no results by returning before final response construction. A unit test adds an empty stream fixture and verifies no events are emitted or final response is requested.

Changes

Empty stream handling

Layer / File(s)	Summary
Stream guard and validation tensorrt_llm/serve/responses_utils.py, tests/unittest/llmapi/apps/test_responses_empty_stream.py	process_streaming_events tracks the final result, yields chunk events, and returns for empty streams; the test verifies empty output and no final response call.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title follows the required ticket/type format and accurately summarizes the empty-stream fix.
Description check	✅ Passed	The PR explains the issue, change, and test coverage, though it omits the template's explicit checklist section.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (2)

tests/unittest/llmapi/apps/test_responses_empty_stream.py (2)

26-49: 📐 Maintainability & Code Quality | 🔵 Trivial

No QA list update is needed.

This change only adds a narrow unit test under tests/unittest/; no integration or release schedule is affected.

As per path instructions, unittest-only changes do not require an entry under tests/integration/test_lists/qa/.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unittest/llmapi/apps/test_responses_empty_stream.py` around lines 26 -
49, The comment confirms that no implementation change or QA list update is
required. Keep the narrow unit test
test_process_streaming_events_handles_empty_stream under tests/unittest/, and do
not add an entry under tests/integration/test_lists/qa/.

Source: Path instructions

---

21-28: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Annotate the async test helpers.

Add return annotations to _empty_stream and run to satisfy the Python typing guidelines.

Proposed fix

+from collections.abc import AsyncGenerator
+
-async def _empty_stream():
+async def _empty_stream() -> AsyncGenerator[None, None]:
     return
     yield  # pragma: no cover - makes this an async generator
...
-    async def run():
+    async def run() -> None:

As per coding guidelines, “Always annotate functions.”

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unittest/llmapi/apps/test_responses_empty_stream.py` around lines 21 -
28, Add return type annotations to the async helper functions _empty_stream and
the nested run inside test_process_streaming_events_handles_empty_stream, using
the appropriate async-generator and coroutine return types consistent with their
implementations.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/unittest/llmapi/apps/test_responses_empty_stream.py`:
- Around line 26-49: The comment confirms that no implementation change or QA
list update is required. Keep the narrow unit test
test_process_streaming_events_handles_empty_stream under tests/unittest/, and do
not add an entry under tests/integration/test_lists/qa/.
- Around line 21-28: Add return type annotations to the async helper functions
_empty_stream and the nested run inside
test_process_streaming_events_handles_empty_stream, using the appropriate
async-generator and coroutine return types consistent with their
implementations.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 643f0d76-2cb7-487f-b0bd-990420148a9d

📥 Commits

Reviewing files that changed from the base of the PR and between b4e7808 and a980a70.

📒 Files selected for processing (2)

• tensorrt_llm/serve/responses_utils.py
• tests/unittest/llmapi/apps/test_responses_empty_stream.py