Skip to content

Destroy nvexec continues_on storage on CUDA errors - #2190

Open
fallintoplace wants to merge 1 commit into
NVIDIA:mainfrom
fallintoplace:fix/nvexec-continues-on-error-cleanup
Open

Destroy nvexec continues_on storage on CUDA errors#2190
fallintoplace wants to merge 1 commit into
NVIDIA:mainfrom
fallintoplace:fix/nvexec-continues-on-error-cleanup

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Summary

  • destroy host-constructed completion storage before propagating CUDA setup errors
  • keep the existing deferred destruction on successful paths
  • add a regression using pinned host memory to force cudaMemPrefetchAsync to fail

Why

continues_on constructs non-trivial completion storage on the host before several CUDA calls. An early error returned without running the active completion value destructor, and the operation state only released the raw allocation.

Cleanup now happens before the downstream completion operation because that operation may destroy the operation state.

@copy-pr-bot

copy-pr-bot Bot commented Aug 11, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@fallintoplace
fallintoplace marked this pull request as draft August 11, 2026 16:50
@fallintoplace
fallintoplace force-pushed the fix/nvexec-continues-on-error-cleanup branch from b4d280f to bb13404 Compare August 11, 2026 16:57
@ericniebler

Copy link
Copy Markdown
Collaborator

why is this a draft PR?

@ericniebler

Copy link
Copy Markdown
Collaborator

/ok to test bb13404

@fallintoplace
fallintoplace marked this pull request as ready for review August 11, 2026 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants