Parameter limit#7803
Draft
dralley wants to merge 2 commits into
Draft
Conversation
Assisted By: Claude Code
PostgreSQL's wire protocol limits bind parameters to 65,535 per statement. When Django ORM's filter(field__in=python_list) generates WHERE field IN ($1, $2, ..., $65536+), it exceeds this limit when using server-side cursors (.iterator()). This introduces a safe_in() utility that uses a custom Django lookup (= ANY(%s)) for large lists, passing the entire list as a single PostgreSQL array parameter regardless of size. For small lists, the standard __in lookup is used unchanged. Applied safe_in() to all vulnerable code paths in pulpcore: - RepositoryVersion.get_content(), added(), removed() - import_repository_version() content mapping Also updated the test to use .iterator() so it reliably exercises the server-side cursor path that triggers the parameter limit. Assisted-By: claude-opus-4.6
4 tasks
Contributor
Author
|
Follow-up to #6784 This is probably a candidate for using the "run plugin CI against pulpcore main branch" automation. |
mdellweg
reviewed
Jun 17, 2026
| values = list(values) | ||
| if len(values) < POSTGRES_MAX_QUERY_PARAMS: | ||
| return Q(**{f"{field_name}__in": values}) | ||
| return Q(**{f"{field_name}__any_array": values}) |
Member
There was a problem hiding this comment.
Would there be a downside when we always used this array method?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📜 Checklist
See: Pull Request Walkthrough