fix: populate() with reserve_jobs=True ignores restrictions (#1413)#1414
Open
dimitri-yatsenko wants to merge 2 commits intomasterfrom
Open
fix: populate() with reserve_jobs=True ignores restrictions (#1413)#1414dimitri-yatsenko wants to merge 2 commits intomasterfrom
dimitri-yatsenko wants to merge 2 commits intomasterfrom
Conversation
Fixes #1413 — populate() with reserve_jobs=True was ignoring restrictions passed by the caller. _populate_distributed() refreshed the job queue with the correct restrictions but then fetched all pending keys without filtering them. The fix intersects the pending query with _jobs_to_do(restrictions), matching the behaviour of _populate_direct(). Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
…ctions Adds test_populate_reserve_jobs_respects_restrictions which verifies that _populate_distributed() honours the caller's restriction when reserve_jobs=True. The test seeds a full job queue for all subjects, then calls populate(restriction, reserve_jobs=True, refresh=False) for a single subject and asserts that only that subject's rows were created. Also fixes the restrict() call in _populate_distributed to use semantic_check=False, matching the pattern in jobs.py refresh(), because the jobs table PK has different attribute lineage than key_source. Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1413.
_populate_distributed()was passing*restrictionstoself.jobs.refresh()but then fetching all pending jobs without applying those restrictions. This meantpopulate(restriction, reserve_jobs=True)processed every pending key instead of only those matching the restriction.Root cause: The pending key query at line 495 used
self.jobs.pendingunfiltered:Fix: Intersect with
_jobs_to_do(restrictions), reusing the existing method that correctly applies restrictions (identical to_populate_direct()behaviour):Test plan
populate(restriction, reserve_jobs=True)processes only keys matching the restrictionpopulate(reserve_jobs=True)(no restriction) is unchangedpixi run test -k autopopulate🤖 Generated with Claude Code