Optimize sync for repositories with specific tag lists

[parmstro]

Add bypass optimization to skip expensive /tags/list pagination when syncing container repositories with specific (non-wildcard) tag lists.

When include_tags contains only specific references without wildcards, and exclude_tags is empty or contains only harmless patterns that won't match the includes (e.g., *-source), the sync bypasses /tags/list enumeration entirely and fetches manifests directly.

Cosign companion tags (signatures, attestations, SBOMs) are discovered via concurrent HEAD request probing instead of full tag enumeration, maintaining security artifact discovery while avoiding expensive pagination through deep tag lists.

Performance impact: Reduces sync time from 3-8 minutes to ~20 seconds for repositories with 50,000+ tags when syncing specific digest references.

Changes:

• Add auto_discover_cosign field to ContainerRemote model (default True)
• Implement _can_bypass_taglist() detection in ContainerFirstStage
• Add _discover_cosign_companions_without_taglist() for HEAD probing
• Add _tag_exists() helper for tag existence validation
• Pass mirror parameter through synchronize() to ContainerFirstStage
• Add database migration for auto_discover_cosign field
• Add comprehensive unit test coverage

closes #2474
Assisted by: Claude Sonnet 4.5 (model ID: claude-sonnet-4-5@20250929)

[gerrod3]

Thanks for the contribution, but we won't accept this as is.

We do not allow adding migrations in older branches. It prevents upgrading to a newer version.

From what I read it seems the slow part is the tag pagination fetching, did you try increasing the number of results for each tag request with ?n=. https://distribution.github.io/distribution/spec/api/#listing-image-tags I think if we provide a value, say 1000, then we could drastically reduce the time doing fetches.

[gerrod3]

@parmstro Not sure if your bot is going crazy, pushing to the wrong repo, but I'm going to close this now.

[parmstro]

Sorry Gerrod. Yes close it, it was my bad command when the system had upstream pointed at the pulp repo instead of my fork. Many apologies. It won't happen again. I am going to provide a enhancement for 2.29 as soon as I can get my upstream dev environment configured. I will ensure that it is not posted as a PR until thoroughly tested on my end and all the ansible stuff will stay in the development branch on my system. Eventually the ansible plays that magically generate the OCP products and repositories in Foreman/Satellite will be available from my github. Perhaps they can be incorporated as sample playbooks in foreman-ansible-modules. Sorry again. (Yes, I am Canadian and we apologize a lot. Sorry. .... See!) Cheers, Paul

…

On Thu, Sep 10, 2026 at 5:53 PM Gerrod ***@***.***> wrote: *gerrod3* left a comment (pulp/pulp_container#2475) <#2475 (comment)> @parmstro <https://github.com/parmstro> Not sure if your bot is going crazy, pushing to the wrong repo, but I'm going to close this now. — Reply to this email directly, view it on GitHub <#2475?email_source=notifications&email_token=ABMAI2LLMK2D24UXGXQ2RS35OMPENA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKNRSGU4TGNBYGEZKM4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#issuecomment-5625934812>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABMAI2O2NKSRP7UWHYBGVB35OMPENAVCNFSNUABFKJSXA33TNF2G64TZHMZDCMZZGYYTCMRWHNEXG43VMU5TKMRRGY4DMMRUGIYKC5QC> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/ABMAI2JW2P6W3OTWAMXAEMD5OMPENA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKNRSGU4TGNBYGEZKM4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJKTGN5XXIZLSL5UW64Y> and Android <https://github.com/notifications/mobile/android/ABMAI2KREFNCWDFULWNFS5L5OMPENA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKNRSGU4TGNBYGEZKM4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLTGN5XXIZLSL5QW4ZDSN5UWI>. Download it today! You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Paul Armstrong Senior Principal Technical Specialist Red Hat 647-285-0044 Get Red Hat Developer for Individuals <https://developers.redhat.com/register> Sign up with a personal email to get free RHEL for personal use. Use and extend Granite open source AI models, develop software and learn new skills with no cost, self-support subscriptions.