Skip to content

Deduplicate concurrent manifest fetches#19

Merged
Shane98c merged 2 commits intomainfrom
Shane98c/manifest-single-flight
Apr 27, 2026
Merged

Deduplicate concurrent manifest fetches#19
Shane98c merged 2 commits intomainfrom
Shane98c/manifest-single-flight

Conversation

@Shane98c
Copy link
Copy Markdown
Collaborator

This fixes bursts of duplicate manifest requests I was seeing in zarr-layer in some virtual datasets.

  • Add a generic single-flight cache helper for concurrent cache-aside loads.
  • Use it for manifest reads so parallel chunk lookups share one in-flight manifest fetch per manifest ID.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a singleFlight utility to coalesce concurrent cache misses for the same key into a single fetch operation, specifically for manifest loading in ReadSession. This optimization reduces redundant network requests during fan-out reads. Feedback suggests improving compatibility for environments without global DOMException and refining promise initialization to avoid type-casting hacks.

Comment thread src/cache/single-flight.ts
Comment thread src/cache/single-flight.ts Outdated
@Shane98c Shane98c merged commit 95a2f11 into main Apr 27, 2026
12 checks passed
@Shane98c Shane98c deleted the Shane98c/manifest-single-flight branch April 27, 2026 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant