Skip to content

hashindex: track unresolved pack location with F_PENDING flag#9828

Merged
ThomasWaldmann merged 3 commits into
borgbackup:masterfrom
mr-raj12:chunkindex-pending-flag
Jun 30, 2026
Merged

hashindex: track unresolved pack location with F_PENDING flag#9828
ThomasWaldmann merged 3 commits into
borgbackup:masterfrom
mr-raj12:chunkindex-pending-flag

Conversation

@mr-raj12

@mr-raj12 mr-raj12 commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Description

A chunk's pack location (pack_id, obj_offset, obj_size) isn't known yet between put() buffering it in the PackWriter and flush() writing the pack. The old code signalled this by writing the UNKNOWN_BYTES32 sentinel into pack_id and comparing against it later. Since every 256-bit value is a valid pack_id, reserving one as a marker is in-band signalling.

This replaces that with a ChunkIndex user flag, F_PENDING. add() sets it, update_pack_info() clears it once the real location is known, and is_pending() is the query used by repository.list() and get() in place of the old sentinel comparison. UNKNOWN_BYTES32 is still written as filler for the fixed-width record, but nothing reads it for meaning now.

Follow-up to #9821.

Checklist

  • PR is against master
  • New code has tests and docs where appropriate
  • Tests pass (run tox or the relevant test subset)
  • Commit messages are clean and reference related issues

Replace the UNKNOWN_BYTES32 pack_id sentinel (in-band signalling) with a
ChunkIndex system flag, queried via is_pending(). Follow-up to borgbackup#9821.
@mr-raj12 mr-raj12 marked this pull request as ready for review June 29, 2026 01:33
@mr-raj12 mr-raj12 marked this pull request as draft June 29, 2026 01:34
@codecov

codecov Bot commented Jun 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 85.14%. Comparing base (2dcb50f) to head (1206808).
⚠️ Report is 11 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/borg/repository.py 75.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #9828      +/-   ##
==========================================
- Coverage   85.23%   85.14%   -0.09%     
==========================================
  Files          92       93       +1     
  Lines       15297    15337      +40     
  Branches     2299     2306       +7     
==========================================
+ Hits        13038    13059      +21     
- Misses       1570     1586      +16     
- Partials      689      692       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Comment thread src/borg/repository.py Outdated
@ThomasWaldmann

ThomasWaldmann commented Jun 29, 2026

Copy link
Copy Markdown
Member

Not sure: maybe check whether it would be better to use a user flag for F_PENDING.

The slight complication you have in the hashindex code (can't use __setitem__) would go away and you could set/clear the flag from python code.

The "system flags" are rather hashtable-internal things, like tracking what's new and then only writing what's new.

@mr-raj12

Copy link
Copy Markdown
Contributor Author

Not sure: maybe check whether it would be better to use a user flag for F_PENDING.

The slight complication you have in the hashindex code (can't use __setitem__) would go away and you could set/clear the flag from python code.

The "system flags" are rather hashtable-internal things, like tracking what's new and then only writing what's new.

switched F_PENDING to a user

@mr-raj12 mr-raj12 marked this pull request as ready for review June 29, 2026 18:19
Comment thread src/borg/hashindex.pyx
F_PENDING moves to the user flag range so it is set/cleared via normal
assignment; flush()'s store-failure path no longer re-checks is_pending.
@mr-raj12 mr-raj12 force-pushed the chunkindex-pending-flag branch from c640a65 to 1206808 Compare June 30, 2026 03:14
@ThomasWaldmann ThomasWaldmann merged commit 52fdfc7 into borgbackup:master Jun 30, 2026
17 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants