Skip to content

CAS avoid manifest get on promote - #2528

Draft
k-morozov wants to merge 10 commits into
antalya-26.6from
perf/cas-avoid-manifest-get-on-promote
Draft

k-morozov wants to merge 10 commits into
antalya-26.6from
perf/cas-avoid-manifest-get-on-promote

Conversation

@k-morozov

@k-morozov k-morozov commented Oct 9, 2026 •

Copy link
Copy Markdown

1'000 parts were inserted:

  ┌──────────────┬──────────┬───────┬────────┬───────────┐
  │    Metric    │ Baseline │ Patch │ Delta  │ Reduction │
  ├──────────────┼──────────┼───────┼────────┼───────────┤
  │ S3 GetObject │    7,997 │ 6,997 │ -1,000 │    12.50% │
  └──────────────┴──────────┴───────┴────────┴───────────┘

Changelog category (leave one):

  • Performance Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Reduced CAS object storage requests by reusing the proof recorded when a manifest is staged. Promotion no longer reads the manifest back, and cleanup deletes abandoned staged manifests using the recorded incarnation without a preliminary HEAD request.

Documentation entry for user-facing changes

...

CI/CD Options

Exclude tests:

  • Fast test
  • Integration Tests
  • Stateless tests
  • Stateful tests
  • Unit tests
  • Performance tests
  • Aarch64 tests
  • All with ASAN
  • All with TSAN
  • All with MSAN
  • All with UBSAN
  • All with Coverage
  • All Regression
  • Disable CI Cache

Regression jobs to run:

  • Fast suites (mostly <1h)
  • Aggregate Functions (2h)
  • Alter (1.5h)
  • Benchmark (30m)
  • CAS (content-addressed storage; Antalya only)
  • ClickHouse Keeper (1h)
  • Iceberg (2h)
  • LDAP (1h)
  • OAuth (5m)
  • Parquet (1.5h)
  • RBAC (1.5h)
  • SSL Server (1h)
  • S3 (2h)
  • S3 Export (2h)
  • Swarms (30m)
  • Tiered Storage (2h)

@k-morozov k-morozov changed the title Perf/cas avoid manifest get on promote CAS avoid manifest get on promote Oct 9, 2026
@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Workflow [PR], commit [e98f128]

k-morozov and others added 7 commits October 9, 2026 16:02
`promote` no longer re-reads a manifest staged by the same `PartWriteTxn`, so the
"body-absent precommit is rejected" contract now applies only to manifests not staged
there. Also document that `staged_manifest_proofs` entries must never be erased,
because `precommitAdd` uses them as the A3 barrier.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Extract `blobDependenciesOf` so `stageManifest` and the non-staged path of `promote`
use the same filter. Cleanup of staged manifests now logs at debug level when the
exact-incarnation remove returns `Gone` or `Mismatch`, or throws, instead of
silently swallowing it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a test that promotes its own staged manifest with Inline and Blob entries: it
fails with `no dependency proof` while one blob is missing, then commits without a
manifest read. Add a test that `abandon` keeps the live precommit body and removes
another staged manifest without `HEAD`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The record only says what a confirmed put-if-absent returned in `stageManifest`. It
does not prove that the body still exists at `promote` time, so "proof" overstated it.
Rename the member to `staged_manifests` and say "record" in the docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The blob dependencies of a `StagedManifest` keep the order the caller passed the
entries, which can differ from the path-sorted encoded body. `promote` does not detect
a staged body removed outside the protocol; the read path and `fsck` report it later.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant