Repository navigation
Consolidate repeated commit tasks and warn when a transaction queues many - #8156
Merged
Merged
Conversation
Contributor
Author
TeamCity resultsUpdated 2026-10-11 17:33 PDT for
✅ passed · 🟡 failed, not a blocker · ❌ needs attention · 🕒 not current (ran before the branch's latest commit) Failures
Sample Manager Starter postgres 🕒
|
labkey-jeckels
marked this pull request as ready for review
October 12, 2026 01:09
labkey-adam
approved these changes
Oct 14, 2026
| // Counts this transaction's monitors rather than the shared throttle, which other threads' commits also hit | ||
| private static int warningsFor(boolean commit, int count, IntConsumer register) | ||
| { | ||
| List<RepeatedCommitTaskMonitor> monitors; |
Contributor
There was a problem hiding this comment.
Could be a Stream instead
Contributor
Author
There was a problem hiding this comment.
True. Switched.
| /** The stack through the first frame outside DbScope's commit task plumbing: the code that registered the task */ | ||
| private static String getStackKey(Throwable t) | ||
| { | ||
| StackTraceElement[] frames = t.getStackTrace(); |
Contributor
There was a problem hiding this comment.
Use SqlExecutingSelector.getStackKey(), which is identical?
Contributor
Author
There was a problem hiding this comment.
The two differed in their "plumbing frames" list. I consolidated and switched to using Class instead of hard-coded class names.
Contributor
|
Manual testing looks good. Confirmed with a list configured for entire list indexing. On develop, indexEntireList() was called once per deleted row; on the FB, indexEntireList() was called once per batch of deletes. |
labkey-jeckels
added a commit
to LabKey/ehrModules
that referenced
this pull request
Oct 15, 2026
…1225) ## Rationale This PR makes EHR's post-commit demographics work run once per transaction instead of once per row: new-animal recaching, and updates to async demographics providers. It uses the accumulating commit task added in the related platform PR. ## Related Pull Requests - LabKey/platform#8156 ## Changes - Recaching of newly created demographics records is batched per container - Async demographics provider updates are batched per container and provider, covering every id changed in the transaction
8 tasks done
labkey-jeckels
added a commit
that referenced
this pull request
Oct 20, 2026
…8193) ## Rationale Big transactions spend seconds after commit reloading cache entries they've already invalidated; this replays each transaction's DatabaseCache work as one task and skips what a later removal or clear undoes. A container delete measured 1,194 post-commit reloads taking 2-17 s, nearly all for ontology caches that the same delete clears. Also adds a DEBUG summary of large commit task queues, since the throttled repeated-task warning can't show what fills one. ## Related Pull Requests - #8156 ## Changes - Replay a transaction's cache removals, clears, and reloads after commit from a single commit task, in the order they happened - Skip reloads that a later removal of the same key, or clear of the same cache, would discard - Log the most common task classes and their run times for large commit task queues at DEBUG ## Tasks - [x] Claude Code Review - [x] Manual Testing @labkey-martyp - [x] Delete container with lots of sample types and datasets (perf) - [x] Stale data check across sessions - [x] Transaction rollback doesn't update cache - [x] Batched cache clears (ClearMaterialSourceCacheTask) - [x] Warning for unbatched cache clears (datasets) - [x] Test Automation --------- Co-authored-by: labkey-tchad <tchad@labkey.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale
Instead of doing work per row, we're better served by doing it once per transaction. Also, a warning will help us find the call sites adding lots of commit tasks.
Related Pull Requests
Changes
Tasks