Skip to content

feat(partition): add :table mode for :processing_batch_size option#15

Merged
cabol merged 1 commit intomainfrom
cabol/fixes-and-enhancements
Mar 12, 2026
Merged

feat(partition): add :table mode for :processing_batch_size option#15
cabol merged 1 commit intomainfrom
cabol/fixes-and-enhancements

Conversation

@cabol
Copy link
Copy Markdown
Contributor

@cabol cabol commented Mar 7, 2026

Allow passing the ETS table directly to the processor instead of reading and batching internally. When :processing_batch_size is set to :table, the processor receives the table name and has full control over how to read and process the data. The processor can optionally keep the table for later processing via :ets.rename/2 + :ets.give_away/3.

Also includes ASCII data flow diagrams in module docs, CI migration to GitHub Actions, and minor documentation fixes.

Closes #14


Note

Medium Risk
Changes the core partition processing path to optionally hand off the raw ETS table to user code and alters table-lifecycle semantics (including conditional deletion), which could affect processing correctness and resource cleanup if misused. CI and tooling updates are low risk but broaden supported Elixir/OTP versions and release metadata.

Overview
Adds a new :table mode for :processing_batch_size so processors can receive the handed-off ETS table name directly (instead of batched :ets.select results), enabling custom read/processing strategies and optional table retention via :ets.rename/2 + :ets.give_away/3.

Updates PartitionedBuffer.Partition to support the new mode and to only delete the handed-off table if it still exists under its original name, with new tests covering both Queue and Map behaviors.

Also bumps version to 0.4.0, expands option docs and module docs with ASCII flow diagrams, adds repo-specific agent/usage-rules files + usage_rules dev dependency, loosens the minimum Elixir requirement to ~> 1.17, and migrates CI from CircleCI to GitHub Actions (matrix across Elixir/OTP with caching, lint, coverage, and dialyzer).

Written by Cursor Bugbot for commit e0ec13f. This will update automatically on new commits. Configure here.

@cabol cabol force-pushed the cabol/fixes-and-enhancements branch from 86c9b5c to 7ce0481 Compare March 7, 2026 14:57
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Dialyzer step skipped when PLT cache exists
    • Updated the GitHub Actions condition so dialyzer runs whenever matrix.dialyzer is true while keeping PLT creation cache-gated.

Create PR

Or push these changes by commenting:

@cursor push 17e96eaea8
Preview (17e96eaea8)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -121,4 +121,4 @@
 
       - name: Run dialyzer
         run: mix dialyzer --format github
-        if: ${{ matrix.dialyzer && steps.plt-cache.outputs.cache-hit != 'true' }}
+        if: ${{ matrix.dialyzer }}
This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.

Comment thread .github/workflows/ci.yml Outdated
@cabol cabol force-pushed the cabol/fixes-and-enhancements branch from 7ce0481 to 1b5f9bd Compare March 7, 2026 15:09
@cabol cabol requested a review from andyleclair March 7, 2026 15:32
Comment thread .github/workflows/ci.yml Outdated
@cabol cabol changed the title feat(partition): add :table mode for processing_batch_size option feat(partition): add :table mode for :processing_batch_size option Mar 12, 2026
@cabol cabol force-pushed the cabol/fixes-and-enhancements branch from 1b5f9bd to 04ef65b Compare March 12, 2026 18:52
Allow passing the ETS table directly to the processor instead of
reading and batching internally. When `:processing_batch_size` is set
to `:table`, the processor receives the table name and has full control
over how to read and process the data. The processor can optionally
keep the table for later processing via `:ets.rename/2` + `:ets.give_away/3`.

Also includes ASCII data flow diagrams in module docs, CI migration
to GitHub Actions, and minor documentation fixes.

Closes #14
@cabol cabol force-pushed the cabol/fixes-and-enhancements branch from 04ef65b to e0ec13f Compare March 12, 2026 18:56
@cabol cabol merged commit ec0443a into main Mar 12, 2026
5 of 6 checks passed
@cabol cabol deleted the cabol/fixes-and-enhancements branch April 5, 2026 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support :table mode for processing_batch_size

2 participants