Skip to content

AIP-76: Add PartitionAtRuntime authoring API to Task SDK#65447

Open
anishgirianish wants to merge 2 commits intoapache:mainfrom
anishgirianish:aip-76-partition-at-runtime-authoring-api
Open

AIP-76: Add PartitionAtRuntime authoring API to Task SDK#65447
anishgirianish wants to merge 2 commits intoapache:mainfrom
anishgirianish:aip-76-partition-at-runtime-authoring-api

Conversation

@anishgirianish
Copy link
Copy Markdown
Contributor

@anishgirianish anishgirianish commented Apr 17, 2026


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

First of three PRs decomposing draft #65300 for AIP-76, per the design agreed in #44146.

Adds the Task SDK producer surface for runtime partition keys:

  • PartitionAtRuntime() marker timetable
  • PartitionKey(key, extra) for per-event metadata
  • outlet_events[asset].partition_keys = [...] / .add_partition(...)
  • self.partition_keys = [...] inside @asset via _AssetSelfProxy
  • outlet_events kwarg injection in @asset / @asset.multi

No server-side consumption yet, that lands in PR 2 (core behavior + fan-out) along with PR 3 (example DAG + docs).

cc: @Lee-W

related: #44146
related: #65300


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@anishgirianish anishgirianish force-pushed the aip-76-partition-at-runtime-authoring-api branch from 38335fb to e71d7e3 Compare April 17, 2026 22:21
@anishgirianish anishgirianish marked this pull request as draft April 17, 2026 23:26
@anishgirianish anishgirianish force-pushed the aip-76-partition-at-runtime-authoring-api branch from 939a95e to 08d1a66 Compare April 17, 2026 23:34
@anishgirianish anishgirianish marked this pull request as ready for review April 18, 2026 01:39
@anishgirianish anishgirianish force-pushed the aip-76-partition-at-runtime-authoring-api branch from 08d1a66 to ee3ee08 Compare April 18, 2026 01:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant