Stop airflow-core tests from requiring the celery provider - #72249
Draft
rjgoyln wants to merge 2 commits into
Draft
Stop airflow-core tests from requiring the celery provider#72249rjgoyln wants to merge 2 commits into
rjgoyln wants to merge 2 commits into
Conversation
The core suite uses CeleryExecutor as its stand-in for "a distributed executor" and the [celery] section as its stand-in for "config a provider contributed", so a scoped `uv sync --project airflow-core` environment cannot run it unless the celery provider is installed. One of those uses is a conftest import, which takes the whole unit/cli directory down with it. The celery CLI integration test also sat under airflow-core, where the celery integration is never started, so it has never actually run — long enough for the subprocess target it asserts on to be renamed underneath it.
rjgoyln
force-pushed
the
core-tests-drop-celery
branch
from
August 29, 2026 12:31
3469109 to
f5b4daa
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
unit/cli/conftest.pyimports the celery executor at module level, so with the provider uninstalled nothing underunit/clicollects at all — the first celery blocker to running the core suite in a scopeduv sync --project airflow-core. The rest are stand-ins:CeleryExecutorwherever a test wants an executor that is not local,[celery]wherever it wants an option a provider contributed.Where core has its own equivalent the stand-in is replaced rather than skipped, so the test keeps running celery-free. The provider-config toggle test moves to
[standard] venv_install_method; its subject is the context manager, and the one property celery carried thatstandarddoes not — metadata beatingprovider_config_fallback_defaults.cfg— is whattest_provider_metadata_overrides_cfg_fallbackis for.TestProviderConfigPriorityrows are marked by the section that owns them rather than by which ones fail.[celery]and[celery_kubernetes_executor]both come fromproviders/celery/provider.yaml, and several of their options resolve to the same value from the core cfg fallback, so those cases would keep passing without proving anything.test_executor_loader.py's module-levelimportorskiphid the whole file rather than the four cases that need celery. Its amazonimportorskip, the cncf.kubernetes import inunit/cli/conftest.pyand the dev-group entry itself stay for the remaining slices.The celery CLI integration test moves under the provider because
testable-core-integrationsis kerberos, otel and redis —INTEGRATION_CELERYis never true underairflow-core, so the test has never run. It does now, which is how the stale_bundle_cleanup_mainexpectation, renamed in #62655, surfaced; correcting it is the only assertion this PR changes.Tests
Without
apache-airflow-providers-celery,airflow-core/tests/unitandtests/integrationcollect clean whereunit/clipreviously failed outright, and the touched files skip instead of failing — 11 intest_configuration.py, 4 inunit/executors, 2 intest_cli_parser.py, one each in the scheduler and standalone command tests. With the provider installed nothing changes.test_sensitive_valuesandtest_providers_manager.py::test_clistill fail in a scoped venv and are left alone: both assert against the full provider set and already miss keycloak and edge3 there.markers.pyandunit/cli/conftest.pyoverlap #71868, whereskip_if_not_installedis identical, so whichever lands second is a dedupe.related: #71641
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 5) following the guidelines