Skip to content

Stop airflow-core tests from requiring the celery provider - #72249

Draft
rjgoyln wants to merge 2 commits into
apache:mainfrom
rjgoyln:core-tests-drop-celery
Draft

Stop airflow-core tests from requiring the celery provider#72249
rjgoyln wants to merge 2 commits into
apache:mainfrom
rjgoyln:core-tests-drop-celery

Conversation

@rjgoyln

@rjgoyln rjgoyln commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Summary

unit/cli/conftest.py imports the celery executor at module level, so with the provider uninstalled nothing under unit/cli collects at all — the first celery blocker to running the core suite in a scoped uv sync --project airflow-core. The rest are stand-ins: CeleryExecutor wherever a test wants an executor that is not local, [celery] wherever it wants an option a provider contributed.

Where core has its own equivalent the stand-in is replaced rather than skipped, so the test keeps running celery-free. The provider-config toggle test moves to [standard] venv_install_method; its subject is the context manager, and the one property celery carried that standard does not — metadata beating provider_config_fallback_defaults.cfg — is what test_provider_metadata_overrides_cfg_fallback is for.

TestProviderConfigPriority rows are marked by the section that owns them rather than by which ones fail. [celery] and [celery_kubernetes_executor] both come from providers/celery/provider.yaml, and several of their options resolve to the same value from the core cfg fallback, so those cases would keep passing without proving anything.

test_executor_loader.py's module-level importorskip hid the whole file rather than the four cases that need celery. Its amazon importorskip, the cncf.kubernetes import in unit/cli/conftest.py and the dev-group entry itself stay for the remaining slices.

The celery CLI integration test moves under the provider because testable-core-integrations is kerberos, otel and redis — INTEGRATION_CELERY is never true under airflow-core, so the test has never run. It does now, which is how the stale _bundle_cleanup_main expectation, renamed in #62655, surfaced; correcting it is the only assertion this PR changes.

Tests

Without apache-airflow-providers-celery, airflow-core/tests/unit and tests/integration collect clean where unit/cli previously failed outright, and the touched files skip instead of failing — 11 in test_configuration.py, 4 in unit/executors, 2 in test_cli_parser.py, one each in the scheduler and standalone command tests. With the provider installed nothing changes.

test_sensitive_values and test_providers_manager.py::test_cli still fail in a scoped venv and are left alone: both assert against the full provider set and already miss keycloak and edge3 there.

markers.py and unit/cli/conftest.py overlap #71868, where skip_if_not_installed is identical, so whichever lands second is a dedupe.

related: #71641


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5)

Generated-by: Claude Code (Opus 5) following the guidelines

The core suite uses CeleryExecutor as its stand-in for "a distributed
executor" and the [celery] section as its stand-in for "config a provider
contributed", so a scoped `uv sync --project airflow-core` environment cannot
run it unless the celery provider is installed. One of those uses is a
conftest import, which takes the whole unit/cli directory down with it.

The celery CLI integration test also sat under airflow-core, where the celery
integration is never started, so it has never actually run — long enough for
the subprocess target it asserts on to be renamed underneath it.
@rjgoyln
rjgoyln force-pushed the core-tests-drop-celery branch from 3469109 to f5b4daa Compare August 29, 2026 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant