Skip to content
Open
2 changes: 1 addition & 1 deletion .ai/references/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Follow the style introduced in [#14113](https://github.com/huggingface/diffusers
- `HunyuanDiTAttentionPool` (`src/diffusers/models/embeddings.py`) shows the same failure without an MHA module: a plain `nn.Module` that hands its `q_proj` / `k_proj` / `v_proj` / `c_proj` weights to `torch.nn.functional.multi_head_attention_forward`, so all four projections stay offloaded rather than just one. `HunyuanDiT2DModel` opts out of group offloading entirely with `_supports_group_offloading = False`.
- Before adding a skip or an exclusion, confirm the failure still reproduces — several existing skips are stale, having outlived the upstream cause.
- **A migration that surfaces a `src/` gap marks the test `xfail`, it does not patch the pipeline.** Give the marker a module-level name and a `reason` naming the exact gap (`PNDM_*` in `tests/pipelines/pndm/test_pndm.py` is the worked example), and prefer `strict=True` so the marker reports XPASS — and gets deleted — the day the pipeline is fixed. Use `strict=False` only when one mark covers a group whose members do not all fail. Marking a whole test class keeps the mixin's own marks (`@is_memory`, `@require_accelerator`) intact; overriding individual inherited tests drops the decorators they were declared with, so re-declare those too.
- **`from_pipe` tests** (a pipeline that is a variant of an existing one — PAG, AnimateDiff, ...) compose the shared `FromPipeTesterMixin` (`tests/pipelines/testing_utils/from_pipe.py`, exported from `..testing_utils`) in their own test class. It derives the original pipeline from `pipeline_class.__name__`; set `original_pipeline_repo` on the test class to pull it from a repo other than the default for that class. The unittest-era `PipelineFromPipeTesterMixin` in `tests/pipelines/test_pipelines_common.py` is what it replaces.
- **`from_pipe` tests** (a pipeline that is a variant of an existing one — PAG, AnimateDiff, ...) compose the shared `FromPipeTesterMixin` (`tests/pipelines/testing_utils/from_pipe.py`, exported from `..testing_utils`) in their own test class. It derives the original pipeline from `pipeline_class.__name__`; set `original_pipeline_repo` on the test class to pull it from a repo other than the default for that class.
- **A hardware gap is a conditional skip, not an xfail.** When a test fails only because the runner's cuDNN build has no kernel for an op — `RuntimeError: GET was unable to find an engine to execute this computation`, as Sana's depthwise `Conv2d` hits in bfloat16 — wrap the call in `skip_if_no_cudnn_engine()` (`tests/testing_utils.py`). It skips on that error and re-raises every other `RuntimeError`, so the test still runs wherever the kernel exists.
- **PAG pipelines** also compose `PAGPipelineTesterMixin` (`tests/pipelines/pag/testing_utils.py`) in place of `PipelineTesterMixin`: it adds `test_pag_disable_enable` and `test_pag_inference` on top, driven by `base_pipeline_class` and the `pag_*` knobs on the test class. Keep `test_pag_applied_layers` per pipeline — which layers PAG resolves to is model-specific.
- **`encode_prompt` reading a component that isn't a text encoder or tokenizer?** `test_encode_prompt_works_in_isolation` rebuilds the pipeline with only the components whose names contain `text` or `tokenizer`. When `encode_prompt` also needs another one — a `processor` used for chat templating, say — list it in `text_stack_component_names` on the config class rather than re-implementing the test.
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/nightly_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,6 @@ jobs:
pytest -n 1 --max-worker-restart=0 --dist=loadfile \
--make-reports=tests_torch_minimum_version_cuda \
tests/models/test_modeling_common.py \
tests/pipelines/test_pipelines_common.py \

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer exists.

tests/pipelines/test_pipeline_utils.py \
tests/pipelines/test_pipelines.py \
tests/pipelines/test_pipelines_auto.py \
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/pr_tests_gpu.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@ on:
- "src/diffusers/loaders/lora_base.py"
- "src/diffusers/loaders/lora_pipeline.py"
- "src/diffusers/loaders/peft.py"
- "tests/pipelines/test_pipelines_common.py"
- "tests/pipelines/testing_utils/**.py"
- "tests/models/test_modeling_common.py"
- "tests/models/testing_utils/**.py"
Comment on lines -17 to +19

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To have parity in our CI when test changes happen.

- "examples/**/*.py"
workflow_dispatch:

Expand Down
1 change: 0 additions & 1 deletion .github/workflows/release_tests_fast.yml
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,6 @@ jobs:
pytest -n 1 --max-worker-restart=0 --dist=loadfile \
--make-reports=tests_torch_minimum_cuda \
tests/models/test_modeling_common.py \
tests/pipelines/test_pipelines_common.py \

@sayakpaul sayakpaul Sep 3, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer exists.

tests/pipelines/test_pipeline_utils.py \
tests/pipelines/test_pipelines.py \
tests/pipelines/test_pipelines_auto.py \
Expand Down
2 changes: 1 addition & 1 deletion tests/models/transformers/test_models_transformer_flux.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@


# TODO: This standalone function maintains backward compatibility with pipeline tests
# (tests/pipelines/test_pipelines_common.py) and will be refactored.
# (tests/pipelines/testing_utils/ip_adapter.py) and will be refactored.
def create_flux_ip_adapter_state_dict(model) -> dict[str, dict[str, Any]]:
"""Create a dummy IP Adapter state dict for Flux transformer testing."""
ip_cross_attn_state_dict = {}
Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/deepfloyd_if/test_if.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,11 @@
slow,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
PipelineOffloadTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/deepfloyd_if/test_if_img2img.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@
slow,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
PipelineOffloadTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@
slow,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
PipelineOffloadTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/deepfloyd_if/test_if_inpainting.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@
slow,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
PipelineOffloadTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@
slow,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
PipelineOffloadTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/deepfloyd_if/test_if_superresolution.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@
slow,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
PipelineOffloadTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/kandinsky/test_kandinsky.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,11 @@
slow,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
MemoryTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/kandinsky/test_kandinsky_img2img.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,11 @@
slow,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
MemoryTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/kandinsky/test_kandinsky_inpaint.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,11 @@
require_torch_accelerator,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
MemoryTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/shap_e/test_shap_e.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,11 @@
require_torch_accelerator,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
MemoryTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/shap_e/test_shap_e_img2img.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,11 @@
require_torch_accelerator,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
MemoryTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
Loading
Loading