Skip to content

feat(model): Add ER SDE / DPM++ 2M Scheduler Support For Anima#9125

Open
kappacommit wants to merge 47 commits intoinvoke-ai:mainfrom
kappacommit:anima-er-sde
Open

feat(model): Add ER SDE / DPM++ 2M Scheduler Support For Anima#9125
kappacommit wants to merge 47 commits intoinvoke-ai:mainfrom
kappacommit:anima-er-sde

Conversation

@kappacommit
Copy link
Copy Markdown
Contributor

@kappacommit kappacommit commented May 5, 2026

Summary

ER SDE is the primary recommended scheduler for Anima. After testing, it produces the best and most coherent outputs I've seen for the model.

Structurally, the easiest way to add support for ER SDE for Anima, was to also add support for SD (1.5, 2, SDXL). So this PR adds support for Anima and SDXL.

ER SDE is not available in diffusers, so a custom implementation was required.

Reference implementation was here: https://github.com/QinpengCui/ER-SDE-Solver this is where all of the fancy math and logic comes from (MIT licensed, free to use)

I also add support for DPM++ 2M Scheduler

Related Issues / Discussions

Requested in discord https://discord.com/channels/1020123559063990373/1149506274971631688/1501016017009246379

QA Instructions

  • Use any SDXL model
  • Select the ER SDE scheduler
  • Generate an image successfully

--

  • Use any Anima model
  • Select the ER SDE scheduler
  • Generate an image successfully

--

  • Use any Anima model
  • Select the DPM++ 2M SDE scheduler
  • Generate an image successfully

Merge Plan

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

Your Name and others added 30 commits May 5, 2026 01:16
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ch code

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rminal-step test

Address code review feedback on order-1 ER-SDE stepper:
- Add docstring preconditions to integral helpers noting the logarithmic
  singularity at lam=0 (callers must guard sigma_next>0).
- Tighten terminal-step test from atol=1e-5 to torch.equal — the algebra
  is exact when sigma_next=0, not approximate.
…ssertion

Address code review feedback on the 2nd-order Taylor extension tests:
- Assert state.old_d_x0 equals the analytically-computed d_x0 = 0.2v / (1.5 - 4.0)
  rather than just checking it's non-None.
- Document that x_t is intentionally re-used across calls (state threading
  test, not trajectory correctness).
- Document the order-2 correction coefficient magnitude that justifies the
  atol=1e-3 threshold in the engagement test.
… margin

Address code review feedback on Task 3:
- Comment why have_two_back checks both old_d_x0 and sigma_prev_prev
  (the sigma~=1 boundary path can break the joint invariant).
- Document the analytically verified ~0.0004 per-element correction
  magnitude that justifies the atol=1e-3 threshold in the order-3 test.
…test

Address code review feedback on Task 4:
- Add an in-file comment above ANIMA_SCHEDULER_MAP explaining the
  convention: schedulers with custom code paths (er_sde) live in the
  Literal+labels only, not the map.
- Hoist `import typing` to module-level in test_anima_schedulers.py
  (was inline-imported in two test functions).
- Pin the er_sde label value (== "ER-SDE"), not just key existence.
…aming

Address code review feedback on Task 5:
- Comment explaining why fresh_noise is float32 (matches er_sde_rf_step's
  dtype contract with x_t.to(float32)).
- Bridging comment at the inpaint extension call clarifying that
  sigma_next here means the same thing as sigma_prev in the Euler branch
  and the AnimaInpaintExtension API.
Windows-side typegen run flipped these to backslashes. Restore the
canonical forward-slash form to match upstream.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@github-actions github-actions Bot added python PRs that change python files invocations PRs that change invocations backend PRs that change backend files frontend PRs that change frontend files python-tests PRs that change python tests labels May 5, 2026
Your Name and others added 2 commits May 5, 2026 10:16
Sort imports + format per project ruff config (line-length 120).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The animaScheduler field inlined its enum (originally to add er_sde
when the shared schema didn't have it yet). Now that zAnimaSchedulerField
already includes er_sde, reference the shared zParameterAnimaScheduler
to match the pattern used by scheduler/fluxScheduler/zImageScheduler.
Drops the redundant .default('euler') — initial value comes from
getInitialParamsState.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@kappacommit kappacommit changed the title feat(model): Add ER SDE Scheduler Support For SDXL & Anima feat(model): Add ER SDE / DPM++ 2M Scheduler Support For Anima May 5, 2026
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@lstein lstein self-assigned this May 7, 2026
@lstein lstein added the v6.13.x label May 7, 2026
@lstein lstein moved this to 6.13.x Theme: MODELS in Invoke - Community Roadmap May 7, 2026
@lstein
Copy link
Copy Markdown
Collaborator

lstein commented May 7, 2026

@JPPhoto I've assigned this to you and labeled it for 6.13.0 because it does fit the theme of model support enhancements. But let me know if you think it is too late to get it in, and we'll go for 6.14.

@JPPhoto
Copy link
Copy Markdown
Collaborator

JPPhoto commented May 7, 2026

@JPPhoto I've assigned this to you and labeled it for 6.13.0 because it does fit the theme of model support enhancements. But let me know if you think it is too late to get it in, and we'll go for 6.14.

Let's see if we can get it in, but if timing is too tight then this shouldn't block the release.

Copy link
Copy Markdown
Collaborator

@JPPhoto JPPhoto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • invokeai/app/invocations/anima_denoise.py:506 - Anima dpmpp_2m and dpmpp_2m_sde ignore clipped img2img/inpaint sigma schedules. denoising_start / denoising_end slice sigmas, and initial latents are noised with that clipped sigmas[0], but these DPM++ schedulers fall back to set_timesteps(num_inference_steps=total_steps) because Diffusers does not accept sigmas= there. That regenerates a full schedule starting near sigma 1.0. Example: steps=30, denoising_start=0.5 should start at sigma 0.75, but the scheduler starts around 0.9997. To expose this issue, add a test that sets up Anima DPM++ with denoising_start=0.5 and asserts the scheduler's first sigma/timestep matches the clipped schedule.
  • Test gap: invokeai/tests/backend/flux/test_anima_schedulers.py:80 - The DPM++ schedule equivalence test only covers a full schedule. It misses denoising_start and denoising_end, which is where the runtime path diverges.
  • Test gap: invokeai/tests/app/invocations/test_anima_denoise_er_sde_dispatch.py:8 - The test comment says numerical correctness is gated by a parity test against er_sde_rf_step, but that parity test is not present in this PR. The included ER-SDE tests are mostly construction, finite-output, and structural tests, so they do not prove parity with the previous Anima ER-SDE implementation.
  • Test gap: invokeai/backend/stable_diffusion/schedulers/schedulers.py:95 - er_sde is added to the general SD/SDXL scheduler map, but coverage is only synthetic VP smoke testing. There is no integration test through get_scheduler() or a real SD/SDXL denoise path proving config loading, prediction type override, and scheduler stepping work together.

DPMSolverMultistepScheduler doesn't accept sigmas= in diffusers 0.35.1,
so the fallback previously called set_timesteps(num_inference_steps=total_steps)
which regenerated a full schedule from sigma_max, ignoring denoising_start/end.

When the scheduler supports set_begin_index, call set_timesteps with the
full step count and offset into it, so the internal flow_shift applies
correctly and denoising starts at the right sigma.

Also fixes the inpaint sigma_prev lookup and the timestep loop to use the
same offset, and corrects the false parity-test reference in the ER-SDE
dispatch test docstring.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@kappacommit
Copy link
Copy Markdown
Contributor Author

  • invokeai/app/invocations/anima_denoise.py:506 - Anima dpmpp_2m and dpmpp_2m_sde ignore clipped img2img/inpaint sigma schedules. denoising_start / denoising_end slice sigmas, and initial latents are noised with that clipped sigmas[0], but these DPM++ schedulers fall back to set_timesteps(num_inference_steps=total_steps) because Diffusers does not accept sigmas= there. That regenerates a full schedule starting near sigma 1.0. Example: steps=30, denoising_start=0.5 should start at sigma 0.75, but the scheduler starts around 0.9997. To expose this issue, add a test that sets up Anima DPM++ with denoising_start=0.5 and asserts the scheduler's first sigma/timestep matches the clipped schedule.

    • Test gap: invokeai/tests/backend/flux/test_anima_schedulers.py:80 - The DPM++ schedule equivalence test only covers a full schedule. It misses denoising_start and denoising_end, which is where the runtime path diverges.

    • Test gap: invokeai/tests/app/invocations/test_anima_denoise_er_sde_dispatch.py:8 - The test comment says numerical correctness is gated by a parity test against er_sde_rf_step, but that parity test is not present in this PR. The included ER-SDE tests are mostly construction, finite-output, and structural tests, so they do not prove parity with the previous Anima ER-SDE implementation.

    • Test gap: invokeai/backend/stable_diffusion/schedulers/schedulers.py:95 - er_sde is added to the general SD/SDXL scheduler map, but coverage is only synthetic VP smoke testing. There is no integration test through get_scheduler() or a real SD/SDXL denoise path proving config loading, prediction type override, and scheduler stepping work together.

  • Feedback 1+2 (DPM++ img2img bug): Fixed and added a passing test covering the clipped-schedule case.

  • Feedback 3 (false parity claim): This was just a comment issue. It reference a previous implementation of er_sde, which never existed. Updated the comment, but kept the test because it still has value.

  • Feedback 4 (SD/SDXL integration test): Not addressed — inconsistent with the rest of the codebase where no other
    scheduler has a stepping integration test.

Your Name and others added 2 commits May 6, 2026 23:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend PRs that change backend files frontend PRs that change frontend files invocations PRs that change invocations python PRs that change python files python-tests PRs that change python tests v6.13.x

Projects

Status: 6.13.x Theme: MODELS

Development

Successfully merging this pull request may close these issues.

3 participants