feat(model): Add ER SDE / DPM++ 2M Scheduler Support For Anima#9125
feat(model): Add ER SDE / DPM++ 2M Scheduler Support For Anima#9125kappacommit wants to merge 47 commits intoinvoke-ai:mainfrom
Conversation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ch code Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rminal-step test Address code review feedback on order-1 ER-SDE stepper: - Add docstring preconditions to integral helpers noting the logarithmic singularity at lam=0 (callers must guard sigma_next>0). - Tighten terminal-step test from atol=1e-5 to torch.equal — the algebra is exact when sigma_next=0, not approximate.
…ssertion Address code review feedback on the 2nd-order Taylor extension tests: - Assert state.old_d_x0 equals the analytically-computed d_x0 = 0.2v / (1.5 - 4.0) rather than just checking it's non-None. - Document that x_t is intentionally re-used across calls (state threading test, not trajectory correctness). - Document the order-2 correction coefficient magnitude that justifies the atol=1e-3 threshold in the engagement test.
… margin Address code review feedback on Task 3: - Comment why have_two_back checks both old_d_x0 and sigma_prev_prev (the sigma~=1 boundary path can break the joint invariant). - Document the analytically verified ~0.0004 per-element correction magnitude that justifies the atol=1e-3 threshold in the order-3 test.
…test Address code review feedback on Task 4: - Add an in-file comment above ANIMA_SCHEDULER_MAP explaining the convention: schedulers with custom code paths (er_sde) live in the Literal+labels only, not the map. - Hoist `import typing` to module-level in test_anima_schedulers.py (was inline-imported in two test functions). - Pin the er_sde label value (== "ER-SDE"), not just key existence.
…aming Address code review feedback on Task 5: - Comment explaining why fresh_noise is float32 (matches er_sde_rf_step's dtype contract with x_t.to(float32)). - Bridging comment at the inpaint extension call clarifying that sigma_next here means the same thing as sigma_prev in the Euler branch and the AnimaInpaintExtension API.
Windows-side typegen run flipped these to backslashes. Restore the canonical forward-slash form to match upstream. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sort imports + format per project ruff config (line-length 120). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The animaScheduler field inlined its enum (originally to add er_sde
when the shared schema didn't have it yet). Now that zAnimaSchedulerField
already includes er_sde, reference the shared zParameterAnimaScheduler
to match the pattern used by scheduler/fluxScheduler/zImageScheduler.
Drops the redundant .default('euler') — initial value comes from
getInitialParamsState.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
@JPPhoto I've assigned this to you and labeled it for 6.13.0 because it does fit the theme of model support enhancements. But let me know if you think it is too late to get it in, and we'll go for 6.14. |
Let's see if we can get it in, but if timing is too tight then this shouldn't block the release. |
JPPhoto
left a comment
There was a problem hiding this comment.
invokeai/app/invocations/anima_denoise.py:506- Animadpmpp_2manddpmpp_2m_sdeignore clipped img2img/inpaint sigma schedules.denoising_start/denoising_endslicesigmas, and initial latents are noised with that clippedsigmas[0], but these DPM++ schedulers fall back toset_timesteps(num_inference_steps=total_steps)because Diffusers does not acceptsigmas=there. That regenerates a full schedule starting near sigma1.0. Example:steps=30,denoising_start=0.5should start at sigma0.75, but the scheduler starts around0.9997. To expose this issue, add a test that sets up Anima DPM++ withdenoising_start=0.5and asserts the scheduler's first sigma/timestep matches the clipped schedule.- Test gap:
invokeai/tests/backend/flux/test_anima_schedulers.py:80- The DPM++ schedule equivalence test only covers a full schedule. It missesdenoising_startanddenoising_end, which is where the runtime path diverges. - Test gap:
invokeai/tests/app/invocations/test_anima_denoise_er_sde_dispatch.py:8- The test comment says numerical correctness is gated by a parity test againster_sde_rf_step, but that parity test is not present in this PR. The included ER-SDE tests are mostly construction, finite-output, and structural tests, so they do not prove parity with the previous Anima ER-SDE implementation. - Test gap:
invokeai/backend/stable_diffusion/schedulers/schedulers.py:95-er_sdeis added to the general SD/SDXL scheduler map, but coverage is only synthetic VP smoke testing. There is no integration test throughget_scheduler()or a real SD/SDXL denoise path proving config loading, prediction type override, and scheduler stepping work together.
DPMSolverMultistepScheduler doesn't accept sigmas= in diffusers 0.35.1, so the fallback previously called set_timesteps(num_inference_steps=total_steps) which regenerated a full schedule from sigma_max, ignoring denoising_start/end. When the scheduler supports set_begin_index, call set_timesteps with the full step count and offset into it, so the internal flow_shift applies correctly and denoising starts at the right sigma. Also fixes the inpaint sigma_prev lookup and the timestep loop to use the same offset, and corrects the false parity-test reference in the ER-SDE dispatch test docstring. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Summary
ER SDE is the primary recommended scheduler for Anima. After testing, it produces the best and most coherent outputs I've seen for the model.
Structurally, the easiest way to add support for ER SDE for Anima, was to also add support for SD (1.5, 2, SDXL). So this PR adds support for Anima and SDXL.
ER SDE is not available in diffusers, so a custom implementation was required.
Reference implementation was here: https://github.com/QinpengCui/ER-SDE-Solver this is where all of the fancy math and logic comes from (MIT licensed, free to use)
I also add support for DPM++ 2M Scheduler
Related Issues / Discussions
Requested in discord https://discord.com/channels/1020123559063990373/1149506274971631688/1501016017009246379
QA Instructions
--
--
Merge Plan
Checklist
What's Newcopy (if doing a release after this PR)