fix(ui): recall the right VAE and text encoder per model base by Pfannkuchensack · Pull Request #9489 · invoke-ai/InvokeAI

Pfannkuchensack · 2026-08-10T19:20:04Z

Summary

Model-specific VAEs and text encoders were recalled into the wrong Redux slot, or not recallable at all, depending on the model base. This fixes the gating, wires the missing handlers into the recall UI, and adds the Anima denoise node needed to produce that metadata from a workflow.

Several bases keep their standalone components in dedicated params slots (params.fluxVAE, params.animaVaeModel, …) but record them in the shared metadata.vae / metadata.qwen3_encoder fields. Whether a handler may fire therefore depends on two axes: where the image came from (metadata.model.base) and which base is currently selected (which decides which slot is live). Neither axis was checked consistently.

What was broken

Symptom	Cause
Anima images showed a duplicate VAE row, and "recall all" also wrote the Anima VAE into the SD `params.vae` slot	`VAEModel` excluded `z-image`, `flux2`, `krea-2` — but not `anima`
Anima and FLUX.2 Klein images pushed their encoder into the Z-Image slot and cleared `zImageQwen3SourceModel`	`Qwen3EncoderModel` had no base gate at all, yet dispatched hard into the Z-Image slots
A Krea-2 image recalled while Anima was selected wrote its VAE into the Anima slot	The Anima handlers gated only on the selected base, never on the image's provenance
FLUX.1: the VAE recall button did nothing	The graph reads `params.fluxVAE`, the generic handler writes `params.vae` — no handler ever dispatched `fluxVAESelected`
Anima, Z-Image and FLUX.1 VAE / encoder had no row and no recall button in the metadata panel	The handlers were missing from the hand-maintained `IMAGE_METADATA_ACTION_HANDLERS`
Anima workflows could only record metadata by hand-chaining Metadata Item Linked nodes	`metadata_linked.py` had a self-recording denoise node for SD, FLUX.1 and Z-Image, but not for Anima

The first three were partly self-healing — the modelSelected listener clears foreign slots on the next model switch — but the panel stayed wrong and a user's Z-Image encoder selection was silently dropped.

Changes

features/metadata/parsing.tsx

New BASES_WITH_DEDICATED_VAE_HANDLER set replaces the base !== 'x' && … chain in VAEModel, documenting per base which slot and handler own it. qwen-image and wan are deliberately absent — they write qwen_image_vae / wan_vae_model and never collide.
New Flux1VAEModel handler dispatching fluxVAESelected.
Qwen3EncoderModel → ZImageQwen3EncoderModel, gated on base === 'z-image'. A variant-based check would not work here: isQwen3EncoderModelConfig is defined as variant !== 'qwen3_06b', which Klein encoders satisfy too — and the slot choice is a property of the main model, not of the encoder.
Both Anima handlers now check provenance via assertMetadataModelBase. The VAE additionally asserts parsed.base === 'anima' (meaningful, since isAnimaVAEModelConfig is base-driven); the encoder deliberately gets no base assert, since Anima encoders are identified by variant — with a comment saying so, to stop a future cleanup from "fixing" it.

ImageMetadataViewer/ImageMetadataActions.tsx — list Flux1VAEModel, ZImageVAEModel, ZImageQwen3EncoderModel, ZImageQwen3SourceModel, AnimaVAEModel, AnimaQwen3EncoderModel. Listing Flux1VAEModel is mandatory, not cosmetic: without it, adding flux to the exclusion set would remove the FLUX.1 VAE row outright.

invocations/metadata_linked.py — new anima_denoise_meta node ("Denoise - Anima + Metadata"), mirroring z_image_denoise_meta with two Anima specifics: the CFG value is recorded as cfg_scale (Z-Image writes guidance, which for Anima would land in the FLUX guidance param), and the standalone VAE / Qwen3 encoder are optional inputs, since Anima cannot derive them from the transformer field. Unconnected inputs write nothing rather than a null the recall handlers would fail to resolve. schema.ts regenerated.

Tests

parsing.test.tsx: gating for AnimaVAEModel (incl. rejecting Krea-2 provenance and metadata with no model at all), AnimaQwen3EncoderModel (incl. pinning that any encoder base parses), ZImageQwen3EncoderModel, Flux1VAEModel; the generic VAEModel matrix extended by flux and anima.
ImageMetadataActions.test.tsx: a toContain check for the six newly listed handlers, plus a guard test that diffs the whole registry against the UI list. Its allowlist is split into "hidden by design" (CreatedBy, ImageSize) and a commented snapshot of pre-existing gaps (HiDiffusion, Wan, Qwen-Image sub-models, Gemini/OpenAI/Seedream, Z-Image seed variance) — meant to shrink, not grow.
test_denoise_noise_inputs.py: the seed-selection case matching the FLUX / Z-Image tests, plus one that pins the metadata keys against the recall contract (cfg_scale set, guidance absent, model / VAE / encoder recorded) and one that checks unconnected component inputs are omitted.

pnpm lint, pnpm lint:tsc, the full frontend suite (1809 tests / 144 files), ruff, and the touched pytest modules all pass.

Out of scope

T5EncoderModel has the same missing base gate and writes into the FLUX.1 T5 slot. Left alone deliberately.
Krea-2, Qwen-Image and FLUX.2 still have no *_denoise_meta counterpart either; only Anima is added here.
Note that the provenance check requires model (with base: "anima") in the metadata for the Anima VAE / encoder rows to recall. The new node always writes it; a hand-built Metadata Item Linked chain that omits it will not recall those two fields.

Manual verification

Anima image → metadata panel shows one VAE row and one Qwen3 Encoder row, each with a working recall button.
"Recall all" on an Anima image while SDXL is selected: model switches to Anima, both Anima slots hold the values from the image (not the listener's defaults), the SD VAE slot stays empty, Z-Image slots untouched.
FLUX.1: the VAE recall now lands in the FLUX VAE select.
An Anima workflow using Denoise - Anima + Metadata, with the loader's VAE and Qwen3 Encoder connected and its metadata output wired into the L2I node, produces an image whose parameters recall in full.
SDXL and FLUX.2 Klein behave as before, in particular no VAE row disappeared.

Several bases write their standalone VAE / text encoder into the shared `metadata.vae` and `metadata.qwen3_encoder` fields but recall them into their own params slot. The generic handlers did not account for that: - `VAEModel` excluded z-image, flux2 and krea-2 but not anima, so an Anima image rendered a duplicate VAE row and additionally wrote its VAE into the shared `params.vae` slot, which no Anima graph reads. - `Qwen3EncoderModel` had no base gate at all while dispatching hard into the Z-Image slots, so Anima and FLUX.2 Klein images pushed their encoder into `zImageQwen3EncoderModel` and cleared `zImageQwen3SourceModel` on the way. - The Anima handlers gated only on the currently selected base, never on the image's own `metadata.model.base`, so a Krea-2 image recalled while Anima was selected wrote its VAE into the Anima slot. Replace the growing `base !== 'x' && ...` chain with a documented `BASES_WITH_DEDICATED_VAE_HANDLER` set, rename `Qwen3EncoderModel` to `ZImageQwen3EncoderModel` and gate it on z-image, and harden both Anima handlers with `assertMetadataModelBase` (plus a base assert on the VAE, which is base-identified - the encoder is variant-identified, so it deliberately gets none). The same slot mismatch existed for FLUX.1, whose VAE lives in `params.fluxVAE`: its recall silently landed in `params.vae` and had no effect on generation. Add a dedicated `Flux1VAEModel` handler for it. Finally, `AnimaVAEModel`, `AnimaQwen3EncoderModel`, `Flux1VAEModel` and the three Z-Image handlers were missing from IMAGE_METADATA_ACTION_HANDLERS, so those parameters had neither a row nor a per-parameter recall button. List them, and add a guard test that diffs the UI list against the handler registry with an explicit allowlist, so the list cannot drift behind again.

Workflows only carry recallable parameters if the graph builds the metadata itself. `metadata_linked.py` offers two ways to do that: chaining Metadata Item Linked nodes, or a denoise node that writes its own metadata - but the latter existed only for SD, FLUX.1 and Z-Image. Add `anima_denoise_meta`, mirroring `z_image_denoise_meta`, with two Anima specifics: - The CFG value is recorded as `cfg_scale`, matching both the Anima graph builder and the UI's recall handler. Z-Image writes `guidance`, which for Anima would land in the FLUX guidance param instead. - The VAE and Qwen3 encoder are exposed as optional inputs. Anima loads both standalone, so unlike SD/FLUX they cannot be derived from the transformer field, and without them a recalled Anima image would be missing exactly the two components that are hardest to pick by hand. Unconnected inputs write nothing rather than a null the recall handlers would fail to resolve.

JPPhoto

Merge blockers:

invokeai/frontend/web/src/features/metadata/parsing.tsx:1546-1552: Flux1VAEModel lacks image-provenance gating. A Z-Image image using a FLUX VAE now exposes this newly registered row and recalls it into params.fluxVAE. Test: current base flux, metadata model base z-image, VAE base flux; parsing must reject.
invokeai/frontend/web/src/features/gallery/components/ImageMetadataViewer/ImageMetadataActions.tsx:77-78: Newly exposed Z-Image VAE and Qwen3 handlers check only selected base, not metadata.model.base. Foreign Flux/Anima metadata can populate zImageVaeModel or zImageQwen3EncoderModel and clear the source slot. Test: current base z-image, parse shared fields from non-Z-Image metadata; both must reject.
invokeai/frontend/web/src/features/metadata/parsing.tsx:1855-1856: Rejects FLUX VAEs, although invokeai/app/invocations/anima_model_loader.py:56-68 explicitly permits them and the new metadata node records any connected VAE at invokeai/app/invocations/metadata_linked.py:812-864. Valid Anima workflows using a FLUX VAE cannot recall that VAE. Test: Anima metadata with model.base="anima" and vae.base="flux"; Anima VAE parsing must succeed.

Suggestions:

Instead of selected-base-only checks, require metadata model provenance for every shared-field handler.
Consider accepting all VAE bases supported by Anima, or reject those inputs in the metadata node.

Notes:

The first two are real, concrete UI bugs. They affect per-parameter recall when viewing one model's image while another model base is selected. The PR newly adds those handlers to the UI, making the paths reachable. Recall all is safer because MainModel runs first. These share one fix: require metadata.model.base provenance checks in the Flux/Z-Image handlers.
The third is also real, but narrower. Anima's backend node explicitly permits a FLUX VAE fallback, while the new Anima parser rejects it. A valid Anima workflow can therefore record a VAE that cannot be recalled.

VAE and text-encoder handlers that read a shared metadata field (`vae`, `qwen3_encoder`, `qwen3_source`) checked only the currently selected base, not where the image came from. Viewing one model's image while another base was selected could write a foreign model into the live slot, and in the Z-Image case silently clear the user's Qwen3 source selection. All six affected handlers now assert `metadata.model.base` in addition to the selected base. Also widen the Anima VAE slot to FLUX VAEs. The Anima model loader carries no `ui_model_base` on its VAE input and both `anima_l2i` and `anima_i2l` branch explicitly on `FluxAutoEncoder`, so a workflow-built Anima image may legitimately record one. This needs a separate `isAnimaCompatibleVAEModelConfig`: `isAnimaVAEModelConfig` also feeds Krea-2's VAE pool and must stay base-driven.

JPPhoto

Fix:

invokeai/frontend/web/src/services/api/types.ts:330 and invokeai/frontend/web/src/features/metadata/parsing.tsx:1870 reject valid 16-channel Wan VAEs, although Anima accepts them. Workflows using VAE_Checkpoint_Wan_Config can record one through anima_denoise_meta, but recall silently leaves the stale VAE. Effect: wrong VAE or generation failure. Likelihood: Medium. Recovery: manually reselect the VAE in the workflow. Test: add Wan-16 acceptance and Wan-48 rejection cases to invokeai/frontend/web/src/features/metadata/parsing.test.tsx and types.test.ts.

Suggestions:

Consider centralizing Anima VAE compatibility around backend geometry: Anima, Flux, and Wan with latent_channels=16; reject Wan-48.

The Anima model loader takes any VAE the backend can decode with — an Anima-base (Wan/QwenImage) VAE, a FLUX VAE, or a plain Wan VAE in the 16-channel A14B geometry. The frontend gated on base alone, so a workflow-built Anima image recording a Wan VAE could not recall it and recall silently left the stale VAE in place. Gate on the backend geometry instead: isAnimaCompatibleVAEModelConfig now also accepts Wan VAEs with latent_channels === 16, and rejects the 48-channel Wan2.2-VAE (TI2V-5B), which is the same AutoencoderKLWan class but a different latent space. Submodels stay out — a main model's bundled VAE carries no latent_channels, so its geometry is unverifiable. Since latent_channels is lost in ModelIdentifierField, AnimaVAEModel now gates on the full model config: parseModelConfig is split out of parseModelIdentifier, which becomes a thin wrapper over it. The resolution order (key, then hash, then name/base) and its fallthrough behaviour are unchanged. isAnimaVAEModelConfig stays base-driven — Krea-2 draws its own VAE pool from it and must not be offered FLUX or Wan VAEs.

JPPhoto

Close! This should be fixed:

parsing.tsx: Metadata rows are parsed only on [metadata, handler, store]; selected base changes do not trigger reparse, and recall does not revalidate. After opening Anima metadata while another model is selected, recalling MainModel leaves Anima VAE/encoder rows hidden. Conversely, switching away after a row appears leaves a stale button that can mutate an inactive slot. Likelihood: normal workflow. Test: switch base after mounting ImageMetadataActions; assert rows update and stale clicks do nothing.

Suggestions:

Subscribe to selected base in the metadata datum hook and reparse on changes; also validate the base again inside each recall callback.

Addresses review 4987208275 on invoke-ai#9489. Metadata handlers gate on the currently selected base, which they read imperatively via `selectBase(store.getState())` inside `parse` — they have to, because parsing also happens outside React, in recall-all and in hotkeys. The datum hooks depended only on `[metadata, handler, store]`, so a row carried the verdict of whichever base was selected when it mounted, and the metadata viewer stays open across model switches. Opening Anima metadata under another model and then recalling MainModel left the Anima VAE and encoder rows hidden; switching away after a row appeared left a live recall button pointed at a slot no longer in play. The three hooks now subscribe to the selected base and reparse when it changes. That reparse is asynchronous, so a click can still land while a stale row is on screen. Recalls therefore re-run the handler's own gate at click time: `recallIfStillValid` parses again and dispatches only if the gate still admits the metadata. It recalls the value the row displays, not the reparsed one — a collection row owns a single item, while `parse` returns the whole set. Fixing the hooks alone would have changed nothing: `SingleMetadataParsed` and `CollectionMetadataParsed` called `handler.recall` / `handler.recallOne` directly and resolved the store themselves, and the hooks' `recall`, `recallAll` and `recallOne` had no callers anywhere. The row components now take the hook's validating callback instead, which is the only recall path left outside `parsing.tsx`. Tested through `recallIfStillValid`, which is exported as a plain function for that purpose: it recalls under a matching base, does nothing after the base moves on, does nothing when provenance fails, and passes the row's own value through. The reparse half is not covered — the project has no testing-library or DOM environment and does not do UI tests, so it would need a new dev dependency to assert.

The FLUX.1 and Z-Image VAE slots were auto-filled from the wider flux+flux2 pool while their pickers and recall handlers accept FLUX.1 only, and every VAE recall handler rejected the main-model VAE submodels its picker offers - so the VAE row silently disappeared instead of recalling. - default `params.fluxVAE` and `params.zImageVaeModel` from `isFlux1VAEModelConfig` - share `parseVAEModelIdentifier` across the FLUX.1 / FLUX.2 / Z-Image / Anima handlers, each gating on its own picker's guard - prefer a native Anima VAE over the FLUX/Wan fallbacks when defaulting the slot - call type guards with one argument in `buildModelsSelector`; `Array#filter` was passing the index as `excludeSubmodels` - surface a toast when a revalidated recall is skipped - collapse the three metadata datum hooks into one and export the verdict function so the base-change reparse is testable

JPPhoto

Another thing to fix:

invokeai/frontend/web/src/features/metadata/parsing.tsx:1908: Anima recall checks only type === 'qwen3_encoder; it does not require variant === 'qwen3_06b' like the Anima picker. Anima metadata containing a 4B/8B encoder therefore poisons state; next generation expects 1024-wide embeddings and can fail. Effect: invalid Anima recall. Likelihood: rare. Test: reject non-qwen3_06b configs. Recovery: clear and select the 0.6B encoder.

Suggestions:

Instead of only checking encoder type, resolve the full config and apply isAnimaQwen3EncoderModelConfig.

selectFluxVAEModels lost its last consumer when the FLUX.1 and Z-Image VAE defaults moved to selectFlux1VAEModels. Removing the selector also leaves isFluxVAEModelConfig unused, which knip flags in turn - both go.

Qwen3 encoders are split by `variant`, not by base: Anima uses the 0.6B (hidden_size 1024), Z-Image and Klein the 4B/8B (2560/4096). A ModelIdentifierField carries no variant, so all three recall handlers accepted each other's encoders and left the slot with one whose embeddings the transformer cannot consume. Resolve the full config and apply the guard each slot's picker is built from - isAnimaQwen3EncoderModelConfig for Anima, isQwen3EncoderModelConfig for Z-Image and Klein.

JPPhoto

invokeai/frontend/web/src/features/parameters/components/Advanced/ParamAnimaModelSelect.tsx:27-58: Anima now accepts Flux and Wan VAEs, but modelsLoaded.ts:75-100,196-218 never validates params.animaVaeModel. Removing the selected VAE leaves stale state and later Anima generation can fail. Effect: stale model selection and generation failure. Likelihood: plausible after model removal/refetch. Recovery: manually clear/reselect VAE. Test: refresh model list without selected VAE; assert Anima VAE state clears.
docs/src/content/docs/features/Workflows/community-nodes.mdx:421-441: new Anima metadata node is absent from Metadata Linked Nodes documentation. Effect: users cannot discover its Anima-specific metadata inputs. Likelihood: normal for documentation users. Recovery: inspect node registry/source. Test: build docs and verify Anima Denoise + Metadata appears.

Edge case:

invokeai/frontend/web/src/features/metadata/parsing.tsx:1522-1530, 1897-1902: with no active main model, generic VAE recall dispatches to inactive params.vae; dedicated Flux/Z-Image/Flux.2/Krea-2/Anima slots stay unchanged. Effect: individual VAE recall silently fails. Likelihood: plausible during startup or with no main model. Recovery: select or recall main model first. Test: click VAE metadata action with params.model = null; assert dedicated slot update.

Pfannkuchensack added 2 commits August 10, 2026 17:23

Pfannkuchensack requested review from JPPhoto, blessedcoolant, dunkeroni and lstein as code owners August 10, 2026 19:20

github-actions Bot added python PRs that change python files invocations PRs that change invocations frontend PRs that change frontend files python-tests PRs that change python tests labels Aug 10, 2026

Merge branch 'main' into fix/metadata-recall-anima-gating

84ea23e

lstein assigned JPPhoto Aug 17, 2026

lstein added the 6.14.1 label Aug 17, 2026

lstein added this to Invoke - Community Roadmap Aug 17, 2026

lstein moved this to 6.14.1: Bug fixes to 6.14.0 in Invoke - Community Roadmap Aug 17, 2026

JPPhoto and others added 3 commits August 18, 2026 20:36

Merge branch 'main' into fix/metadata-recall-anima-gating

0d34668

Merge branch 'main' into fix/metadata-recall-anima-gating

465d0db

Merge branch 'main' into fix/metadata-recall-anima-gating

902944e

JPPhoto requested changes Aug 18, 2026

View reviewed changes

Pfannkuchensack requested a review from JPPhoto August 19, 2026 01:16

Merge branch 'main' into fix/metadata-recall-anima-gating

b25870b

JPPhoto requested changes Aug 19, 2026

View reviewed changes

Pfannkuchensack requested a review from JPPhoto August 20, 2026 03:17

JPPhoto requested changes Aug 20, 2026

View reviewed changes

Pfannkuchensack requested a review from JPPhoto August 20, 2026 21:51

JPPhoto and others added 2 commits August 20, 2026 19:34

fix(ui): recall Anima VAE submodels

5af9c7d

Pfannkuchensack added 2 commits August 21, 2026 21:37

Chore typegen

2665674

Fix export

359787a

JPPhoto requested changes Aug 21, 2026

View reviewed changes

Pfannkuchensack added 2 commits August 21, 2026 22:14

fix(ui): drop the now-dead flux+flux2 VAE predicate

67f1972

selectFluxVAEModels lost its last consumer when the FLUX.1 and Z-Image VAE defaults moved to selectFlux1VAEModels. Removing the selector also leaves isFluxVAEModelConfig unused, which knip flags in turn - both go.

Pfannkuchensack requested a review from JPPhoto August 21, 2026 20:44

JPPhoto requested changes Aug 22, 2026

View reviewed changes

Navigation Menu

Uh oh!

fix(ui): recall the right VAE and text encoder per model base - #9489

fix(ui): recall the right VAE and text encoder per model base#9489
Pfannkuchensack wants to merge 16 commits into
invoke-ai:mainfrom
Pfannkuchensack:fix/metadata-recall-anima-gating

Pfannkuchensack commented Aug 10, 2026

Uh oh!

JPPhoto left a comment

Uh oh!

JPPhoto left a comment

Uh oh!

JPPhoto left a comment

Uh oh!

JPPhoto left a comment

Uh oh!

JPPhoto left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Pfannkuchensack commented Aug 10, 2026

Summary

What was broken

Changes

Tests

Out of scope

Manual verification

Uh oh!

JPPhoto left a comment

Choose a reason for hiding this comment

Uh oh!

JPPhoto left a comment

Choose a reason for hiding this comment

Uh oh!

JPPhoto left a comment

Choose a reason for hiding this comment

Uh oh!

JPPhoto left a comment

Choose a reason for hiding this comment

Uh oh!

JPPhoto left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants