I'm using stable-diffusion.cpp to generate LTX-2.3 videos with CUDA. The diffusion stage is reasonably fast, but Audio VAE decoding becomes disproportionately slow as the number of video frames increases.
Environment
- VRAM: 12 GB on RTX 3060
- RAM: 32 GB
- Model:
ltx-2.3-22b-distilled-1.1-UD-Q4_K_M.gguf
- Video VAE:
ltx-2.3-22b-distilled_video_vae.safetensors
- Audio VAE:
ltx-2.3-22b-distilled_audio_vae.safetensors
- LLM:
gemma-3-12b-it-IQ4_XS.gguf
- Embeddings:
ltx-2.3-22b-dev_embeddings_connectors.safetensors
- Backend: CUDA
--offload-to-cpu is required because my GPU does not have enough VRAM to hold the entire model- Resolution:
640x384
- FPS:
24
- Steps:
1
- Sampling: Euler
- CFG:
1.0
--temporal-tiling--diffusion-fa
Command
sd-cli -M vid_gen \
--diffusion-model ~/ComfyUI/models/diffusion_models/$DIFFUSION_MODEL \
--vae ~/ComfyUI/models/vae/$VIDEO_VAE \
--audio-vae ~/ComfyUI/models/checkpoints/$AUDIO_VAE \
--llm ~/ComfyUI/models/text_encoders/$LLM \
--embeddings-connectors ~/ComfyUI/models/text_encoders/$EMBEDDING \
-p "$PROMPT" \
--cfg-scale 1.0 \
--sampling-method euler \
-v \
-W 640 -H 384 \
--diffusion-fa \
--backend cuda0 \
--offload-to-cpu \
--max-vram -1 \
--temporal-tiling \
--video-frames 120 \
--fps 24 \
--steps 1 \
-t 12 \
-o t2v.webmProblem
The diffusion stage is fast:
[INFO ] stable-diffusion.cpp:7119 - generating latent video completed, taking 18.84s
However, decoding the audio latent takes almost 2.5 minutes:
[DEBUG] ggml_extend.hpp:2193 - ltx_audio_vae compute buffer size: 289.29 MB(VRAM)
[INFO ] ltx_audio_vae.hpp:1047 - ltx audio vae decode completed, taking 145.54s
[INFO ] stable-diffusion.cpp:7152 - decoding audio latent completed, taking 145.55s
The video VAE itself takes only about 22 seconds:
[INFO ] vae.hpp:220 - computing vae decode graph completed, taking 22.12s
[INFO ] stable-diffusion.cpp:6547 - decode_first_stage completed, taking 22.12s
Audio VAE scaling with video length
I tested with different video frame lengths and duration seems to be increasing quadratically:
- 33 frames: 15 seconds
- 65 frames: 50 seconds
- 120 frames: 145 seconds
- 121 frames: 168 seconds
I am not sure if this is a bug or a limitation. But the same generation on comfy uses takes only about 45 seconds included video and audio generation. Is there a command line flag that I must use to fix this?
I'm using
stable-diffusion.cppto generate LTX-2.3 videos with CUDA. The diffusion stage is reasonably fast, but Audio VAE decoding becomes disproportionately slow as the number of video frames increases.Environment
ltx-2.3-22b-distilled-1.1-UD-Q4_K_M.ggufltx-2.3-22b-distilled_video_vae.safetensorsltx-2.3-22b-distilled_audio_vae.safetensorsgemma-3-12b-it-IQ4_XS.ggufltx-2.3-22b-dev_embeddings_connectors.safetensors--offload-to-cpuis required because my GPU does not have enough VRAM to hold the entire model640x3842411.0--temporal-tiling--diffusion-faCommand
sd-cli -M vid_gen \ --diffusion-model ~/ComfyUI/models/diffusion_models/$DIFFUSION_MODEL \ --vae ~/ComfyUI/models/vae/$VIDEO_VAE \ --audio-vae ~/ComfyUI/models/checkpoints/$AUDIO_VAE \ --llm ~/ComfyUI/models/text_encoders/$LLM \ --embeddings-connectors ~/ComfyUI/models/text_encoders/$EMBEDDING \ -p "$PROMPT" \ --cfg-scale 1.0 \ --sampling-method euler \ -v \ -W 640 -H 384 \ --diffusion-fa \ --backend cuda0 \ --offload-to-cpu \ --max-vram -1 \ --temporal-tiling \ --video-frames 120 \ --fps 24 \ --steps 1 \ -t 12 \ -o t2v.webmProblem
The diffusion stage is fast:
However, decoding the audio latent takes almost 2.5 minutes:
The video VAE itself takes only about 22 seconds:
Audio VAE scaling with video length
I tested with different video frame lengths and duration seems to be increasing quadratically:
I am not sure if this is a bug or a limitation. But the same generation on comfy uses takes only about 45 seconds included video and audio generation. Is there a command line flag that I must use to fix this?