Hey everyone, I’ve been testing the MiniMax-H3 Ref2VA workflow with the new acceleration LoRA (PTD + RefAsymOPD), which cuts steps from 40 down to 6–8 while surprisingly improving quality and realism. You can find the model weights and custom nodes here: **taobao-content-ai/MiniMax-H3-PTD-RefAsymOPD-LoRAs** **Why it stands out** Unlike typical distillation methods that sacrifice quality for speed, RefAsymOPD is optimized for reference-to-video, reducing background blur and AI artifacts. The sharpness and background plausibility at just 6–8 steps are incredible, feeling way less “plastic” than expected. **Available Variants & Usage** * **8-Step:** Max quality and stability. * **6-Step:** Max speed, matching baseline quality. It is a drop-in replacement compatible with both **Diffusers** and **ComfyUI**. If you already have your MiniMax-H3 environment set up, you just need to swap the checkpoint file name. Getting a 5x speedup while maintaining (or even improving) full-quality results feels like pure magic. Highly recommended for open-source devs!

For now, I tried testing a similar reference-audio issue using publicly available materials: * * * ## Short answers to your eight questions I would treat your goals as **two separate conditioning problems** , not as a reason to discard your WebUI or workflow: (a) preserve a recognizable speaker while generating _new, correctly timed dialogue_ without an extra voice, and (b) preserve the _person_ while replacing the original scene. I reproduced a related reference-audio anomaly using another person’s public example, but **not your exact overlapping-voices output**. The distinction matters throughout. **1. How closely should Ref2VA reproduce the speaker?** Using an audio clip to guide the speaker’s timbre and delivery is a documented Ref2VA use case. The official full-reference format distinguishes `reference` (use characteristics without directly copying the signal) from `fully_copy` (reuse the source audio). It does **not** promise a particular numerical voice-similarity score or perfect identity for every newly scripted line. A recognizable voice is a reasonable goal; it is useful to judge **voice resemblance** , **correct words** , **intelligibility** , **speech timing** , and **unwanted additional voices** separately. Success on one is not proof of success on the others. **2. Is the apparent “two speakers at once” behavior known?** There are neighboring reports: ComfyUI #16155 describes unexpected speech and timing errors with an audio reference; MiniMax-H3 discussions #64 and #76 include reports of reference material intruding into dialogue or clipped/gibberish vocalization; MiniMax-H3 #17 concerns voice binding between characters. Those are **related symptoms, not proof of the same bug**. Your “two voices” might mean genuinely simultaneous speakers, an extra utterance, speaker drift, or an echo/mixed track. Without your generated audio I cannot decide which. In a separate test using #16155’s public assets, extra words were recognized in **3/3 seeds with audible reference audio** , versus **0/3** with equally long silence or zeroed audio latents **while retaining the reference block**. This points toward the _encoded reference values_ , not merely the presence of a slot. It neither reproduces your overlapping voices nor identifies a root cause; the controls and limitations are below. **3. Could three LoRAs interfere with the audio?** Yes, potentially: a “visual” adapter may alter shared audiovisual layers, depending on the modules it targets. Your three LoRAs appear to be **sequential patches before the final sample** , not three separate renders. Compare one fixed-seed run with all custom LoRAs off; if the problem clears, restore them individually. My related anomaly occurred **without LoRAs or Heretic** , so those are not prerequisites for _that_ behavior, but they could still contribute in _your_ pipeline. **4. Is strength`1.0` on each LoRA too much; should it be `0.5–0.7`?** Three `1.0` settings are worth testing, but **they do not add up to a meaningful global “3.0 strength.”** Effects depend on target layers and adapter interactions. Nor is `0.5–0.7` a universal safe range. Try **all off → one at a time → necessary combinations** , then sweep strengths only where a difference appears; this preserves your visual goal. **5. Could the pruned INT8 ConvRot checkpoint be responsible?** Still an open variable. My runs used `minimax_h3_ref2va_pruned_int8_convrot.safetensors` and produced both clean ASR-control transcripts and reference-dependent extras: INT8 does not **uniformly** break dialogue, but might affect this failure. **No matched official-BF16 test exists.** With 12 GB VRAM, check the graph and one-seed contrasts first. The model card and native ComfyUI guide document the distinct deployment artifacts; successful loading alone does not establish output parity. **6. Must the audio be attached to both Ref2VA nodes (`400` and `587`)?** **Not necessarily.** `MiniMaxH3ReferenceToVideo` builds conditioning and an initial audio-video latent; it is not a sampler. Your plan to attach audio to **final node`587`** may be sound. Check the **JSON actually submitted to`/prompt` after Gradio rewrites it**: trace both final BasicGuider conditioning **and** sampler latent, and establish what `400` still supplies. Adding audio to `400` without that trace could introduce a new constraint. The native implementation and node docs are useful references. **7. Why does the old background persist despite requesting a basement?** A full reference picture encodes both **the person and the old scene**. If a first-frame/guide anchor survives in the actual graph, it may impose still more scene continuity. Distinguish _identity reference_ from _frame preservation_ : compare the full picture with a **subject-focused crop or background-removed reference** , keeping the target basement/action/seed fixed. You have **already** softened preservation wording, added a single-subject constraint, and switched `max` to `match`; their outcome is not established. I did not reproduce this background issue. **8.`Is ref_image_size="max"` …?** The public post ends at this heading, so I will not invent the missing question. `ref_image_size` controls **image preprocessing** , not a linear reference-strength setting. The official node docs explain that `match` scales references toward the output pixel area, while `max` keeps greater detail (up to the 2048px short-edge reference convention), at potentially much higher token/compute cost. Neither is a dedicated background switch; the result after your `match` change is unknown. ## Suggested default route: preserve the features, isolate the boundary 1. **Freeze what actually runs.** Save one failing run’s final API JSON **after** file uploads, prompt edits, LoRA chaining, and the WebUI’s input rewrites. Follow the final guider’s conditioning and sampler’s latent back through `587`, `400`, and the media loaders. This is mostly a static check and costs no render. Do not assume the original workflow screenshot equals the submitted graph. 2. **Use one paired voice test, not a rebuild.** At the same seed and settings, compare the current audible reference with **all custom LoRAs off**. If the bad voice remains, prioritize reference roles, native Audio VAE conditioning, and model/encoder differences. If it clears, reintroduce adapters one by one. Continue to use the audible reference for the _target_ result: silence/zero latent are diagnostic controls, not voice cloning. 3. **If extra utterances remain, verify the reference contract.** Bind `<Audio N>` to `(S1)`, use `reference` rather than `fully_copy` for new dialogue, keep the sample clean, and specify the exact line. For timing, connect speech to an **on-screen action** as well as a shot interval; absolute timestamps alone do not guarantee onset. A shorter clip is optional, **not a universal fix** : cropping still left extras in 2/3 seeds. 4. **Handle the background independently.** Start from your already-updated `match` setting; compare full-scene versus subject-focused image. Check whether anything actually anchors the first frame or reuses an old visual guide. Score _person identity_ and _new location_ separately, even if the LoRA stack remains unchanged. 5. **Escalate when a cheaper comparison identifies a boundary.** Standard Qwen versus Heretic, alternative checkpoints, matched official BF16, or ComfyUI versus Diffusers can be useful, but each adds downloads, VRAM demand, and possible unrelated numerical differences. They are not prerequisites for the first two checks. The goal remains **new speech in the target voice** and **the same person in a new room**. ## First checks: what to observe before another expensive render Useful optional evidence: one failing **submitted** `/prompt` JSON (private paths removed), a short generated-audio excerpt, the intended line, seed, reference duration, and checkpoint/adapter names. These are not prerequisites for the suggestions below. **No-render audio check:** If the voice sounds doubled, compare **ComfyUI’s raw decoded audio** (if retained) with the **final Gradio/WebUI-exported or muxed audio** from the same run. Check left and right channels separately before downmixing. **Only after export?** Inspect muxing, duplicated tracks, and post-processing first. **Already in the raw audio?** Focus on generation, reference binding, or decoding. If the raw track was not saved, mark this comparison as unavailable instead of inferring an export bug; when both tracks exist, no new inference is needed. Gradio workflow template -> file substitutions / prompt edits / adapter chain -> actual payload sent to POST /prompt -> LoadAudio -> 587.ref_audios.ref_audio_0 -> 587.positive -> final BasicGuider (?) -> 587.LATENT -> final sampler input (?) -> 400 outputs -> which actual consumers, if any? -> final H3 model / guider / sampler -> decode / output The question marks are **edges to check** , not accusations that they are wrong. `ref_audios.ref_audio_0` is a **zero-based API field** , while model text uses the **one-based** label `<Audio 1>`; that apparent mismatch is normal. Other enabled audio-bearing reference media can affect label ordering, so inspect the packed/reference list rather than treating numbering as a universal equivalence. To classify “double voice,” distinguish: **overlapping independent speakers** , **extra speech before/after the line** , **timbre drift** , and **duplicated/reverberant audio**. These suggest different causes; ASR alone cannot distinguish them. ## A related public reproduction, in brief I used the **public image and voice WAV from someone else’s** ComfyUI issue #16155, where the requested line was `This park sucks.` around 2 seconds. This is not a reproduction of your particular reference, output, multi-node graph, or two-simultaneous-voices claim. My Colab setup used native ComfyUI, pruned INT8 H3 Ref2VA, a **standard (non-Heretic) Qwen** , **no LoRAs** , 512×288, 124 frames at 24 fps (~5.17 s), 20 sampling steps, and three deliberately reused seeds (`4242`, `5151`, `6262`). The decisive panel kept the reference block in place while changing the encoded values: Reference input | Extra words recognized beyond requested line | Interpretation ---|---|--- Original audible 7.05 s WAV | **3/3 seeds** , both Whisper models | Related anomaly reproduced in this setup Same-duration **time-reversed** WAV | **2/3** agreed; **1/3** ASR disagreement | Ordinary forward-intelligible English is not clearly required Same-duration **PCM silence, Audio-VAE encoded** | **0/3** , both ASR models | Reference block still exists; encoded silence is not a zero latent Original reference **latent zeroed after encoding** | **0/3** , both ASR models | Same reference block shape, but an artificial out-of-distribution control Cropping the reference still left extras in **2/3** seeds. Two Whisper versions agreed on most observations but are not human listening tests; neither words alone nor waveform activity establish concurrent speakers, literal copying, or restored voice identity. **Interpretation:** audio-derived conditioning values matter in this small panel, while the responsible feature and root cause remain unknown. The expanded controls follow. A. Reproduction protocol, per-condition results, and evidence limits (click for more details) B. How reference labels, sound latents, and timing interact (click for more details) C. Two Ref2VA nodes: trace the final API graph, not the node count (click for more details) D. LoRAs, Heretic Qwen, the INT8 checkpoint, and 12 GB VRAM (click for more details) E. Keeping the person while changing the environment (click for more details) F. Symptom decision tree, further research, and useful links (click for more details)

I am not running the official MiniMax-H3 Diffusers demo directly. I built a custom Gradio WebUI around the ComfyUI MiniMax-H3 implementation and communicate with ComfyUI through its API. The WebUI loads a ComfyUI workflow JSON, modifies the API prompt programmatically, uploads the input/reference files to ComfyUI, and then submits the resulting prompt to the ComfyUI `/prompt` API. The workflow is based on a MiniMax-H3 Ref2VA / Live Wallpaper style workflow and contains two `MiniMaxH3ReferenceToVideo` passes. In my current graph: * first/intermediate `MiniMaxH3ReferenceToVideo`: node `400` * final `MiniMaxH3ReferenceToVideo`: node `587` The final BasicGuider / sampler path consumes the conditioning from the second/final Ref2VA node. I intentionally attach the reference audio only to the final Ref2VA pass. The intermediate pass does not receive reference audio. The relevant API structure generated by my WebUI is effectively: LoadAudio(reference_audio) -> MiniMaxH3ReferenceToVideo.ref_audios.ref_audio_0 For the final Ref2VA node, the API prompt contains the equivalent of: "ref_audios.ref_audio_0": ["<LoadAudio node id>", 0] The first/intermediate `MiniMaxH3ReferenceToVideo` node does not have `ref_audios.ref_audio_0`. This uses ComfyUI’s native MiniMax-H3 reference-audio implementation rather than manually generating audio embeddings in my own WebUI. ComfyUI internally performs approximately: waveform = audio["waveform"] sr = audio["sample_rate"] if sr != audio_vae.audio_sample_rate: waveform = comfy.audio.resample( waveform, sr, audio_vae.audio_sample_rate ) z = audio_vae.encode( waveform[:1].movedim(1, -1) ) and then puts the resulting audio latent into MiniMax-H3 conditioning as a reference block. So the reference audio is not merely mentioned in the text prompt. It is actually passed through `ref_audios` and encoded by the MiniMax-H3 Audio VAE. * * * ## Models used ### Main Ref2VA transformer minimax_h3_ref2va_pruned_int8_convrot.safetensors Loaded as the MiniMax-H3 Ref2VA diffusion model in ComfyUI. ### Text / multimodal encoder qwen3vl_32b_heretic_minimax_h3_nvfp4.safetensors ### Audio VAE minimax_h3_audio_vae_fp32.safetensors This is loaded through ComfyUI’s VAE loader and is used for both MiniMax-H3 audio handling and reference-audio encoding. ### Video VAE minimax_h3_video_vae_fp16.safetensors The Audio VAE is the ComfyUI-formatted MiniMax-H3 Audio VAE, not a separate model from `models/audio_encoders`. My `models/audio_encoders` directory is essentially empty except for the placeholder file. MiniMax-H3 reference audio is going through the Audio VAE path. * * * ## LoRAs used in the problematic generation I used three custom LoRAs simultaneously: AfterMidnight_ref2va_h3_sexytime_rank64_v1.1.safetensors AfterMidnight_ref2va_h3_softer_rank64_v1.safetensors Astro%20nsfw.safetensors Strength: 1.0 1.0 1.0 The WebUI inserts these sequentially into the model chain before sampling rather than generating separate outputs and merging them afterward. In other words, I am not doing an ensemble of three separately rendered videos. The model is effectively passed through the LoRA chain and then sampled once. The original Live Wallpaper workflow also contains its own internal workflow/model modification path, so this is not a completely minimal official Ref2VA graph. * * * ## Custom WebUI behavior The WebUI performs the following operations before submitting the ComfyUI API prompt: 1. Loads the original workflow. 2. Converts it into an API prompt. 3. Replaces input image/video/audio nodes with uploaded files. 4. Appends the user’s requested edit/action to the MiniMax-H3 prompt. 5. Applies selected custom LoRAs sequentially. 6. Disables the WebUI’s automatic prompt rewriting where possible. 7. Locates the final `MiniMaxH3ReferenceToVideo` node. 8. Removes reference-audio inputs from intermediate Ref2VA nodes. 9. Connects the uploaded reference audio only to the final Ref2VA node using: ref_audios.ref_audio_0 10. Submits the graph to ComfyUI through its normal API. The WebUI itself does not synthesize the voice. Actual audio generation is still performed by MiniMax-H3 / ComfyUI. * * * # Problem 1: reference voice is very abnormal The reference audio contains one speaker. However, the generated result often sounds as if **two voices / two speakers are speaking simultaneously**. It is difficult to describe exactly, but it sounds like overlapping vocal identities rather than one clean speaker. The generated voice is also substantially different from the voice in the supplied reference audio. This is not a small pitch difference. The speaker identity itself feels different. At times the generated audio is also difficult to understand and resembles speech-like vocalization rather than clean intelligible dialogue. The strange part is that the Audio VAE clearly appears to be functioning: * audio is generated, * it is not completely silent, * it is not simply white noise, * it often has speech-like prosody, but speaker identity and intelligibility can be severely wrong. This makes me suspect one of the following: * reference-audio conditioning problem, * interaction between Ref2VA and the INT8 ConvRot transformer, * incorrect audio-latent alignment, * Audio VAE preprocessing issue, * interference from stacked LoRAs, * or interference from the two-pass workflow. * * * # Problem 2: scene/environment instructions are mostly ignored I explicitly requested a different environment. For example, I specified: basement as the intended setting/background. However, the generated result did not meaningfully create a basement environment. Instead, the model mostly preserved the background of the source/reference image. The result looked much closer to: > animate the existing photograph for approximately 10 seconds than: > keep the person identity but regenerate the environment according to the requested scene. The character may move, but the environment prompt is effectively ignored. This also happened with other visual editing instructions. Some simple pose/action instructions can work, but larger scene or appearance changes are much less reliable. * * * ## Prompt preservation logic Earlier versions of my WebUI used a strong preservation instruction similar to: fully preserve identity, face, body appearance, clothing, and visual continuity I suspected this could directly conflict with scene/edit instructions. I have since changed this behavior. The current version is intended to preserve primarily: * face, * identity, * subject continuity, while allowing the prompt to modify: * clothing, * shoes, * pose, * body position, * environment, * background. I also added an explicit instruction to keep exactly one instance of the subject to reduce duplicate-person generation. Despite this, large environmental changes still appear weak compared with preservation of the source image. * * * ## Reference-image strength The original workflow used a very strong reference-image setting. I previously used: ref_image_size = max I have changed the final Ref2VA pass to: ref_image_size = match because I suspected `max` was causing excessive visual anchoring to the source image. The scene-following problem still needs further testing after this change. * * * ## Audio VAE memory modification I tested Because my GPU has 12 GB VRAM, I previously experimented with manually unloading the Audio VAE immediately after reference-audio encoding. I tried calling the ComfyUI model patcher’s unload path after: audio_vae.encode(...) This proved too aggressive, so I removed that modification. The current version does **not** manually call: patcher.unpatch_model(...) after reference-audio encoding. I now leave model/offload management to ComfyUI. So the abnormal audio described above should not be interpreted as being intentionally decoded through a separately modified Audio VAE implementation. * * * ## Hardware GPU VRAM: 12 GB The system has substantially more normal RAM available. Because of the 12 GB VRAM limit, I am using the pruned INT8 ConvRot Ref2VA transformer instead of the full original BF16 transformer. This is also why I am interested in whether the quantized/pruned Ref2VA model may behave differently from the original official transformer for reference audio. * * * # Questions I would appreciate clarification on the following. ### 1. How closely should Ref2VA preserve speaker identity? When `ref_audios.ref_audio_0` is supplied, should the generated speech normally sound recognizably like the supplied speaker? Or is the reference audio intended mainly for prosody/style rather than speaker identity? ### 2. Has anyone seen a “two speakers at once” effect? Is there a known failure mode where reference-audio conditioning produces something that sounds like two overlapping speakers or mixed voice identities? ### 3. Could stacked LoRAs cause this? Could these three LoRAs at strength `1.0` each: AfterMidnight_ref2va_h3_sexytime_rank64_v1.1 AfterMidnight_ref2va_h3_softer_rank64_v1 Astro interfere with MiniMax-H3 audio/reference conditioning? In particular, can a visual/video LoRA modify enough of the transformer to damage speaker conditioning or audio generation? ### 4. Is strength 1.0 + 1.0 + 1.0 excessive for Ref2VA? Would a lower value such as approximately 0.5-0.7 per LoRA generally be safer when stacking several MiniMax-H3 LoRAs? ### 5. Could the pruned INT8 ConvRot Ref2VA checkpoint be responsible? I am currently using: minimax_h3_ref2va_pruned_int8_convrot.safetensors rather than the original large official Ref2VA transformer shards. Could pruning or INT8 ConvRot quantization noticeably degrade the `audio_patch_proj` / reference-audio path or audio-video conditioning? ### 6. Does MiniMax-H3 require reference audio on both Ref2VA passes? My workflow contains two `MiniMaxH3ReferenceToVideo` nodes. Currently: intermediate pass: no reference audio final pass: reference audio enabled Is this valid? Or should reference audio be supplied to every Ref2VA conditioning stage in a two-pass workflow? ### 7. Why might environment instructions be ignored? Is Ref2VA expected to strongly preserve the entire source-image background? If I want: same person / same identity different environment is there a recommended reference strength, `ref_image_size`, conditioning strategy, or workflow structure? ### 8. Is `ref_image_size="max"`