Nodes/IronLeo78 MTV/IronLeo78 MTV Omni (v0.17.8)
ComfyUI Node

IronLeo78 MTV Omni (v0.17.8)

Minimax MTV Omni: loader + MiniMax H3 reference-to-video + sampling + decode + Create Video, in one node. v0.0.5: the node does not save the clip any more. Outputs: video (Create Video's result, settings hardcoded: 24 fps, sRGB, h264) -> link it to a Save Video node; filename (the clip's name as always named, <Song>/<Song>_seg004_A_00001_, no extension: it names the .json, the manifest row and the latent files; if you link it to a Save Video's filename_prefix, Save Video adds its own _00001_ after it). Models: FastH3 OmniRef (8-step distilled MiniMax H3) + optional custom LoRAs; sigma shift 12 / 3, attention backend + VSA, Qwen3-VL text encoder and the 2 VAEs are hardcoded (v2.3: the optional 'models' input takes a Model Loader node that can replace them, and the step counts). Media: video (segment by segment) + up to nine reference images (the Pictures input) + custom audio (whole song, or only its Vocals / Guitars / Drums / Bass: Audio Stem; the stems are separated once per song and saved to output/<Song>/stems/). Sampling is hardcoded (BasicGuider, euler, simple, denoise 1.0); 8 steps (euler / simple), the FastH3 OmniRef recipe. The noise seed (default 1978) is set on the Model Loader node. Choose Audio latent: Custom Audio / Audio From Video keep the audio you give as is (the model never generates audio); MiniMax Audio works like the official ref2v template (the model generates the sound). Save Latent ON: the sampled latent (video + audio .latent files) and its conditioning (.conditioning) are saved in the song folder with the clip's name, for a later refine / upscale in another workflow. OFF (default): no latent files. Files are tagged <Song>_seg<segment>_A (Song = the Custom Audio File name, if one is selected) and logged in a manifest CSV. No video preview on the node (the live latent preview while sampling stays). v0.0.9: no status box on the node either; the run report leaves on the third output, status (STRING): link it to a Preview Any / text node to read it. v2.5: segment panel (count / next / Reset), progress in the report, a quiet stop after the last segment, and a <clip>.json sidecar with the full prompt next to every clip. v2.6: jump to any segment (number field + Go / Latest in the panel), and the Guide can read the previous segment's last frame back from its clip. v2.8: Scene 2 / prompt_2 / Picture4-6 are gone (one clip per Run). Pictures (optional input, since v0.1.20 the only way to give reference pictures): the IronLeo78-MTV-Omni-Prompt-Generator's nine slots on one wire; every filled slot is sent as <Picture n>. by IronLeo78

By ironleo78-sys·Created 6 days ago·Updated about 4 hours ago· 3
IronLeo78 MTV Omni (v0.17.8)
  • models
  • Pictures
  • video
  • filename
  • status
◄Use Videotrue►
◄Video FileNone►
◄Choose Audio latentCustom Audio►
◄Custom Audio File▾►
◄Audio StemFull►
◄megapixels0.2►
◄duration5.0►
◄Next Segment On Runfalse►
◄prompt►
◄Guidefalse►
◄Save Latentfalse►
◄Latent overlapfalse►
◄Overlap (s)2.0►
◄LTX Continuefalse►
CategoryIronLeo78/MTV Omni

Inputs (16)

NameTypeDefaultDescription
Use VideoBOOLEANtrueON = v2v: the video is loaded and is the reference for megapixels/width/height. OFF = i2v: no frames are loaded, the Picture_1 input is the reference. Exception: with Choose Audio latent = Audio From Video, a selected Video File still supplies its soundtrack (audio only). Set Video File to None for silence.
Video FileCOMBONone1 options: None
Choose Audio latentCOMBOCustom AudioWhere the sound of the clip comes from. MiniMax Audio = like the official MiniMax H3 ref2v template: no bypass, the model's own audio latent goes to the sampler and the model generates the sound itself (voice, effects, music). A Custom Audio File / video soundtrack, if any, is then only sent as a reference (use <Audio 1> in the prompt). Audio From Video = the video soundtrack is encoded into the audio latent and frozen (silence if none). Custom Audio = the Custom Audio File is encoded into the audio latent and frozen (silence if none): the model never invents sound, the picture follows your music. Custom Audio also decides which audio Next Segment On Run moves through: the song is cut into chunks as long as the real video segment (frames / fps: duration 5 s = 124 frames = 5.17 s), one chunk per Run (works with Use Video OFF too); the segment counter at the top of the node decides which chunk is next (Jump).
Custom Audio FileCOMBO1 options: None
Audio StemCOMBOFullWhich part of the Custom Audio File is used. Full = the whole mix. Vocals / Guitars / Drums / Bass = only that stem (Guitars is Demucs' 'other' stem: guitars + keys + synths). The first time a stem is chosen for a NEW song, Hybrid Demucs separates all four (about 13 s for a 2 minute song) and saves them to output/<Song>/stems/<Song>_<Stem>.wav. After that every Run just reads the file. Needs a Custom Audio File.
megapixelsFLOAT0.20.1–1Target size. Use Video ON: the video sets the aspect ratio and width/height. Use Video OFF: Picture_1 does. The other pictures always keep their own aspect ratio.
durationFLOAT5.00–10000Duration in seconds. Also sets the video segment length when Use Video is ON. durationS = this value unchanged; durationF = Minimax frame count.
Next Segment On RunBOOLEANfalseON: every Run moves on to the next segment (video frames, and/or the custom audio chunk when Choose Audio latent = Custom Audio). OFF: the same segment on every Run: the first one, or the one you chose with Go in the panel (the counter does not move).
promptSTRINGMTV_Prompt: the prompt, e.g. the output of the IronLeo78-MTV-Omni-Prompt-Generator. Refer to references as <Picture i> (the connected Picture_n inputs) / <Video 1> / <Audio j>. Numbers count only what is actually sent: an empty picture slot is skipped (Picture_1 + Picture_3 connected = <Picture 1> and <Picture 2>). Audio order: video soundtrack first (if any), then custom audio. The status output lists the exact mapping.
GuideBOOLEANfalseOFF by default (v0.6.2). Only works with Next Segment On Run = ON. ON: the last frame of the previous run is used as a guide for the first frame of this segment, so the clips continue each other. The first segment (or a run with no saved frame from the segment before) runs without a guide. OFF: no guide.
Save LatentBOOLEANfalseOFF by default (v0.6.2). ON: the sampled latent (video + audio .latent files) and the conditioning (.conditioning: prompt + pictures, no guide) are saved in the song folder, named like the clip (see the filename output), so you can refine / upscale it later in another workflow. OFF: no latent files. (The clip itself is saved by the Save Video node you link.)
Latent overlapBOOLEANfalseEXPERIMENTAL, OFF by default. ON: the last seconds of the loaded video (Use Video ON) are pinned inside the sampled latent (latent noise mask), so the new clip really continues them: same pixels, motion and speed across the cut. The clip is sampled with the overlap in front and the overlap is cut off again, so each clip costs about 40 % more sampling. Needs Use Video ON and Choose Audio latent = Custom Audio with a song (the song then also covers the overlap). Replaces Guide for this clip; Save Latent is skipped (the saved latent would not match the clip).
Overlap (s)FLOAT2.01–4Latent overlap: how many seconds at the end of the loaded video are pinned (rounded up to the model's frame rule). 2 s is enough.
LTX ContinueBOOLEANfalseOFF by default. ON: after the clip is made, LTX 2.5 extends it by about 20 s in the same Run (whole segments, so the next Run starts where the extension ends). The last 49 frames of the clip start the LTX pass, the Huihui LLM writes the LTX prompt from your prompt and the last frame. A song (Custom Audio) or a video soundtrack (Audio From Video) goes through RAW for the whole clip and also guides LTX; with MiniMax Audio, H3 and LTX each keep their own sound. Needs the LTX files of the IronLeo78 LTV node. If the extension fails the H3 clip is kept.
modelsoptIRONLEO78_MTV_MODELSOptional. Plug in the IronLeo78 MTV Omni Model Loader to replace the hardcoded diffusion model, custom LoRAs, text encoder, VAEs and step counts. Nothing connected = the hardcoded defaults.
PicturesoptIRONLEO78_PICTURESOptional. All nine reference pictures on one wire: connect the IronLeo78-MTV-Omni-Prompt-Generator's Pictures output. Empty slots are skipped (the <Picture n> numbers count only the filled ones).

Outputs (3)

NameTypeDescription
videoVIDEO—
filenameSTRING—
statusSTRING—