Create H3 RefMod
Compress a character reference into a few hundred tokens
- refs_image
- refs_video
- refs_bundle
- mask
- av_encoder
- vae
- mods
This is the node people actually mean when they say the pack is "LoRAs without training." MiniMax H3 is genuinely good at reference conditioning - the community reaction to its motion control was "like Kling" - but every native reference video injects thousands of tokens into the packed sequence, which is why people started asking whether you could compress that. Extract H3 RefMod turns one or more stills or clips of a concept into a RefMod: a small .safetensors containing a latent that still rides H3's native ref path, at a fraction of the compute.
And here's the part that surprises people: there is no diffusion model involved and no training. Extraction only needs the H3 video VAE. The heaviest thing in this workflow is the encode.
How it works
Plug stills into the autogrow ref_image_1, video frames into ref_video_1 (a + button adds more slots of each type). Each ref is a view of the same concept. Refs get stacked along the latent time axis into one video-kind mod, so a multi-image moodboard keeps each image's own content instead of averaging it into mush.
Then pick a mode:
training(default) - resize toref_resolution, VAE-encode, then average-pool to a small grid (pool_h×pool_w, auto-fit to the source's aspect ratio so a portrait isn't squished square) and refine the small latent against the full encode with gradient steps. 16×16 = 64 tokens per frame, which is the pack's "concept sweet spot"; 32×32 = 256; 64×64 = 1024 (full-mode parity).encode- store the straight full-res encode atref_resolution. Max identity, but an MB-sized mod at ~1K tokens per 1024px image.
The identity dial is refinement steps, not a lesson in who the person is - the docstring is explicit that it doesn't teach the model which attribute to keep. 500 is the default, 0 is pure pooling. Small grids lose faces and fine detail; that's the whole trade.
Inputs a beginner actually sets
Honestly: name, mode, ref_resolution, the pool dials, identity, and max_tokens. max_tokens (default 5120) is your safety valve - if stacked refs overflow it, near-duplicate latent frames get dropped first, then frames are resampled. If you know you'll only ever use one concept per file, concept_type and description are optional metadata that also feed the loaders' prompt_hint output. Connect a subject mask with background_retention when encode mode keeps pulling in background you don't own. merge (training only) refines one consensus latent against every ref - good for a folder of angles; motion_only is an experiment for capturing motion without appearance.
Wire the mods output straight into Apply H3 RefMod for an instant test, or set save and pick the file back up later with Load H3 RefMods (the name shows in its dropdown after a reload). Save goes to the first registered refmods root - models/refmods by default, or wherever you've mapped it in extra_model_paths.yaml - with an optional subfolder.
Install
ComfyUI Manager (search "ComfyUI-MiniMaxH3Mod") or:
cd ComfyUI/custom_nodes
git clone https://github.com/Luisacaotica/ComfyUI-MiniMaxH3Mod
Restart. Deps are light (safetensors, numpy, Pillow). You need the H3 weights and its video VAE already working in a current ComfyUI - the pack explicitly doesn't import ComfyUI-MiniMaxH3 - and H3's licence is territory-restricted (US, EU, UK, Korea excluded).
Common issues
Don't expect magic from a small grid: the README calls the "8×8 concept / 16×16 identity" sweet spot folklore, not a validated split. If a face refuses to come through, go bigger on the grid or switch to encode - pooling is lossy in exactly the way that destroys facial identity. Fast motion is the pack's known hard case: a burst of hand signs smears into slow softness, so extract only the moments that matter and describe the motion in your prompt. And remember extraction and voice are separate: appearance lives here, audio lives in Extract H3 Audio RefMod or the Master node.
Inputs (24)
| Name | Type | Default | Description |
|---|---|---|---|
| name | STRING | my_concept | Saved mod name (appears in the Load H3 RefMods dropdown after a reload). |
| mode | COMBO | Compressed Reference | Compressed Reference pools the latent and optionally refines its reconstruction. Full Reference stores the VAE encode, subject to resolution/frame/token limits. Neither mode trains H3 weights. Legacy mode values remain accepted. |
| concept_type | COMBO | generic | Metadata only; does not select a learning algorithm. What this mod represents — 'identity' (a specific person/character), 'pose_motion' (a pose/dance/gesture/camera move), 'clothing', 'background', 'style', or 'generic'. Stored in the mod and used by the loaders' prompt_hint output (merges concept_type + description into a string you can concat onto your CLIP prompt). 'identity' in training mode with a small grid also triggers a warning nudging you toward 'encode' mode or a bigger grid — pooling is lossy in exactly the way that destroys facial identity. |
| background_retention | FLOAT | 0.000–1 | Only used when 'mask' is connected. Floor weight for the region outside the mask: 0 = that region collapses to a heavily blurred copy of itself (kills specific structure like a skyline/treeline while staying smooth and in-distribution), 1 = mask has no effect. Middle values (0.3-0.6) partially blur instead of fully. |
| ref_resolution | INT | 1024256–2048 | Target short edge in px (downscale only, never upscale), applied to BOTH modes: 'encode' stores at that res, 'training' encodes smaller too (it pools to a grid anyway, so native-res encoding is wasted compute — this is the main speed dial for training mode). 1024 is a good default; 512 halves encode cost; 2048 = official max fidelity, 4x the tokens of 1024. |
| pool_h | INT | 162–64 | Pooled mode: spatial latent grid after pooling. The grid is auto-fit to the source's aspect ratio (long edge = max of the two dials, other edge derived), so a portrait person isn't squished into a square grid (the 'fat/chubby' distortion). Square sources keep the exact dial value. 16x16 = 64 tokens/frame (concept sweet spot); 32x32 = 256; 64x64 = 1024, full-mode parity for identity. |
| pool_w | INT | 162–64 | Pooled mode: grid width (long edge if the source is wider than tall). |
| latent_frames | INT | 161–2147483647 | Per-video temporal limit. Encode mode samples up to this many source frames before VAE encoding and causal 4k+1 trimming; training mode pools to up to this many latent frames after encoding. Set at least the source frame count to avoid encode-mode sampling. Images use 1. Higher values increase memory and token cost; max_tokens can still reduce the result. |
| identity | INT | 5000–2000 | Compressed Reference only: optimization steps to reduce latent reconstruction error. 0 uses pooling alone. This is not identity strength or model training. |
| merge | BOOLEAN | false | Merge mode (training only): instead of stacking each ref's own pooled latent, optimize ONE shared grid against every full encode jointly — the result lands on what's COMMON across all the views (structure, motion, identity) rather than any single shot's framing/background. Ideal for a collection: many angles of a subject, a folder of similar clips -> one tiny consensus mod, one ref block's worth of tokens. Keeps every full encode in VRAM during refinement. Off = stack (each ref keeps its own frames). Ignored when mode='encode'. |
| motion_only | BOOLEAN | false | EXPERIMENTAL — extract only what MOVES. Video refs are converted to per-frame temporal differences (|f[t+1] - f[t]|, normalized) before encoding, so the mod carries where/how things move and the static appearance (background, the lineart look, an outfit) never enters the latent. For a lineart animation this keeps the moving lines and drops the static drawing. Needs video refs — a still has no motion, so image refs keep their appearance (warned). Training mode only; the ref channel is content-based, so treat the result as a soft motion guide, not a ControlNet. Combines with 'merge'. |
| multiplier | INT | 11–10 | Data multiplier: repeat the extracted ref N times along time so a short video/GIF (few tokens) isn't drowned out by the main video's tokens. Each repeat duplicates the same latent frames, so attention weight on the ref scales roughly with N. 1 = no repeat; file size grows with N. |
| max_tokens | INT | 51200–2147483647 | Hard cap on the total tokens the mod injects (0 = no cap; 5120 is a good performance default). If the stacked refs exceed it, near-duplicate latent frames are dropped first (video refs are full of frames that differ only by noise — each one still costs a token per spatial patch in every block), then frames are resampled to fit. The cap is honored after the multiplier. Lower latent_frames/ref_resolution instead to avoid wasting encode work: ~23K tokens = one 1024px encode-mode video ref at 16 frames. |
| description | STRING | Optional text describing the concept (e.g. 'a ginger woman with messy hair', 'an animation style', 'handheld camera movement'). Stored in the mod and printed in the info block — documentation only, no wiring. | |
| save | BOOLEAN | true | Save the mod to mods/ so Load H3 RefMods can pick it up later. |
| refs_imageopt | COMFY_AUTOGROW_V3 | — | |
| refs_videoopt | COMFY_AUTOGROW_V3 | — | |
| refs_bundleopt | H3_REF_LIST | All images/videos from a Load H3 RefMod Folder node, appended after the autogrow refs (bulk extraction). | |
| maskopt | MASK | Subject mask (or a batch, one per reference in order: images then videos) marking what to keep at full weight. Everything outside the mask collapses toward a heavily blurred copy of itself per spatial cell (stays in-distribution — a flat noise-mix here decodes as a woven/static texture instead of 'nothing'), controlled by background_retention. Fixes 'encode' mode pulling in a background/style that doesn't belong to the subject. A single mask broadcasts to every reference; a batch must match the reference count. | |
| av_encoderopt | MINIMAX_H3_AV_ENCODER | Legacy MiniMax-H3 VAERef. Loads its video checkpoint using the native ComfyUI VAE; prefer vae to share an already loaded VAE. | |
| vaeopt | VAE | Standard VAE, used when av_encoder is not connected. | |
| extraction_presetopt | COMBO | manual | manual preserves controls. identity_encode: Full Reference, resolution=1024, steps=0, merge/motion_only off. style_experimental: Compressed Reference, pool=8x8, steps=150, merge/motion_only off. motion_sequence: Compressed Reference, pool=16x16, merge/motion_only off; preserves frame limit and Refinement Steps. It keeps appearance, not frame differences. |
| subfolderopt | STRING | Optional folder inside models/refmods, for example celebs or voices. | |
| budget_policyopt | COMBO | truncate | On max_tokens overflow: truncate uses the existing frame reduction; error stops without saving. 0 max_tokens disables the cap. |
Outputs (1)
| Name | Type | Description |
|---|---|---|
| mods | H3_REF_MODS | Bundle with this one mod at strength 1.0. Feed it to Apply H3 RefMod (or Load H3 RefMods after saving). |