HY-Motion Encode Text

Before HY-Motion can animate anything, it has to understand your sentence. That's this node's entire job - and it does it twice, with two different models, because Tencent's HY-Motion 1.0 architecture was trained to take conditioning from both a classic CLIP encoder and a modern LLM. You give it an llm and a text; it hands back a single conditioning blob that Generate consumes.

Here's the mechanism, because it explains both the setup and the VRAM pain. The node runs your text through CLIP (clip-vit-large-patch14) to get the 768-dim "visual text" embedding, then through the Qwen3 LLM you loaded - wrapped in HY-Motion's own chat template, with the model's hidden states pulled out rather than a generated reply - to get the 4096-dim context embedding the Motion DiT expects. Two encoders, two embeddings, one condition. If the LLM's hidden size doesn't match 4096, the node inserts a learned linear layer to convert it, which is its way of tolerating smaller Qwen3 variants without crashing.

The upshot: the only input you actually touch is the text box (multiline, defaults to "A person is walking forward."). Feed it whatever motion you want - "a person is walking forward" is fine, and Chinese works too, since the Qwen encoder reads it natively. The llm input is just the wire from whichever LLM loader you chose.

The parts that bite

The CLIP model loads itself lazily on first run, but the README's "downloads automatically on first use" is optimistic. The shipped code loads it with local_files_only=True and searches ComfyUI/models/HY-Motion/ckpts/clip-vit-large-patch14 (or ComfyUI's text_encoders folder). If it isn't there, you get a FileNotFoundError telling you exactly where to put it. Plan to fetch it yourself.

Second, this node is where the pack's memory reputation comes from. The Qwen3-8B encoder is ~16GB at full precision - bigger than the diffusion network it's feeding. That's why the pack exists with quantized loaders: int8 (~8GB), int4/bnb-4bit/AWQ (~4GB), and GGUF (Q4_K_M ~5GB). On a 12GB card, full Qwen is not happening; the quantized options are the entire point. This is also why the README's CPU offload advice exists - you can run the prompter on CPU and keep the encoder on GPU, or vice versa.

Workflow position

Wire it after a Load LLM node, feed it either your own text or the rewritten_text from HY-Motion Rewrite Prompt, and run its conditioning output into HY-Motion Generate alongside a loaded network. That's the whole graph. If your generated motions look like they're ignoring the prompt, the first thing to check isn't this node - it's whether you're using a heavily quantized encoder and whether cfg on Generate is doing anything. For most people, int4-to-AWQ quality on the encoder is a non-issue and the memory savings are enormous; Q8-vs-fp16 debates barely matter for a motion skeleton the way they do for image prompts.