JoyAI_Echo_SM_Encoder

Here's the thing that trips people up the first time: the text box on JoyAI_Echo_SM_Encoder is not a positive prompt. It's a shot list. One line per shot, in story order, and each line gets encoded separately and fed into the generator as its own segment of a multi-shot video. Think of this node as where you write the script - the KSampler later is where it becomes film.

JoyAI-Echo is a "minute-level multi-shot" model: it's built to take a sequence of scene prompts and hold character identity and voice across the whole thing via a paired audio-visual memory bank. None of that works if the conditioning is one giant blob of text, which is exactly why the encoder exists as its own node rather than being glued into the KSampler. You give it a structured list, it gives you structured conditioning.

How it works

infer_joyai_text runs each line of your prompt through the Gemma wrapper the Clip node loaded, producing paired video-context and audio-context embeddings per shot, then saves the whole thing to ComfyUI/output/joy_echo_te_cond.pt. Then it releases the text encoder - the engine is deliberately two-stage so the 24 GB-class Gemma isn't hogging VRAM while the generator runs. That release is why your KSampler can sit there waiting for the conditioning: by the time it runs, the encoder is gone.

Two prompt sources, either works:

• prompt (multiline) - type one shot per line. The node splits on newlines and wraps them in a temp JSON ({"prompts": [...]}).
• prompt_files - path to a JSON file with the same {"prompts": [...]} structure. A later update added a "Select json file upload" button right on the node, so you can pick the file in the UI instead of typing a path. Useful when your shots are long or come out of an LLM.

The other two inputs are the memory knobs: prefetch_count (0–64, default 1) controls how many encoder layers get prefetched during streaming, and enable_streaming (bool, default false) turns on layer streaming for the encoder. Both exist to make the 12B Gemma fit on smaller cards; if you're not OOMing on encoding, leave them alone.

Output is a single te_cond (CONDITIONING) that wires into JoyAI_Echo_SM_KSampler.

Install

Nothing special beyond the pack itself - same clone-into-custom_nodes or Manager install as the rest of ComfyUI_JoyAI_Echo. But note the encoder has no model inputs of its own: it needs a clip from JoyAI_Echo_SM_Clip, which means your Gemma + connector files have to be in place (see that node's article) before this node can do anything.

Common issues

• No prompt or prompt_files - you left both empty. Type at least one line, or pick a JSON. Straightforward error, easy to hit on a fresh workflow where you expected a default prompt to exist.
• Run-order errors. This is a genuinely sequential graph: the encoder must run before the KSampler, and early users hit exactly this - the KSampler running first, or the encoder erroring because the left-hand node hadn't run. Wire te_cond and let the queue handle order, but don't try to skip the encoder.
• A stale prompt_files path that still errors after you delete it. Reported by early adopters: the path text sticks around even when you've switched to typing in prompt. If you get a file-not-found on a workflow where you're only using the text box, clear the prompt_files field out properly (or use the file-picker button, which writes the path itself).
• KSampler runs without the encoder - it'll quietly load the last joy_echo_te_cond.pt from ComfyUI/output instead. If you're seeing old story conditioning, that's why: the KSampler falls back to the saved file when te_cond isn't wired.
• Encoding feels slow. You're pushing a 12B encoder through a whole script. That's normal; it's also one pass, and the actual generation time dwarfs it. If it's very slow, the Clip node is probably set to infer_device: cpu.