SCAIL Auto Prompt Builder

The whole point of SCAIL-2 is that you don't fight a pose skeleton - you drop in a reference image, a driving video, and a well-written prompt, and it does the character replacement. The catch is the prompt. SCAIL-2 lives on a Wan 2.1 base and wants long prose that says "replace only this person, keep everything else from the video exactly as it is," and hand-writing that for every clip is where the blank-page problem shows up. SCAILAutoPromptBuilder is a small local VLM wrapper that writes it for you.

What it actually does

Wire in your reference image and the sampled frames of your source video, hit Queue, and this node samples the first, middle, and last frame from the video batch, resizes everything to max_side, and feeds the whole stack to a local vision-language model with a strict instruction: identify the reference subject's stable identity, identify what's in the source video, and write a single clean diffusion prompt that replaces only the main subject while preserving clothing, background, pose, lighting, camera movement, and any non-target people. It also bakes in the continuity language you'd forget - no identity drift, no extra limbs, no face transfer to background people.

No API, no key, no uploads. It loads the VLM with local_files_only=True and never phones home. That's the "auto" in the name, and it's the reason this is worth a look over pasting your clip into a chat model.

The model it needs (this is the real install step)

The node itself installs trivially - ComfyUI Manager, search "SCAIL Auto Prompt Builder", or:

cd ComfyUI/custom_nodes
git clone https://github.com/klimentiy23/scail-auto-prompt-builder

then restart ComfyUI. Its dependencies are torch, transformers, Pillow, numpy - all things a modern portable install already has. The actual setup cost is the model: it expects a Qwen3-VL-2B-Instruct (or similar vision LLM) sitting under D:/comfyui/models/LLM/Qwen-VL/Qwen3-VL-2B-Instruct. Note the D:/ - that's the author's Windows layout baked into the default. On any other machine, set the root explicitly:

COMFYUI_MODELS_DIR=/path/to/your/comfyui/models

The node scans <root>/LLM for anything with a vision token in its config, so drop the model folder in there and model_folder will auto-detect it.

Inputs that matter

• reference_image - the subject's identity. This is appearance only; the node explicitly tells the VLM not to copy the reference's background or pose.
• video_frames - the sampled source frames, usually straight from LoadVideo. Resize the video first if you can; it's what max_side is for.
• device - defaults to cpu. CUDA is roughly 16 seconds versus 1–2 minutes for the first CPU pass, but keep unload_after=True either way so the VLM is released before Wan/SCAIL grabs the VRAM.
• user_hint - the one thing worth typing. "Replace only the woman in the black cowboy outfit," "keep the hat and handbag from the video." Empty is fine for a first pass.
• fail_mode - if the VLM returns garbage, fallback_template (default) hands you a sane generic SCAIL prompt; raise_error fails loudly instead.

Two outputs: prompt (STRING - feed it to CLIPTextEncode positive) and diagnostics, which tells you which model ran, on what device, and whether the VLM call succeeded or fell back.

Where people get burned

The biggest trap is the missing model: no download happens, ever. If you don't have a VLM at the LLM path, you get a FileNotFoundError or silently the fallback template, and the diagnostics string is where you'll see it. Preview the generated prompt before a long render - a 2B VLM on a multi-person clip will occasionally misattribute who's wearing what, which is the known weakness of every small captioner (llm-in-comfyui). And remember what this node is: a text writer, not the masker or the chunker. The actual SCAIL-2 long-video work still needs the companion SCAIL Auto Extend node, and SCAIL-2's 81-frame native window and cross-chunk identity drift are on you to manage. For a one-button replacement baseline it's genuinely the easiest way to get a solid prompt - just don't expect it to fix the model's face drift.