Nodes/Eric's Prompt Enhancers for ComfyUI/Image-to-Video Prompt Expander
ComfyUI Node

Image-to-Video Prompt Expander

Turn a still into a video prompt the model can actually follow

By EricRollei·Created 11 months ago·Updated 8 months ago· 45
Image-to-Video Prompt Expander
  • image
  • positive_prompt
  • negative_prompt
  • image_description
  • status
motion_descriptionslowly turns head and smiles at camera
presetcinematic
expansion_tierenhanced
reference_moderecreate_exact
use_vision_modeltrue
vision_backendlm_studio
vision_endpointhttp://localhost:1234/v1
expansion_backendlm_studio
expansion_endpointhttp://localhost:1234/v1
temperature0.7
camera_movementauto
shot_sizeauto
motion_speedauto
lighting_typeauto
time_of_daymaintain current
positive_keywords
negative_keywords
save_to_filefalse
filename_baseimg2vid_prompt

Image-to-video is the workflow most local video people actually run - anchor the first frame to a real image and the consistency problems of pure text-to-video mostly evaporate. The catch is that I2V models still need a prompt that describes what's in the frame and what should move. This node does both halves for you: a vision model reads your starting image, then a second LLM pass turns that description plus your motion direction into a prompt the video model can follow.

It's the pack's dedicated I2V node (Eric's Prompt Enhancers), and it's a genuinely nice fit for the Wan 2.2 and LTX workflows people are running - those models reward detailed, physical descriptions, and punishing short prompts is a known failure mode in the ecosystem.

How it works

Two stages, two LLM calls. First the vision pass: your image gets captioned by a vision-capable model - either a VL model loaded in LM Studio or Ollama (vision_backend), or the pack's local Qwen3-VL backend. Then the expansion pass takes that caption, your motion_description, and the aesthetic dropdowns, and writes a video prompt that keeps the subject recognizable while adding camera work and movement.

The one setting that changes the whole character of the output is reference_mode:

  • recreate_exact - keep the exact character, outfit and lighting; just animate this scene. The default, and what you want 80% of the time.
  • style_transfer - match lighting and mood only, invent a new pose or scene.
  • character_remix - keep the character, change the outfit and setting.
  • face_only - preserve the face, ignore everything else.

motion_description is your plain-language direction ("slowly turns head and smiles at camera"), and camera_movement, shot_size, motion_speed are the usual options, all defaulting to auto so you can ignore them.

The outputs

  • positive_prompt - wire this into your I2V model's conditioning. It's the whole point.
  • negative_prompt - auto-generated, usually worth keeping.
  • image_description - the raw caption from the vision pass. Inspect this when results feel off; it's also handy on its own as a detailed prompt for other things.
  • status - tells you whether the vision pass actually ran (more on that below).

The gotcha: vision is the fragile part

use_vision_model defaults to true, but the vision pass only works if your backend can actually see. With the default lm_studio backend, the model currently loaded in LM Studio must be a vision model - a plain Llama 3 will caption nothing, and you'll get an empty image_description and a weak prompt. Fix it either by loading a VL model in LM Studio, or switch vision_backend to qwen3_vl. That last one is the pack's self-contained path: it loads Qwen3-VL locally via transformers and needs no API server. Just know that the first run downloads a multi-gigabyte model from Hugging Face, and it needs the optional deps first:

pip install transformers>=4.42.0 accelerate>=0.30.0 huggingface_hub>=0.23.0 bitsandbytes>=0.43.0

The expansion pass can also use its own backend (expansion_backend), so you can pair a small vision model for captioning with a bigger LLM for writing.

Install

Shared with the pack: ComfyUI Manager → search "Local LLM Prompt Enhancer", or clone https://github.com/EricRollei/Local_LLM_Prompt_Enhancer.git into ComfyUI/custom_nodes/, pip install -r requirements.txt, restart. The core requirements file is just requests; the transformers stack above is only needed if you use the qwen3_vl backend.

One workflow note: if you're already on the Advanced video node, you don't need this one - the Advanced node's reference_image input does the same image-to-video pass with auto mode detection and more reference modes. This node is the simpler, more focused version. For most people, that's the right one.

CategoryEric Prompt Enhancers

Inputs (20)

NameTypeDefaultDescription
imageIMAGE
motion_descriptionSTRINGslowly turns head and smiles at camera
presetCOMBOcinematic7 options: custom, cinematic, surreal, action, stylized, noir, +1
expansion_tierCOMBOenhanced5 options: auto, basic, enhanced, advanced, cinematic
reference_modeCOMBOrecreate_exactrecreate_exact: Keep exact character, outfit, lighting - animate this style_transfer: Match lighting/mood only, create new pose/scene character_remix: Use character but change outfit/setting face_only: Preserve face identity only, ignore background/lighting
use_vision_modelBOOLEANtrue
vision_backendCOMBOlm_studiolm_studio: Uses currently loaded model in LM Studio ollama: Uses currently loaded model in Ollama qwen3_vl: Auto-detects local Qwen3-VL model (no API server needed)
vision_endpointSTRINGhttp://localhost:1234/v1lm_studio/ollama: API endpoint URL qwen3_vl: Leave default, or specify custom model path like 'A:\path\to\model'
expansion_backendCOMBOlm_studiolm_studio: Uses currently loaded model in LM Studio ollama: Uses currently loaded model in Ollama qwen3_vl: Auto-detects local Qwen3-VL model (no API server needed)
expansion_endpointSTRINGhttp://localhost:1234/v1lm_studio/ollama: API endpoint URL qwen3_vl: Leave default, or specify custom model path like 'A:\path\to\model'
temperatureFLOAT0.70.1–2
camera_movementCOMBOauto15 options: auto, none, static shot, camera pushes in, dolly in, camera pulls back, +9
shot_sizeCOMBOauto8 options: auto, none, extreme close-up shot, close-up shot, medium close-up shot, medium shot, +2
motion_speedCOMBOauto6 options: auto, very slow, slow, normal, fast, very fast
lighting_typeCOMBOauto7 options: auto, none, soft lighting, hard lighting, edge lighting, rim lighting, +1
time_of_dayCOMBOmaintain current7 options: auto, none, maintain current, sunrise, daytime, sunset, +1
positive_keywordsSTRING
negative_keywordsSTRING
save_to_fileBOOLEANfalse
filename_baseSTRINGimg2vid_prompt

Outputs (4)

NameTypeDescription
positive_promptSTRING
negative_promptSTRING
image_descriptionSTRING
statusSTRING