Nodes/ComfyUI-DeZoomer-Nodes/Video Captioning
ComfyUI Node

Video Captioning

Generates detailed captions for videos using the Qwen2.5-VL model. Input Parameters: - images: Input video frames to process (ComfyUI's IMAGE type) - user_prompt: Detailed instructions for what aspects to analyze (default provided) - system_prompt: Instructions for the model's behavior and output style - model_name: Qwen2.5-VL model to use (7B, 1.5B, or 72B variants) - temperature: Controls randomness in generation (0.1-1.0) - use_flash_attention: Enables faster attention implementation - low_cpu_mem_usage: Optimizes for low CPU memory usage - quantization_type: Memory optimization (4-bit or 8-bit) - keep_model_loaded: Whether to keep the model in memory after processing - seed: Random seed for reproducible generation The node processes video frames and generates comprehensive descriptions covering: - Main content and characters - Object and character details - Actions and movements - Background elements - Visual style - Camera work - Scene transitions

By De-Zoomer·Created about a year ago·Updated about a year ago· 29
Video Captioning
  • images
  • caption
user_prompt1. **Main Content:** * What is the primary focus of the scene? * Who are the main characters visible? 2. **Object and Character Details:** * Don't refer to characters as 'individual', 'characters' and 'persons', instead always use their gender or refer to them with their gender. * Describe the appearance in detail * What notable objects are present? 3. **Actions and Movement:** * Describe ALL movements, no matter how subtle. * Specify the exact type of movement (walking, running, etc.). * Note the direction and speed of movements. 4. **Background Elements:** * Describe the setting and environment. * Note any environmental changes. 5. **Visual Style:** * Describe the lighting and color palette. * Note any special effects or visual treatments. * What is the overall style of the video? (e.g., realistic, animated, artistic, documentary) 6. **Camera Work:** * Describe EVERY camera angle change. * Note the distance from subjects (close-up, medium, wide shot). * Describe any camera movements (pan, tilt, zoom). 7. **Scene Transitions:** * How does each shot transition to the next? * Note any changes in perspective or viewing angle. Please be extremely specific and detailed in your description. If you notice any movement or changes, describe them explicitly.
system_promptYou are a professional video analyst. Please provide an analysis of this video by covering each of these aspects in your answer. Use only one paragraph. DO NOT separate your answer into topics.
model_nameQwen/Qwen2.5-VL-7B-Instruct
temperature0.3
use_flash_attentiontrue
low_cpu_mem_usagetrue
quantization_type4-bit
keep_model_loadedfalse
seed123
CategoryDeZoomerNodes/text

Inputs (10)

NameTypeDefaultDescription
imagesIMAGE
user_promptSTRING1. **Main Content:** * What is the primary focus of the scene? * Who are the main characters visible? 2. **Object and Character Details:** * Don't refer to characters as 'individual', 'characters' and 'persons', instead always use their gender or refer to them with their gender. * Describe the appearance in detail * What notable objects are present? 3. **Actions and Movement:** * Describe ALL movements, no matter how subtle. * Specify the exact type of movement (walking, running, etc.). * Note the direction and speed of movements. 4. **Background Elements:** * Describe the setting and environment. * Note any environmental changes. 5. **Visual Style:** * Describe the lighting and color palette. * Note any special effects or visual treatments. * What is the overall style of the video? (e.g., realistic, animated, artistic, documentary) 6. **Camera Work:** * Describe EVERY camera angle change. * Note the distance from subjects (close-up, medium, wide shot). * Describe any camera movements (pan, tilt, zoom). 7. **Scene Transitions:** * How does each shot transition to the next? * Note any changes in perspective or viewing angle. Please be extremely specific and detailed in your description. If you notice any movement or changes, describe them explicitly.
system_promptSTRINGYou are a professional video analyst. Please provide an analysis of this video by covering each of these aspects in your answer. Use only one paragraph. DO NOT separate your answer into topics.
model_nameCOMBOQwen/Qwen2.5-VL-7B-Instruct5 options: Qwen/Qwen2.5-VL-7B-Instruct, Qwen/Qwen2.5-VL-1.5B-Instruct, Qwen/Qwen2.5-VL-72B-Instruct, Vchitect/ShotVL-7B, Skywork/SkyCaptioner-V1
temperatureFLOAT0.30.1–1
use_flash_attentionBOOLEANtrue
low_cpu_mem_usageBOOLEANtrue
quantization_typeCOMBO4-bit2 options: 4-bit, 8-bit
keep_model_loadedBOOLEANfalse
seedINT1230–18446744073709550000

Outputs (1)

NameTypeDescription
captionSTRING