ComfyUI Coherent Video Sampler Node (V0.3)

A custom node for ComfyUI that enables coherent video generation while maintaining efficient memory usage, specifically optimized for heavy models like Flux.

Features

• 🎥 Frame-by-frame video processing with motion preservation
• 🧠 Efficient memory management for heavy models
• 🔄 Progressive denoising with coherence maintenance
• 💫 Dynamic quality control and motion guidance
• 🎨 Style preservation across frames
• 🛠️ Advanced adjustment controls for fine-tuning

Installation

Install from ComfyUI manager

or

Navigate to your ComfyUI custom nodes directory:

cd ComfyUI/custom_nodes

Clone this repository:

git clone https://github.com/ShmuelRonen/ComfyUI-CohernetVideoSampler.git

Restart ComfyUI

Usage

For Deforum-like results please use 'shuttle-3-diffusion-fp8.safetensors' 4 steps flux model

The node appears in the node menu as "Cohernet Video Sampler".

Core Parameters Guide

The sampler now includes four key adjustment parameters that work together to control different aspects of video generation:

1. denoise (0.0-1.0):

   • Primary denoising control for the sampling process
   • Controls overall deviation from input
   • Lower values (0.3-0.5): Subtle changes, closer to input
   • Higher values (0.7-0.9): More dramatic transformations
   • Recommended: 0.6 for balanced results

2. motion_strength (0.0-1.0):

   • Controls motion intensity between frames
   • Affects transition smoothness
   • Lower values (0.3-0.4): More static, stable output
   • Higher values (0.7-0.8): Pronounced motion, dynamic transitions
   • Recommended: 0.5 for natural movement

3. consistency_strength (0.0-1.0):

   • Maintains visual consistency across frames
   • Controls style preservation
   • Lower values (0.7-0.8): More variation allowed
   • Higher values (0.9-1.0): Strict consistency enforcement
   • Recommended: 0.9 for coherent results

4. denoise_strength (0.0-1.0):

   • Secondary denoising for artifact reduction
   • Fine-tunes final output quality
   • Lower values (0.5-0.7): Preserve more details
   • Higher values (0.8-0.9): Smoother, cleaner output
   • Recommended: 0.8 for balanced detail preservation

Parameter Combinations for Different Effects

High Quality Stable Video

denoise: 0.6
motion_strength: 0.5
consistency_strength: 0.9
denoise_strength: 0.8

Dynamic Movement Priority

denoise: 0.5
motion_strength: 0.7
consistency_strength: 0.8
denoise_strength: 0.7

Maximum Detail Preservation

denoise: 0.4
motion_strength: 0.4
consistency_strength: 0.85
denoise_strength: 0.6

Other Inputs

• model: Your diffusion model (tested extensively with Flux)
• positive: Positive prompt conditioning
• negative: Negative prompt conditioning
• video_latents: Input video in latent space (from VAE Encode)
• seed: Generation seed
• steps: Number of sampling steps
• cfg: Classifier free guidance scale
• sampler_name: Choice of sampler
• scheduler: Choice of scheduler

Memory Management

The node implements several memory optimization techniques:

• Progressive batch processing
• Automatic VRAM cleanup
• Dynamic batch size adjustment
• Efficient latent space operations

This allows it to work smoothly even with memory-intensive models like Flux without OOM errors.

Memory Usage Examples

When using with Flux model:

• 20 frame video @ 512x512: ~8GB VRAM
• 40 frame video @ 512x512: ~10GB VRAM
• Processing happens in windows of frames to maintain stable memory usage

Optimization Tips

1. For Smoother Videos:

   • Increase consistency_strength
   • Decrease motion_strength slightly
   • Keep denoise moderate
   • Maintain high denoise_strength

2. For More Dynamic Videos:

   • Increase motion_strength
   • Decrease consistency_strength slightly
   • Lower denoise_strength for detail
   • Adjust denoise based on desired change level

3. For Maximum Quality:

   • Balance all parameters
   • Use higher consistency_strength
   • Moderate motion_strength
   • Higher denoise_strength

Known Limitations

• Very long videos might need to be processed in segments
• Extreme motion can affect coherence
• High denoise values might reduce motion preservation
• Parameter interactions can be complex

Future Plans

• Additional motion control parameters
• Custom denoising patterns
• Advanced style preservation options
• Multi-model support optimization
• Parameter presets for common use cases

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

License

MIT License

Acknowledgments

• ComfyUI team for the amazing framework
• Flux model team for the inspiration in handling heavy models