ComfyUI Node

Stream_Sampler

Text, video, or your webcam

By smthemex·Created 2 years ago·Updated 2 years ago· 12
Stream_Sampler
  • pipe
  • image
  • image
  • audio
  • fps
info
promptClaymation, a man is giving a talk
video
sampler_type
guidance_scale1.0
diffusion_steps4
num_inference_steps50
noise_strength0.4
seed2
width512
height512
acceleration

Stream_Sampler is the payoff node - the one that actually runs StreamDiffusion and StreamV2V, the real-time diffusion engines this pack exists to put in ComfyUI. Everything before it (model loader, LoRA loader) was just assembling the diffusers pipe; this node lights it up. It's one sampler with three personalities, chosen via sampler_type: txt2img, vdieo2vdieo (yes, that's the real spelling of "video2video" in the dropdown - a typo from the author that never got fixed), and WebCam2Video. Each one builds a StreamV2VWrapper around your pipe and streams frames through it.

The mechanism is the interesting part, and it's why this pack is its own little ecosystem. The vendored engine uses StreamDiffusion's batched denoising plus StreamV2V's "feature bank" - the Looking Backward trick that reuses cached features across frames so video-to-video doesn't re-render every frame from scratch - along with cached attention and ToMe token merging, and even RAFT optical flow internally for the temporal consistency checks. That's a lot of moving parts hidden behind one node, which is both the appeal and the maintenance risk.

The inputs a beginner actually touches:

  • prompt - your text prompt, multiline. The pack appends your trigger word plus "style" automatically, so you don't type it.
  • sampler_type - the mode switch described above.
  • guidance_scale - default 1.0, and this is the one that surprises people. This is an LCM pipeline: CFG 7 is wrong here, the low default is deliberate (the distillation essay calls this exact trap out - keep it near 1 and quality comes from the distilled model, not from guidance).
  • diffusion_steps (default 4) vs num_inference_steps (default 50) - the confusing pair. num_inference_steps is what the LCM prepare call uses; diffusion_steps only matters in video mode, where it spaces out the denoising across your noise_strength window. Lower noise_strength = closer to the source video.
  • video - in video mode, a dropdown listing mp4/webm/mkv/avi files you've dropped into ComfyUI's input folder. Set it to "none" in video mode and the code raises "need video input".
  • width/height - 512 default, but only used for txt2img and webcam. Video mode ignores them and works at the source video's own resolution.
  • acceleration - xformers or tensorrt. The README says flat-out that TensorRT "still has bug", so default to xformers unless you enjoy installing polygraphy, onnx_graphsurgeon, tensorrt and cuda-python for the privilege of debugging.
  • image (optional) - an IMAGE input for the img2img side of webcam mode.

Outputs are image (IMAGE), audio (AUDIO) and fps (FLOAT). In video2video mode the pack reads the source video's soundtrack and frame rate and passes them through - you wire image + audio + fps into whatever writes your video file, and you get the restyled clip back with its original audio intact. In txt2img and webcam modes only the image output is really populated.

Install and models are the same pack-wide story: clone via Manager ("ComfyUI_Streamv2v_Plus") or git clone https://github.com/smthemex/ComfyUI_Streamv2v_Plus into custom_nodes, restart, pip install -r requirements.txt plus whatever's missing (the code pulls in diffusers, transformers, torchvision, opencv-python on top of the README's einops/av/peft/fire). First run hits HuggingFace to fetch configs, and each mode downloads its LCM/VAE pieces on demand.

Gotchas that will actually bite: the webcam mode needs serious VRAM - the README's own note is "cam2video needs more VR, low config runs slow" - so don't judge the pack by a webcam run on a 6 GB card. And if you get the "PEFT backend is required" error anywhere in the chain, pip install -U peft transformers fixes it. This is a real-time niche pack from smthemex, a one-person node factory, so expect it to work best on the exact workflow from the README's example graphs and to be forgiving of nothing else.

CategoryStreamV2V_Plus

Inputs (14)

NameTypeDefaultDescription
pipeMODEL
infoSTRING
promptSTRINGClaymation, a man is giving a talk
videoCOMBO1 options: none
sampler_typeCOMBO3 options: txt2img, vdieo2vdieo, WebCam2Video
guidance_scaleFLOAT1.00–30
diffusion_stepsINT41–1000
num_inference_stepsINT501–1000
noise_strengthFLOAT0.40–1
seedINT20–18446744073709550000
widthINT512256–4096
heightINT512256–4096
accelerationCOMBO2 options: xformers, tensorrt
imageoptIMAGE

Outputs (3)

NameTypeDescription
imageIMAGE
audioAUDIO
fpsFLOAT