StreamDiffusion_Sampler

StreamDiffusion_Sampler is the second half of the jesenzhang/ComfyUI_StreamDiffusion pack and the reason the whole thing exists. Feed it the stream object from StreamDiffusion_Loader, give it a prompt, and it emits an IMAGE - fast enough that with an image wired in, you can push frames at it and get a video-ish, self-inpainting effect. This is the "real-time interactive generation" pitch from the StreamDiffusion paper (arXiv 2312.12491), bolted onto a ComfyUI node. People actually use it for TouchDesigner / webcam art setups, with the usual caveat that round-tripping frames through ComfyUI's websocket adds latency the native demo doesn't have.

How it works, minus the math: normal sampling runs all 50 steps for every image. StreamDiffusion instead precomputes a denoising trajectory but only runs the UNet at a short list of timesteps - the index_list, default "32,40,45" - and runs those remaining steps as a batch (that's use_denoising_batch) while each new frame starts from the previous frame's latent. Consecutive frames share almost all the denoising work, so each extra frame is cheap - the real-time trick isn't fewer steps for one image, it's amortized steps across many. delta (default 1.0) scales the "virtual residual noise" that keeps those consecutive frames from drifting into mush. That's the whole trick behind the speed: not fewer steps for one image, but amortized steps across many.

Now the traps, because there are a few:

• The prompt fields are plain text. No CLIPTextEncode node - you type straight into the node. And the defaults are the strings "CLIP_POSITIVE" and "CLIP_NEGATIVE", which are placeholders the author left in. Generate without changing them and you're literally prompting "CLIP_POSITIVE". Replace them or your output is garbage and you'll have no idea why.
• The model input only accepts the Loader's output, and the Loader only speaks SD 1.5. Keep your 512x512 expectations.
• cfg defaults to 1.2 - that's the LCM guidance regime, and it's right. If you set cfg to 1.0 or below, the node forces cfg_type to "none" (guidance fully off). Text-to-image mode always forces "none" regardless; the full / self / initialize options only come into play for img2img, where they control how classifier-free guidance is applied across the streamed trajectory.
• img2img requires batch_size of 1 - the README says it plainly, and the code enforces it. Plug an image into the optional image input for img2img (an IMAGE, e.g. a webcam frame or the previous output) and you get frame-to-frame continuity.
• latent input is not implemented - it's listed, but the code doesn't touch it. Ignore it.
• steps is the trajectory length, not the number of UNet calls. With denoising batch on (default), the effective batch is len(index_list) × batch_size. Don't read "50 steps" and panic; only your index_list timesteps actually run.

Worth knowing if you go deeper: use_denoising_batch (default on) batches the residual denoising steps so the GPU stays busy instead of stalling between frames; enable_similar_image_filter (default off) skips outputting frames too close to the previous one - threshold 0.98, max skip 10 - which keeps near-duplicate frames from flooding a video pipeline; use_safety_checker is off by default and, if enabled, downloads the CompVis safety checker plus a CLIP model on first use. Both similar-image params are optional inputs you only touch in streaming setups.

Install once for the whole pack: Manager, or git clone https://github.com/jesenzhang/ComfyUI_StreamDiffusion into custom_nodes and restart. No extra pip installs - the repo vendors its dependencies and uses what ComfyUI already has. Output is a plain IMAGE; wire it to a preview or Save Image, or loop it back into image for the infinite effect. The pack's been quiet since early 2025 and it's very much a niche, so expect rough edges - but for live-in-a-box generation, this is the closest ComfyUI gets.