Diffusion SDXL Video (Frame by Frame)

Let's get the title out of the way: Diffusion SDXL Video (Frame by Frame) is not a video model. It's a loop. You hand it a batch of frames and it runs a full SDXL img2img sample on each one, then hands you the results back as one tensor. No temporal attention, no motion module, no magic. What it is is a convenient wrapper that saves you from building the "encode → ControlNet → KSampler → decode" graph dozens of times yourself, which is exactly what people mean when they say "frame-by-frame video."

The author's intended use case is spelled out in the tooltip on the model input: IC-Light. This is the "apply SDXL IC-Light to every frame of a video" workflow - relight your footage so the lighting is consistent across the whole clip. It works with a plain SDXL checkpoint too, per the README ("use compatible (sdxl) checkpoint, LoRa model and ControlNet"), so treat it as a general per-frame SDXL pass with the IC-Light workflow as its showcase.

How it works

The whole thing is one node class that calls ComfyUI's own core nodes - VAEEncode, ControlNetApplyAdvanced, and KSampler are standard ComfyUI nodes, imported directly. There's no custom sampling code and, notably, no extra Python dependencies - the pack's pyproject.toml declares nothing beyond what ComfyUI already ships.

The flow per run: frames get VAE-encoded once, then each latent goes through the same loop - ControlNet conditioning with that frame as the control image, KSampler with your prompt pair, decode, repeat. Output is a single images tensor (IMAGE) with all processed frames, so it plugs straight into a Save Image or video-combine node.

A couple of details worth knowing from the source:

• The very first frame is always skipped, and you can't fix it from the UI. The code clamps frame_start to a minimum of 1, so frame index 0 is dropped no matter what you set. Usually invisible on a long clip, but if your first frame is a title card it matters. Workaround: prepend a dummy frame before the node, or process the leftover first frame separately.
• Per-frame errors don't kill the run. If one frame fails to encode, condition, or sample, the node logs it and moves on. If everything fails it returns your original frames untouched. So you can get fewer frames out than you put in, with no error box - check your output length.

The inputs that actually matter

• model, positive, negative, control_net, vae - the standard five-way: an SDXL checkpoint (an IC-Light SDXL conversion if that's the job), your prompt conditioning, and a ControlNet with its own strength.
• cfg (default 8) - classifier-free guidance. The tooltip has it right: higher tracks the prompt harder, too high crushes quality. 5–8 is the sane SDXL range.
• sampler_name / sampler_scheduler - same list as the core KSampler. euler + normal or karras is a fine starting point.
• sampler_denoise - the single most important dial for video work. At 1.0 (default) every frame is a full redraw from noise; lower it toward 0.4–0.6 to keep the original structure and cut the flicker way down.
• sampler_seed - lock the same seed across frames for a bit more temporal stability.
• controlnet_strength / controlnet_start_percent / controlnet_end_percent - the usual ControlNet apply knobs.
• frame_start / frame_stop / frame_step - pick a sub-range of frames to process (stop 0 means "all frames").

Installing it

The README has no install instructions at all (it's seven lines - don't expect hand-holding). The real way is the standard one:

• ComfyUI Manager → search "ComfyUI-Diffusion-SDXL-Video" → install, or
• cd ComfyUI/custom_nodes && git clone https://github.com/Polygoningenieur/ComfyUI-Diffusion-SDXL-Video then restart ComfyUI.

There's no requirements.txt, no model files bundled, nothing heavy. The pack itself is a single commit from September 2025 and has zero community footprint - you're on your own for support, but there's also very little to break. The models are your responsibility: SDXL checkpoint, ControlNet file, VAE, and if you want the IC-Light workflow, an SDXL IC-Light conversion (the official IC-Light weights are SD 1.5 only, so grab a community SDXL port).

What it can't do

The honest limitation, and the reason this never became a video toolchain: no temporal consistency. Every frame is sampled independently, so you'll see flicker, especially at high denoise. The KB's IC-Light notes put it bluntly - frame-by-frame relighting for video was always the blocker, and no maintained toolchain ever solved it. Keep the denoise low, lock your seed, lean on your ControlNet, and think of this as a restyle/relight pass on top of real footage rather than a way to generate motion.

It's a tiny, opinionated pack that does one thing - but if per-frame SDXL relighting is what you want, it'll save you an afternoon of graph plumbing.