DiffutoonNode

DiffutoonNode is the headline act of this pack. You feed it a real video - a clip of a person dancing, walking, talking - and it re-renders every frame in anime style, keeping the motion, the structure, and the timing intact. It's not generating from nothing; it's toon shading, a video-to-video restyle, and it was a minor sensation when DiffSynth-Studio first demoed it. This node wraps that whole thing into a single box that runs inside ComfyUI, which used to take a research script and a colab to pull off.

How it works

This is a stack of SD 1.5-era tools doing one job, and the source is refreshingly uncomplicated about it. An SD 1.5 checkpoint (supplied via SDPathLoader) sets the look - the shipped workflow's default is the flat2DAnimerge anime merge. The AnimateDiff motion module (the mm_sd_v15_v2.ckpt, downloaded on first run) keeps frames temporally coherent, the same trick the KB documents: a frozen UNet plus a temporal motion module so your video doesn't flicker like a bad GIF. Up to three ControlNet units (from ControlNetPathLoader) steer structure - the bundled workflow runs lineart and tile at 0.5. Then RIFE frame interpolation smooths the output, and a negative embedding, verybadimagenegative_v1.3, is loaded automatically to keep the result clean. Clip skip is fixed at 2, the classic anime checkpoint setting.

The inputs that matter

• source_video_path (VIDEO) - use the pack's LoadVideo node to pick an mp4/webm/mkv/avi from your input folder.
• sd_model_path (SD_MODEL_PATH) - from SDPathLoader.
• postive_prompt / negative_prompt (TEXT) - from DiffTextNode. Yes, "postive" is the actual schema spelling.
• start / length - in seconds; start picks where in the video to begin shading, length how many seconds to shade. -1 means the whole video. The code asserts start + length stays inside the clip, so asking for more than exists is a hard error, not a silent trim.
• animatediff_batch_size / animatediff_stride - the two VRAM dials. The README's own advice: "lower it till you can run." Batch size is how many frames the motion module processes together; stride is how far the sliding window advances between batches.
• vram_limit_level - 0 by default; when ComfyUI kills the process with OOM, the README says try 1. It's a crude offload knob, not a magic fix.
• num_inference_steps (10) and cfg_scale (3) - quality vs. speed. Diffutoon runs a low step count by design.
• controlnet1 / controlnet2 / controlnet3 - optional, from ControlNetPathLoader.

Output is VIDEO - an mp4 written to your output folder. Wire it into PreViewVideo to see it in the UI.

The VRAM reality

Set expectations now: community reports put Diffutoon at roughly 20GB of VRAM at the default 1024 resolution, and the author tested on an 11GB 2080 Ti by cranking batch size and stride way down. This is not a node for a 6GB card unless you're shading a handful of frames at low batch sizes and accepting the wait. It's also genuinely slow - ten-plus diffusion steps per frame at 1024 with a motion module attached. Go make tea.

First-run gotchas

The node's own constructor downloads the AnimateDiff motion module, two annotator models, the RIFE frame-interpolation weights, and the negative embedding from HuggingFace the first time you instantiate it. If you can't reach HuggingFace you'll get a cryptic console message that reads, literally, "you can't attach huggingface? check your net and try again" - that's the code, not a joke. Also note the asymmetry: SDPathLoader offers a ModelScope mirror, but Diffutoon's auto-downloads do not, so a blocked HF is a real wall in parts of the world.

The honest take

By 2026 standards the KB calls AnimateDiff superseded, and Diffutoon is squarely an SD 1.5-era artifact: it's slow, it's VRAM-hungry, and a modern Wan or LTX video node will out-render it for most purposes. But it's also a specific look - smooth anime re-shading of real motion with the structure held by ControlNet - that nothing current reproduces with one click, and this node makes it available to anyone who can run it. Reach for it when you want that exact effect and you have the hardware; skip it if you just want "a video" and don't care about the style.

Install: ComfyUI Manager (search "ComfyUI-DiffSynth-Studio"), or git clone https://github.com/AIFSH/ComfyUI-DiffSynth-Studio into custom_nodes, then pip install -r requirements.txt - which brings in cupy-cuda12x, transformers, controlnet-aux, and friends. ffmpeg on your PATH is non-negotiable for the video side, and the pack targets Python 3.10 / torch 2.3.0+cu121. Expect the first Queue click to spend a while downloading before the shading starts.