DiffSynth: Qwen Video Edit Inference

This is the run button for prompt-driven video editing: you feed it a clip, you type what you want changed, and you get a new clip back. No mask, no per-frame painting, no ControlNet graph. It sits at the end of a chain that starts with a VRAM Config and a pile of ModelConfig nodes, which is the pipeline's own name for "here are the files, go fetch them."

What it actually is

Under the hood the node calls DiffSynth-Studio's QwenVideoEditPipeline, the inference path for yunpeng1998/Qwen-Video-Edit - a research checkpoint that takes Qwen-Image-Edit, the open instruction editor the community ended up standardising on, and repurposes it to edit video latents directly with a light trainable projection glued on. Feature-wise that's the Qwen-Image-Edit promise ("hand it a picture and a sentence") extended to a frame sequence, which is why the pack's example graph wires in both Qwen/Qwen-Image (the text encoder) and Qwen/Qwen-Image-Edit (the processor) alongside the Wan VAE.

Know what you're getting: this is a 360P checkpoint off a paper repo, not a tuned product. It's the node you reach for to see whether instruction video editing is useful to you, and the place to send a clip you'd otherwise mask by hand.

How it works, and the one thing that will bite you

Your clip arrives as an edit_video image batch (frames). The pipeline slices it into windows of num_frames and runs each window through the DiT, conditioned on a latent of the original clip, then decodes. The prompt side is where beginners get stung: the widget is prompts, plural, and it's one prompt per video chunk, one line each. If your clip is 3 chunks long and you typed one sentence, you get a console warning and the tail of your video is dropped. Give it as many lines as there are chunks and it behaves the way you expected from the first frame. It also sets its sampling schedule dynamically from the latent token count, so changing resolution or num_frames changes the schedule, not just the shape.

The inputs that matter

• pipe - from DiffSynth: Qwen Video Edit Loader. Everything else lives or dies on that object.
• edit_video - the clip, as IMAGE frames.
• prompts - one line per chunk, as above.
• num_frames (45), height (384), width (640) - window size and output shape. The pipeline snaps these to its own constraints, so odd numbers get quietly adjusted.
• num_inference_steps (40), seed, cfg_scale (4), negative_prompt - the sampler knobs. Note the negative prompt defaults to a single space rather than empty; leave it unless you have a reason.
• tiled, tile_size (30,52), tile_stride (15,26) - VAE decode tiling. On a 12GB card, turning tiled on is usually the difference between decoding and a CUDA OOM.
• zero_cond_t - an architecture-specific conditioning toggle. Off by default; leave it off unless the model card you're following says otherwise.

Output is a single video socket, type IMAGE - a batch of frames. Wire it to Video Combine, SaveAnimatedWEBP, or anything else that eats a frame batch; the pack has no save node of its own.

Install

ComfyUI Manager → search DiffSynth-ComfyUI → install → restart. Manually:

pip install git+https://github.com/modelscope/DiffSynth-Studio.git
cd ComfyUI/custom_nodes
git clone https://github.com/modelscope/DiffSynth-ComfyUI.git

Then restart ComfyUI. The plugin's own only dependency is diffsynth (2.1.9 or newer). First queue will pull the Qwen text encoder, the edit checkpoint and the Wan VAE - several GB, so start it before you make coffee.

Where people get burned

• Nothing to edit. If edit_video isn't connected, you get a text-to-video-shaped result or an error, not a warning. Load your clip with a video-loader node upstream.
• One prompt for five chunks. Silent frame loss, covered above.
• The OOM on decode, not on sampling. Sampling with VRAM offload survives; the VAE decode is what dies. Flip tiled on.
• Version drift. Every input on this node is introspected from the DiffSynth-Studio installed in your environment at startup. Upgrade the framework without restarting and the sockets can change under you - restart ComfyUI after touching diffsynth.