JoyAI Video Edit

What it does, in one line

Feed it an IMAGE batch and a sentence, get the same number of frames back, edited. No mask, no ControlNet, no reference image, no VAE - the model rewrites the clip per instruction. "Turn the video into a watercolor wash style" is the pack's own example, and that's the whole prompt.

Same family as the still-image instruction editors that ate a big chunk of 2025–26 image work - Kontext, then Qwen-Image-Edit - where the pitch went from "mask it and inpaint" to "say what you want". On video it arrived heavier: Bernini made describing-beats-masking the argument, and the source clip becoming part of the input sequence roughly doubles compute, so a 120-frame edit processes something like 240.

JoyAI-Video-Edit's own angle is streaming. It works in 8-frame chunks inside one session, so the length limit is 600 frames, not 81. Fine detail: 840x480, landscape, fixed.

The mechanism, honestly

The node doesn't run a diffusion model inside ComfyUI. It converts your IMAGE batch to raw RGB uint8 frames, writes them to a job folder (%LOCALAPPDATA%\ComfyUI-JoyAI-Video-Edit\jobs by default) and hands prompt, seed and settings to the worker you started with JoyAI Runtime. Result frames come back, the node rebuilds a float32 0..1 IMAGE tensor, the files get deleted.

Resizing happens worker-side: bicubic, no crop, to the session's 840x480. Steps and temporal ids are baked into the code at the verified values (2 steps, 8 temporal ids), which is why you can't set them. v0.1 exposes only what was run on real hardware.

And the seed is not a reproducibility switch. Part of the latent is sampled from the process-wide CUDA random state, which isn't reset per job, and the fast kernels add jitter on top. Two runs with the same seed differ by roughly 1–4 levels out of 255 on average. Use it to nudge variation, not to lock a result.

Inputs that matter, outputs that matter

runtime comes from JoyAI Runtime - a JOYAI_RUNTIME handle, and the node refuses to run without it ("connect the JoyAI Runtime node").

images is the batch: RGB, 1–600 frames, float 0..1, sides between 64 and 4096 px, and width >= height. RGBA or grayscale is rejected outright rather than quietly converted, so a PNG sequence with alpha fails until you drop it.

prompt is the instruction, multiline and raw - no prompt enhancement, so what you type is what the text encoder sees. Keep it an instruction about the clip, not a caption of it.

seed is the usual 0–2^31-1 integer with control_after_generate off, so it holds still while you iterate on the prompt.

Outputs: images - one frame per input frame at 840x480, ready for SaveImage, a video combine node, whatever. And report, a JSON string with frames in/out, input and output sizes, the profile values and worker/engine timings. Point Preview Any at it; it also shows up in /history.

Wiring it up

The bundled workflow is the shortest honest example: LoadVideo → GetVideoComponents → JoyAI Video Edit → SaveImage, with a Preview Any on the report. Install is the pack:

cd ComfyUI/custom_nodes
git clone https://github.com/hiroki-abe-58/ComfyUI-JoyAI-Video-Edit

Restart ComfyUI (or install the pack by name in ComfyUI Manager) - the nodes have no Python dependencies. But the node alone generates nothing: the model lives in a dedicated WSL2 distro with a Python 3.10 env, joyomni_ops, SageAttention 2.2.0, ~50 GB of weights and a joyai.runtimes.json registering the distro and paths. Budget an evening and read docs/SETUP.md.

Where it goes wrong

Feed it anything vertical and you get taller than wide; v0.1 supports landscape input only. Portrait is refused, not cropped - rotate or crop upstream and re-encode.

Beyond 600 frames the node refuses the job rather than chunking across sessions, so long clips get cut and stitched, and the seams are the part you babysit.

Cancel is coarse. ComfyUI's Cancel stops the job between frames, and during the model load or first-frame initialisation it only takes effect after that step finishes. It's not hung.

Then there are the artifacts that aren't bugs: the pack documents a faint reddish trail or halo lingering next to a recoloured moving object, and strong styles flickering in fine textures frame to frame - watercolour is the named example. Output is always 840x480, so if you need 1080p, upscale after the edit rather than pre-upscaling the input and hoping it survives.

Finally, a word about fit. This isn't the editing tool most people should reach for first in 2026 - Bernini and the LTX IC-LoRAs cover V2V editing with far more community tooling and no WSL2 requirement. JoyAI's case is narrow: long landscape clips, one text instruction, and you've got a 5090 and patience.