Sampler for Image to Video

Every other node in this pack is setup. Load Model drags in the weights, Load LoRA and the two plugins adjust the ride. This one - "Sampler for Image to Video" - is the payoff: you feed it an image and it runs the actual denoising loop that turns it into frames. If your workflow hangs, this is the node doing the work, and it's also the reason Ruyi gets called slow.

What it does

Ruyi is an image-to-video model: a start frame in, a video out. What made it stand out at its December 2024 launch is that it also takes an optional end frame and interpolates between the two - genuine start-to-end animation, which was exactly the gap in the local video scene back then (Hunyuan still had no official I2V, and people were hacking frame interpolation together by hand). The sampler is the only node that can reach that feature.

How it works

Under the hood it runs Ruyi as an inpaint-style pipeline. Your start frame is encoded, everything after it is masked out as "to be generated," and the sampler denoises the latent video while the CLIP image encoder keeps it anchored to your picture. Two of its inputs are worth understanding as keys rather than prompts:

• camera_direction (static, left, right, up, down, auto) and motion (1–4, auto) are not text. They're lookups into the model's precomputed embeddings.safetensors, keyed like p.16x9movie2static. That's why they're a fixed dropdown - you can't type "dolly left" and expect it to work. The tradeoff is that the controls are chunky and discrete; motion 1 is nearly static, motion 4 can shove your subject out of frame.
• base_resolution picks the closest aspect-ratio bucket to your input image (scaled from the model's 512 buckets), then snaps dimensions to multiples of 16. Set 512 for the lighter run or 768 for the model's claimed quality peak.

The sampler supports five schedulers (Euler, Euler A, DPM++, PNDM, DDIM) with DDIM as default, which is what the shipped workflows use.

The inputs that matter

• start_img (required) - the frame you're animating. Any IMAGE output, so Load Image or anything upstream.
• end_img (optional) - Ruyi's party trick. Feed a second frame and the video animates toward it.
• video_length - 8–120 frames in steps of 8; default 72. At 24fps that's 3 seconds; 120 is the full 5.
• motion and camera_direction - see above; defaults are motion 2, camera static, which is the sane starting point.
• steps (default 25) and cfg (default 7). The author's own README suggests cfg 7–10 for best guidance.
• GPU_memory_mode and GPU_offload_steps - your VRAM dials. low_memory_mode drops memory hard (community reported ~4–5GB) but slows generation dramatically; GPU_offload_steps (0–10) shunts temporary variables to RAM, with 7 or so letting a 24GB card handle 512×120.

Output

One output: images (an IMAGE tensor of your frames). That goes straight into a VHS_VideoCombine from ComfyUI-VideoHelperSuite to save an mp4 - which is also why the README makes you install VHS alongside this pack. You can alternatively wire the frames into an upscaler or post-processing chain first.

Installation

Install the pack itself once and it's shared across all the Ruyi nodes:

# ComfyUI Manager: search "Ruyi" and install ComfyUI-Ruyi (plus ComfyUI-VideoHelperSuite)
# or manually:
cd ComfyUI/custom_nodes
git clone https://github.com/IamCreateAI/Ruyi-Models.git
pip install -r Ruyi-Models/requirements.txt

The ~17GB Ruyi-Mini-7B model auto-downloads to ComfyUI/models/Ruyi the first time Load Model runs, so there's nothing extra to fetch for the sampler itself. On the Windows portable build, use ..\..\python_embeded\python.exe -m pip install ... instead of a system pip.

Where people get burned

• It's slow. That's the standing community complaint, not a setup error. A 3090 owner measured ~10 minutes for a 512×72 clip at launch. Expect to wait; low_memory_mode makes the wait longer, not shorter.
• The crash. A few people reported ComfyUI "crashes without errors, just exits" when sampling. Watch your terminal for the real error message - usually it's the model load or an OOM dressed up as a silent exit.
• Black lines on 3:4 / 4:5 videos. That was a real bug in the original weights, fixed by a model update on Dec 24, 2024. Keep Load Model's auto_update on so you don't sit on the broken version.

Ruyi isn't the default I2V anymore - Wan came along and took the crown in early 2025. But nothing else in this pack does what this sampler does, and the end-frame interpolation plus the built-in camera moves are still a legitimately different tool to reach for.