Sora Img2Video
Sora 2 in ComfyUI, minus the VRAM bill
- client
- first_frame_image
- url
- task_id
This is the node for when you want OpenAI's Sora 2 turning your still image into a moving one, and you don't happen to own the compute it takes. The name is a small lie worth knowing up front: nothing runs on your GPU. This is a paid API wrapper - you feed it a first frame, it submits an image-to-video job to UCloud's Modelverse API, and ComfyUI waits for the finished mp4. No model download, no VRAM math, no CUDA out of memory at 3am.
What it does
Sora Img2Video is the image-first sibling of the pack's Sora Text2Video node. You give it one starting frame, an optional prompt steering the motion, and it produces 4, 8, or 12 seconds of video. It's the node you reach for when you have a character render or a keyframe from another workflow and want to see it move - animation-in-betweening, product shots, bringing a still you already like to life.
The inputs that matter
- client - required on every node in this pack. Wire it from the Modelverse Client or Modelverse Secret Client node (see below).
- model -
openai/sora-2/image-to-video(default) orimage-to-video-pro. Pro is the one that supports 1080p. - first_frame_image or first_frame_url - exactly one. The node throws if you give both, and throws again if you give neither. A tensor from any IMAGE source works, or paste a URL.
- prompt - optional, guides the motion. Worth writing even two words; an empty prompt gives you less directional control.
- resolution -
720por1080p. Here's the trap: pick 1080p while the non-pro model is selected and the node refuses to run. It's a hard error, not a silent downgrade. Pro model, or 720p. - duration - 4, 8, or 12 seconds. 12s on pro is where your wait time and bill both grow.
It hands back two strings: url (the finished video's address) and task_id (for tracking the job). Both are plain STRINGs - the video only shows up once you run them into the pack's Modelverse Preview Video node, which downloads the mp4 into ComfyUI/output and gives you a VIDEO tensor to preview.
Installing and keying up
Install through ComfyUI Manager: search "ComfyUl-UCloud" in Custom Nodes Manager, install, restart. Or by hand:
cd ComfyUI/custom_nodes
git clone https://github.com/ucloud/comfyui-plugin
Then restart ComfyUI. Dependencies are light - requests, pillow, numpy, openai, GitPython - and Manager installs them for you. No models to download, ever.
You do need a UCloud ModelVerse API key from the console, and two clean ways to feed it in: the Modelverse Secret Client node (you type a secret name, the key lives in the pack's secrets.json, so it never gets baked into the workflow file) or a config.ini with MODELVERSE_API_KEY=... in the pack folder. The plain Modelverse Client node works too but bakes your key into the workflow - don't share that JSON.
Common issues
- "API key is not set" / 401 Unauthorized - the client node is empty or the key is wrong. Check the client node,
secrets.json, orconfig.ini. - 1080p error on the non-pro model - that's by design. Switch model, not resolution.
- "must provide first_frame_image or first_frame_url" - you forgot the frame entirely. Sora I2V has no pure text fallback; that's what the T2V node is for.
- Long waits - jobs poll every 5 seconds for up to ~15 minutes. Big Pro clips at 1080p ride right up against that. Not a hang, just cloud video.
It costs real money per generation and it's a third-party billing relationship, so check the console before batching 10 requests. For one-off "make my keyframe move" work it's hard to beat from inside the graph.
Inputs (7)
| Name | Type | Default | Description |
|---|---|---|---|
| client | MODELVERSE_API_CLIENT | — | |
| model | COMBO | openai/sora-2/image-to-video | image-to-video: 普通版, image-to-video-pro: Pro版(支持1080p) |
| first_frame_imageopt | IMAGE | 首帧图片 | |
| first_frame_urlopt | STRING | 首帧图片URL (与image二选一) | |
| promptopt | STRING | 提示词,用于指导视频生成 | |
| resolutionopt | COMBO | 720p | 分辨率 (Pro版支持1080p) |
| durationopt | COMBO | 4 | 视频时长(秒): 4, 8, 12 |
Outputs (2)
| Name | Type | Description |
|---|---|---|
| url | STRING | — |
| task_id | STRING | — |