Ovis-U1 Model Loader

Every workflow in this pack starts at this node, because there's exactly one model and it powers all three of the others. That model is Ovis-U1-3B: a 3-billion-parameter "any-to-any" multimodal model from AIDC-AI (Alibaba's applied-AI group, since rehosted under ATH-MaaS on Hugging Face). One set of weights does image understanding, text-to-image generation, and image editing - which is the entire selling point. You load it once here, get an OVIS_MODEL handle, and feed that same handle into the Image to Text, Text to Image, and Image Edit nodes. Load this, and you've got all three without touching a KSampler.

What it actually does under the hood

The node is two things glued together. First, it makes sure the model exists on disk. On a fresh run it looks in ComfyUI/models/ovis/AIDC-AI/Ovis-U1-3B/, and if the files aren't there it calls snapshot_download from huggingface_hub and pulls the whole thing - about 6 GB of sharded safetensors - into that folder. The default repo id (AIDC-AI/Ovis-U1-3B) still resolves fine; it now redirects to the ATH-MaaS copy, which is why the default keeps working without an update. If you'd rather stay offline, just drop the model files in that same models/ovis/AIDC-AI/Ovis-U1-3B/ path yourself and the node will find them and skip the download. It even verifies the download - missing config, an empty file, or a weight under 100 MB triggers a cleanup-and-retry with a reasonably clear error.

Second, it loads the weights with AutoModelForCausalLM. That's where the one gotcha lives: Ovis-U1's architecture is custom code that ships inside the HF repo, not in the transformers library. That's what trust_remote_code is for, and it's on by default for a reason. Flip it off and the load dies. It's the same trust model as BiRefNet and a dozen other HF deployments - the node executes code from the model repo, so treat the model repo like any software you download.

The inputs that matter

• device - auto picks CUDA if it's available, else CPU. This is the setting to touch if you're low on VRAM.
• dtype - bfloat16 is the default and it's right for anything Ampere or newer. On older NVIDIA cards (GTX 10/16-series) that don't do bf16 well, switch to float16. float32 is there if you must, and it doubles your VRAM bill.
• model_repo_id - currently a single choice, AIDC-AI/Ovis-U1-3B. There's no dropdown of alternatives because there are no alternatives yet.
• trust_remote_code - leave it true.

The 3B weights in bf16 run to about 6 GB, plus activations. People report it running fine on 12 GB cards, and 8 GB is survivable with float16. If you genuinely OOM, the node's own error message tells you to try CPU - slow, but it works.

Installing the pack

Same story as any custom node:

cd ComfyUI/custom_nodes
git clone https://github.com/neverbiasu/ComfyUI-Ovis-U1.git
cd ComfyUI-Ovis-U1
pip install -r requirements.txt

Or just use ComfyUI Manager and search for "Ovis-U1". One warning about that requirements.txt: it's the model's kitchen-sink list - deepspeed, trl, moviepy, uvicorn, the works. You don't need ninety percent of it just to run these four nodes; the loader really only needs transformers, huggingface_hub, and torch. If pip install -r requirements.txt trips over something like deepspeed (a classic Windows build pain), skip it and run:

pip install transformers accelerate huggingface_hub

Common issues

• First run looks frozen. That 6 GB download prints progress to the console, but it's not instant. Give it a few minutes on a slow connection.
• 401 / Unauthorized during download. The repo isn't gated, so this is rare, but the node honors the HF_TOKEN environment variable if you ever need it.
• Flash-attention errors. People running the model via the upstream HF Space have hit version pain there (the demo pins flash-attn==2.6.3). The node itself doesn't force a flash-attn build, but if you see flash-attention-related tracebacks, check your transformers version - the model authors tested against Transformers 4.51.3.
• OOM - drop to float16 first, then cpu if you have to.

One honest note: this author (neverbiasu) is the same person who ported OmniGen2, BAGEL, and ChatTTS to ComfyUI - they specialize in these unified multimodal models, and some of those earlier ports needed community workarounds. This one is a cleaner, standalone implementation, but it's a young pack from a small author, so expect rough edges.

The OVIS_MODEL output is pack-internal - it only plugs into this pack's three other nodes. From there, everything else is plain ComfyUI.