DreamOmni2 Text Encoder

The name lies a little, and the lie is the whole point. DreamOmni2 Text Encoder is not a CLIP or T5 replacement. It's a prompt rewriter: you give it a text instruction plus up to four reference images, it genuinely looks at the images, and it hands you back one enriched text prompt that now knows what's in them. You then feed that string into your completely ordinary CLIPTextEncode and sample as normal.

Why would you want a middleman like that? Because words are bad at describing a person's exact face, a fabric texture, or a hairstyle - but a vision model isn't. This is the same trick the editing world's reference-image models (Flux Kontext, Qwen-Image-Edit) pull by feeding latents straight into the transformer. DreamOmni2 - the open Qwen2.5-VL-7B finetune from the DreamOmni team that shipped in late 2025 as a challenger to exactly those models - is being repurposed here as the thing that translates your images into words, so any base model downstream can benefit. That means it slots into a plain Flux or SDXL graph with zero architectural changes.

How it works

Under the hood it's a small vision-language model, DreamOmni2 quantized to NF4 with bitsandbytes, running a chat-template prompt: your text plus the images, with It is generation task. or It is editing task. appended depending on mode. It decodes greedily (do_sample=False, up to 4096 new tokens), so it's deterministic - same images and instruction, same enhanced prompt, every time. The output is trimmed to the model's answer and returned as a plain string.

One honesty note: the source's docstring promises a fused multimodal embedding tensor and an image list as outputs. The actual RETURN_TYPES is a single STRING. What you get out is text, and the visual information enters the diffusion model through that text. If you want true latent-level reference conditioning, that's Kontext's job, not this node's - this one is the cheap, model-agnostic route.

The inputs that matter

• prompt - your instruction. Written conversationally ("make her hair red, keep the jacket") it gets understood better than tag soup, since the model reads it as a message.
• mode - generate (subject-driven generation, identity/pose from a reference) or edit (localized changes to the reference). The only functional difference is that one-word suffix, but it demonstrably steers the output.
• image_1 (required), image_2, image_3, image_4 (optional) - up to four reference images, the main draw over single-image editors.

The single output, prompt, wires into a CLIPTextEncode's text input. The pack's own example workflow does exactly that: enhanced prompt → CLIPTextEncode → FluxGuidance → KSampler, with the reference images stitched and fed to a Flux reference latent.

Installing it

Easiest via ComfyUI Manager - search Vantage-DreamOmni2 and hit install. Or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/vantagewithai/Vantage-DreamOmni2
pip install -r requirements.txt

Then restart ComfyUI. On first run the node downloads vantagewithai/DreamOmni2-nf4 from Hugging Face - several GB, one-time - into ComfyUI/models/dreamomni2, and caches the loaded model for the session.

Where people get burned

• It's a CUDA GPU node. NF4 quantization is bitsandbytes, so no MPS, no CPU. If you're on a Mac this one won't run at all.
• requirements.txt is incomplete. It lists six packages but the code also imports transformers and huggingface_hub. ComfyUI's own environment usually already has both - but if the node fails to import, pip install transformers huggingface_hub fixes it, and the transformers version needs to be new enough for Qwen2.5-VL (anything from mid-2025 on).
• The model folder is hardcoded. It always uses models/dreamomni2; you can't point it at an existing GGUF or download elsewhere. Interrupted downloads clean up and retry next run, which is more than most packs bother with.
• It shares VRAM with your real model. A 7B VLM at NF4 is manageable, but it's loaded alongside Flux or whatever you're actually sampling. If you hit OOM, encode once, save the resulting prompt as a string, then disconnect the node and rerun with the cached text - the output is deterministic anyway.

For an Oct-2025 model repackaged as a utility, this is a surprisingly clean wrapper. It won't beat a dedicated edit model for surgical pixel changes, but for "I want a new scene starring this person, described well enough that my Flux LoRA doesn't fight me," it's genuinely handy.