🖼️ Janus Image Understanding

You've got a folder of reference images and a blank prompt box, or a stack of training images that need captions, and you're tired of describing them by hand. This is the node for that. Janus Image Understanding feeds a picture to DeepSeek's Janus-Pro model and gets text back - a description, an answer to a question, or (by default) a full stable-diffusion-style prompt. It's the most-used node in this pack for good reason, because the thing Janus is genuinely good at is reading images, not drawing them.

What it actually does

Janus-Pro is DeepSeek's unified multimodal model: one weights file that does both vision-language understanding and text-to-image generation. This node rides the understanding half. It takes the model and processor outputs from the 🔮 Janus-Pro Model Loader, converts your ComfyUI IMAGE tensor to a PIL image, wraps it in a VLChat conversation, and runs the language model's generate() - greedy (do_sample=False), so the same image and question give the same answer every time. The response is cleaned of special tokens and handed out as a plain STRING.

The inputs that matter

• image - whatever you're analyzing. Wire in from Load Image or any node that outputs IMAGE.
• question - the prompt for the model. The default is Describe this image in detail for stable diffusion prompt, which makes the node a one-stop image-to-prompt tool. Swap it for plain captioning, object counting, "what's wrong with this render," whatever you need.
• max_tokens - ceiling on the answer length (64–2048, default 512). You'll rarely touch it; the default is plenty for a caption, and you only raise it if the model is cutting itself off mid-description.

The only input you can't skip is the pair from the loader: model (JANUS_PRO_MODEL) and processor (VLC_PROCESSOR). No model, no node.

What comes out

One output: prompt (STRING). Wire it into a CLIP Text Encode to drive a normal diffusion generation, into a text-saving node to keep the caption, or into any other text input. That single wire is the whole appeal - you can build a pipeline that reads an image, writes a caption, and feeds it straight into an SD/SDXL sampler, no manual copy-paste.

Where people actually reach for it: captioning datasets for LoRA training (Janus got a real reputation as a captioner when it launched), turning a reference photo into a starting prompt, or just asking questions about an image you didn't generate. If you want neutral, factual captions rather than SD-flavored hype, change the default question - the default is deliberately tuned for prompt-writing.

Installing the pack

Both nodes ship in greengerong/ComfyUI-JanusPro-PL (the "Janus-Pro ComfyUI Plugin"). Easiest path is ComfyUI Manager - search the pack title and hit install. Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/greengerong/ComfyUI-JanusPro-PL
pip install -r requirements.txt

Then restart ComfyUI. That requirements.txt pulls in the Janus library straight from DeepSeek's GitHub (git+https://github.com/deepseek-ai/Janus.git) plus transformers, torch, bitsandbytes, Pillow, and numpy. Note the bitsandbytes - it's a hard dependency even if you only ever use bf16 precision, so if install fails, that's the usual suspect.

Troubleshooting

• First run stalls / errors on load - the model isn't downloaded yet. The loader pulls it from Hugging Face on first use (the 7B is ~16 GB, the 1B is a couple GB) into models/Janus-Pro. Give it a minute; a slow connection will look like a hang.
• CUDA out of memory - the model itself lives in VRAM before any text is generated. On a small card, switch the loader to Janus-Pro-1B with int4 precision.
• Answers get cut off - raise max_tokens.
• Empty or weird output - check that your image is a normal RGB IMAGE; the node converts tensors to PIL internally and handles most shapes, but a node feeding it a non-image tensor will blow up at the conversion step.

The honest caveat: this is a niche-but-useful tool, not a daily driver for most people. If you don't caption datasets or want image→prompt loops, you probably don't need it. But when you do need local, offline image understanding without an API key or a cloud bill, Janus-Pro is the model, and this is the node.