Doubutsu Image Describer

Let me save you some time: the README opens with a big warning box saying the project is deprecated and no longer maintained, and its own author tells you to use "better vision tools" instead. So the honest headline is that you probably shouldn't install this. But it's also a tidy little example of what a local vision-language model node looked like in 2024 - and if you're here anyway, this is what you need.

What it is

DoubutsuDescriber (shown as "Doubutsu Image Describer", under image/text) runs the Doubutsu small VLM on your GPU - no API, no key, all local. Doubutsu is a 2-billion-parameter model (qresearch/doubutsu-2b-pt-756, Apache-2.0) pretrained on LLaVA-CC3M, paired here with a PEFT LoRA adapter (doubutsu-2b-lora-756-docci) fine-tuned on Google's DOCCI captioning dataset - the "docci" in the name is exactly that. Feed it an image, ask it a question ("Describe the image" by default), and it writes you an answer as a string. The whole trick is captioning plus freeform question-answering, on a card that fits the model.

The catch is timing. This is a mid-2024 idea the ecosystem passed right by - an exact-phrase Reddit search for the node turns up zero threads, and its HF repos have single-digit download counts. For captioning today the community standard is JoyCaption or Florence-2, and captioning quality matters more than dataset size anyway. The one thing this old node does that a tagger doesn't is answer arbitrary questions about an image, which is a job people now do with a Qwen-VL-class node instead.

How it works

The node is a thin wrapper. On first run it loads the base model with trust_remote_code=True - Doubutsu ships custom modeling code, so that flag is mandatory, not a choice - moves it to CUDA in your chosen precision, then applies the LoRA via load_adapter(). Your ComfyUI image tensor gets squeezed down to a PIL image (it even defends against (C, H, W) input by transposing), and the whole thing is handed to the model's answer_question() under autocast. Out pops one STRING.

Note what's missing: no offloading, no CPU path, no Apple Silicon support - everything goes straight to CUDA and the model stays resident. A 2B model in fp16 is roughly 4 GB of VRAM before the adapter and KV cache, so budget for a 6 GB card.

The inputs that matter

All five inputs are required, but you'll touch maybe three of them:

• image - the IMAGE you want described. Feed it a single decoded image from your loader or any VAE-decoded output.
• question - default "Describe the image". The freeform part: ask "what color is the jacket?" or "is there text in this frame?" and you get a targeted answer instead of a caption.
• max_new_tokens - how long the answer may be (1–512, default 128). Bump it for long descriptions, drop it for one-word answers.
• temperature - default 0.1, which is already quite greedy for a captioning task; raise it if answers feel repetitive.
• precision - float16 (default) or bfloat16. The README says bf16 is quicker on GPUs that support it (Ampere and newer).

The single output is a plain STRING - wire it wherever you'd feed a caption: a show-text node, a file writer, or another node's prompt field. The base UI won't print a raw string, so you'll want a text display node.

Installing it

Install via ComfyUI Manager (search "ComfyUI-Doubutsu-Describer") or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/EnragedAntelope/ComfyUI-Doubutsu-Describer
pip install -r requirements.txt

The dependencies are torch, transformers, Pillow, bitsandbytes, peft. Two honest notes: these install into ComfyUI's shared Python environment with no isolation, so a transformers version bump here can break other nodes; and the node never actually imports bitsandbytes - it's in the requirements file anyway. That tells you everything about how maintained this pack is.

The real install hurdle is the models, because ComfyUI's own model downloader won't fetch them. They live inside the node folder at custom_nodes/ComfyUI-Doubutsu-Describer/models/qresearch/..., and the code hard-fails with FileNotFoundError if they're absent. Use the Hugging Face CLI:

cd ComfyUI/custom_nodes/ComfyUI-Doubutsu-Describer
huggingface-cli download qresearch/doubutsu-2b-pt-756 --local-dir models/qresearch/doubutsu-2b-pt-756
huggingface-cli download qresearch/doubutsu-2b-lora-756-docci --local-dir models/qresearch/doubutsu-2b-lora-756-docci

Then restart ComfyUI.

Troubleshooting and the verdict

The failure you'll actually hit is the FileNotFoundError - model or adapter not in the exact path above. Beyond that: CUDA errors mean you're on the no-offload path, and if the model refuses to load, trust_remote_code is involved - treat that with care. Unmaintained custom nodes that execute remote code are exactly the attack surface the ComfyUI community got burned on with the LLMVISION incident. If it breaks, the fix is yours.

My take: skip it for real work - for local captions use a maintained node around JoyCaption or Florence-2. But if you're curious how a tiny VLM gets bolted into a ComfyUI graph, this is a legitimately clean example: short, readable, and it only costs a couple of gigs of VRAM to satisfy the curiosity.