JoyCaption GGUF

This is the node you reach for when you just want a caption and don't want to think about sampling parameters. Feed it an image, pick a caption style, hit run - you get a string of text back. If you've ever hand-typed captions for a LoRA dataset, or fought with BLIP giving you the same three sentences for every image, this is what replaced that job for a lot of people.

JoyCaption itself was built by fpgaminer (fancyfeast) as part of the bigASP SDXL finetune project, because the existing options for auto-captioning training images were bad: ChatGPT is paid and heavily filtered, CogVLM is weaker outside SFW content, and BLIP has been considered obsolete for LoRA work for years - "don't be lazy, caption them accordingly by hand" is the advice that used to follow any mention of it. Free, open weights, and equal treatment of SFW and NSFW content - that's why it's become the community's default for captioning datasets aimed at LLM-encoder models like Flux, Z-Image, Qwen-Image, and Krea 2. This node is 1038lab's GGUF build of it, quantized through llama.cpp to run on a fraction of the VRAM the full HuggingFace model needs, and it's the version the README recommends over the standard nodes for most people.

How it works

The image goes in, gets encoded by JoyCaption's vision component, and the language side generates a caption one token at a time, guided by whichever prompt_style and caption_length you picked. Because the weights are GGUF-quantized and run through llama-cpp-python instead of raw transformers, this node skips the generation-parameter fuss (temperature, top-p, and so on are fixed internally) - that's the whole trade this node makes: fewer knobs, faster to just use.

The inputs and outputs that matter

• image - required, the thing being described.
• model - which GGUF quantization to run: IQ4_XS is the default and the recommended balance of quality and speed for 8GB+ cards; Q2_K if you're squeezed to 6GB, Q6_K/Q8_0 if you've got headroom and want closer-to-full-precision output.
• processing_mode - Auto, GPU, or CPU. Leave it on Auto unless you have a specific reason to force CPU (a VRAM-starved box mid-generation elsewhere, say).
• prompt_style - don't sleep on this one. Beyond the obvious Descriptive/Straightforward options, it also covers Danbooru tag list, e621 tag list, Rule34 tag list, Stable Diffusion Prompt, and MidJourney styles - so the same node can output either a flowing sentence or a comma-separated tag string depending on what your training pipeline expects.
• caption_length - from "very short" to "very long," with "any" as the default that lets the model decide.
• memory_management - Global Cache holds the model resident for fast repeated runs (best on 24GB+ cards), Keep in Memory is the sane middle ground, Clear After Run frees VRAM after every image if you're tight on it.

One output: STRING, the generated caption. Wire it straight into a text file writer, a Caption Saver node, or a CLIP text encode if you're just spot-checking what the model says about an image.

How to install it

Search "ComfyUI-JoyCaption" in ComfyUI Manager, or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/1038lab/ComfyUI-JoyCaption.git
cd ComfyUI-JoyCaption
pip install -r requirements.txt

Because this is a GGUF node, you also need llama-cpp-python built with CUDA support. The pack bundles an automated installer - python llama_cpp_install/llama_cpp_install.py - worth using over a bare pip install, since llama-cpp-python needs to be compiled against your CUDA toolkit to actually use the GPU. Grab a GGUF model plus its vision-projection (mmproj) file from the README's download table and put them in ComfyUI/models/LLM/GGUF, or let the node auto-download the default on first run.

Common issues & troubleshooting

No module named 'llama_cpp' on first run. The most common snag across every llama-cpp-python-based ComfyUI node, and it almost always means the package installed into the wrong Python environment or without CUDA support. Use the automated installer script rather than fighting pip manually.

It's technically working but slow as CPU inference. Confirm processing_mode actually landed on GPU - if your llama-cpp-python build lacks CUDA wheels, "Auto" can quietly fall back to CPU and just crawl.

Captions read too generic or miss what matters in the image. This node has no custom_prompt field - that's the whole simplification. If a caption style plus length isn't specific enough for your dataset, that's the signal to reach for JoyCaption GGUF (Advanced) instead, which exposes the full sampling and custom-prompt controls.

Out of VRAM mid-batch. Drop to a smaller quant first (Q4_K_M or below); memory management mode only changes how much VRAM sits idle between runs, not how much the model itself needs while loaded.