JoyCaption Loader

The pack has two transformers-based loaders, and this is the one that doesn't download anything. The model dropdown lists every directory sitting inside models/LLavacheckpoints (the tooltip on the field says it straight: models live in ComfyUI/models/LLavacheckpoints). Pick the folder, pick your settings, get a model out. It also fully respects ComfyUI's extra_model_paths.yaml, so if your models live on another drive, they show up here too.

So when do you reach for this instead of the Download And Load node? Whenever you don't want the pack fetching weights for you. You already grabbed the model by hand, you're offline, you want a specific local copy rather than the current beta release, or you've got a fine-tuned/custom JoyCaption variant you placed in that folder yourself. The Download And Load node's dropdown is fixed to the three fancyfeast repos; this one will load anything you give it a directory for. Same memory-mode options (Default, Balanced 8-bit, Maximum Savings 4-bit), same precision_default (fp16/bf16/fp32), same device dropdown, same optional lora input, same single joycaption_model output that feeds the caption nodes.

One structural gotcha

The dropdown is built by listing directories in LLavacheckpoints - the loader filters for folders, not loose files. So the model has to live in its own subdirectory, like models/LLavacheckpoints/llama-joycaption-beta-one-hf-llava/, which is exactly the layout the Download And Load node produces when it fetches a repo. If you've dumped files flat into the folder and the dropdown is empty or missing your model, that's the reason. Same rule applies to the separate models/LLavaloras folder used by the Lora node.

How it differs from the GGUF loader

Don't confuse this with the pack's GGUF loader (JoyCaption GGUF Loader), which only appears if you have llama-cpp-python with CUDA support installed. That one reads quantized GGUFs from models/llava_gguf plus a projector from models/llava_mmproj. This Loader is the plain transformers path - the same model the Download And Load node uses, just loaded from disk. GGUF is the VRAM-cheap route for small cards; this one is the quality route that also works without a special llama-cpp install.

Install and first run

Same pack as everything here: ComfyUI Manager → search wo_joycaption_comfyui, or

cd ComfyUI/custom_nodes
git clone https://github.com/without-ordinary/wo_joycaption_comfyui

then restart. The heavy lifting is transformers, torch, accelerate and bitsandbytes (for the quantized modes). No first-run download - that's the point - but you do need the model actually present in LLavacheckpoints, and the folder-name fragmentation between different LLM/VLM packs is a real thing, so double-check where your copy went before blaming the node.