LayerUtility: Load JoyCaption Beta One Model (Advance)

JoyCaption started life in mid-2024 as fpgaminer's answer to a real gap: every good vision-language model at the time was either ChatGPT (expensive, heavily filtered) or something like CogVLM (weaker, also filtered). fpgaminer built JoyCaption specifically to be free, open-weight, and uncensored - trained on SFW and NSFW content in roughly equal measure so it wouldn't choke describing anything past PG. It's since gone through several generations, and "Beta One" (fancyfeast/llama-joycaption-beta-one-hf-llava) is the current one - a LLaVA-style model built on Llama. This node is just the loader: it pulls the model into memory and hands a JOYCAPTIONBETA1_MODEL handle to the actual captioning node downstream.

There's basically one decision to make here, and it's quantization_mode. You get three: nf4 (4-bit, smallest footprint, some quality loss), int8 (middle ground), and bf16 (full precision, best quality, most VRAM). nf4 is the default and it's the sane choice unless you've got the memory to spare - Beta One is a real LLM-scale model under the hood, not a lightweight tagger, so on a card that's tight on VRAM the quantized mode is what keeps this usable alongside whatever generation pipeline it's feeding into. model currently only has one option (the Beta One checkpoint itself), and device is cuda or cpu - cpu will work but expect it to be slow for anything beyond a handful of images.

The one output is joycaption_beta1_model, which plugs straight into the LayerUtility: JoyCaption Beta One node alongside your image and caption settings. Splitting load from inference this way means if you're running multiple captioning nodes in one workflow, you only pay the model-load cost once.

Install the pack itself via ComfyUI Manager (search "ComfyUI Layer Style Advance") or git clone https://github.com/chflame163/ComfyUI_LayerStyle_Advance into custom_nodes/, restart, then run install_requirements.bat (or the Aki variant) to pull in dependencies. The model itself is the bigger lift: on first use it auto-downloads to ComfyUI/models/LLavacheckpoints/llama-joycaption-beta-one-hf-llava, or you can grab the folder manually from HuggingFace (fancyfeast/llama-joycaption-beta-one-hf-llava) and drop it in yourself if you'd rather not wait on an in-workflow download, especially if your connection to Hugging Face is unreliable.

Where this actually breaks: the README calls out a specific transformers version trap that hits JoyCaption nodes - if you see UnboundLocalError: local variable 'clip_processor' referenced before assignment (or the same for text_model) with the model files confirmed present, your transformers package is below 4.43.2 and needs updating. And if you're on transformers 4.45.0+ and instead get a protobuf error about descriptors needing to be regenerated, that's a version mismatch too - either downgrade protobuf to 3.20.3 or set the environment variable PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python. Both are documented, known issues rather than mysteries, so if you hit either, it's a package-version fix, not a broken model download.