JoyCaption-BetaOne-Run

Almost every node in ComfyUI_LLM_Banana is an API wrapper. This one isn't. JoyCaption-BetaOne-Run runs the actual JoyCaption VLM on your own GPU - a real 8B Llama 3.1 + SigLIP2 model - to write natural-language captions for your images. It's the community-standard captioner (fpgaminer's project, covered in the joycaption KG panel) wrapped as a ComfyUI node, and it's uncensored, which is the whole point for a lot of users.

But there's a serious wart, and you should know it before you install: the shipped code has the author's hardcoded model path baked in. The node looks for the model at D:\Ken_ComfyUI_312\ComfyUI\models\LLM\... - a literal Windows path from the developer's machine. On any other computer it fails with "模型路径不存在" (model path does not exist) until you fix it. The intended location is ComfyUI/models/LLM/<model folder>, and you have to download the model there yourself; this node does not auto-download.

What it's for

Captioning. Training LoRA sets, describing reference images for img2img or video, or just cataloging a folder. The preset dropdown covers the standard JoyCaption modes: formal/casual captions, Stable Diffusion prompts, MidJourney prompts, Booru tags, art-critic analysis, product listings, social posts - plus several Chinese-language presets. Pick a preset, wire in an image, get text back.

The inputs that matter

• model - llama-joycaption-beta-one-hf-llava-nf4 (default, ~4-bit quantized, less VRAM) or the unquantized llama-joycaption-beta-one-hf-llava (more accurate, heavier). Get them from fancyfeast on HuggingFace - the nf4 build is the one most people can actually run.
• preset_prompt - 14 presets, the mode selector described above.
• user_prompt - free-form override; combined with the preset if merge_prompt is on.
• image - the IMAGE wire. There's also an optional images_dir for batch processing a folder.
• temperature / top_k / top_p / max_new_tokens / seed - sampling controls. Defaults are sensible (0.6 / 10 / 0.9 / 256).
• use_cache - keeps the loaded model in memory between runs (set to false and it reloads every time, which is slow).
• unload_model - free VRAM after the run, useful when the captioner shares the card with a diffusion model.
• save_img_prompt_to_folder - write captions to disk, handy for training datasets.

Output: a single text STRING.

Install and the path fix

Install the pack, then get the model:

cd ComfyUI/custom_nodes
git clone https://github.com/xuchenxu168/ComfyUI_LLM_Banana
cd ComfyUI_LLM_Banana
pip install -r requirements.txt

requirements.txt pulls transformers, torchvision, and friends. Then download the model folder (the nf4 one to start) into ComfyUI/models/LLM/llama-joycaption-beta-one-hf-llava-nf4. If the node still errors on the hardcoded D:\... path, edit joycaption.py around line 708 and change os.path.join("D:\\Ken_ComfyUI_312\\ComfyUI\\models\\LLM", model) to point at your real models dir. Annoying, but it's one line.

Gotchas

The first run downloads nothing (you supply the weights) and the model itself is ~8B - on a consumer card the nf4 quant is the difference between "works" and "swaps to system RAM." And remember the JoyCaption caveat from the llm-in-comfyui KB doc: multi-subject attribution is its weak spot, so audit important captions by hand.