Joy_caption_alpha_load

This is the loader for JoyCaption Alpha Two - the generation of JoyCaption that added the whole "modes" system (descriptive, MidJourney, Booru tags, art critic, product listing…) that everyone talks about. The plain Joy_caption_load runs the older pre-alpha model; this one is the upgrade path, and if your captioning quality matters, it's the one you want.

It does the same job as the classic loader - assemble SigLIP + a Llama-3.1-8B + image adapter into a JoyPipeline_alpha - but the weights live in models/Joy_caption_alpha/ and come from the Alpha Two checkpoint (cgrkzexw-599808). And instead of a hand-dropped single adapter file, you're placing a whole checkpoint folder: clip_model.pt, image_adapter.pt, and a text_model/ directory.

How the load actually works

Two interesting details in the source are worth knowing:

1. The Alpha Two text_model is a PEFT adapter on top of Llama 3.1 8B. The loader downloads your chosen base Llama into LLM/, then edits the adapter's adapter_config.json to point base_model_name_or_path at that local copy before loading. That's why peft is in the pack's requirements. If you move or rename the LLM folder, this breaks in a confusing way - the config gets rewritten to the old path.
2. The SigLIP weights are custom. It loads the stock google/siglip-so400m-patch14-384 vision model, then overwrites it with the Alpha Two clip_model.pt. You still need the base SigLIP downloaded, but the actual behavior comes from the checkpoint.

The model dropdown gives you two Llama bases:

• Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2 (default) - an uncensored fine-tune, matching JoyCaption's "no refusal" positioning.
• unsloth/Meta-Llama-3.1-8B-bnb-4bit - the same 4-bit quantized base the classic loader uses, friendlier on VRAM and no gated-license dance.

JoyCaption Alpha One ran in as little as ~8.5 GB VRAM; Alpha Two with an 8B base sits in the same ballpark, so plan for ~10 GB.

Node surface

One input, one output:

• Input: model - the two Llama bases above.
• Output: JoyPipeline_alpha - feeds Joy_caption_alpha_run (single image), Joy_caption_alpha_batch / _Dirs (folder batches), and CXH_Ic_lora_Joy_batch (image-pair captioning).

Installing and the download chore

Pack install is the usual: ComfyUI Manager (search "Comfyui_CXH_joy_caption") or

cd ComfyUI/custom_nodes
git clone https://github.com/StartHua/Comfyui_CXH_joy_caption
cd Comfyui_CXH_joy_caption
python -m pip install -r requirements.txt

Then restart. The SigLIP base and the Llama base auto-download to models/clip and models/LLM, but the Alpha Two checkpoint itself must be placed manually at models/Joy_caption_alpha/. The README points at a Baidu Pan mirror and the HF space fancyfeast/joy-caption-alpha-two/tree/main/cgrkzexw-599808. Don't skip this step - the loader fails the moment it tries to open a missing clip_model.pt.

Gotchas

• NVIDIA only - the loader hardcodes .to("cuda").
• If you already ran the classic JoyCaption, you have SigLIP cached; the loader reuses that download. You still need the separate Alpha Two files.
• First load is slow (it's downloading + rewriting a config + loading an 8B model). The pipeline is only built once per model choice, so subsequent runs skip it.