JLC CaptionForge Qwen Caption

Joy is the crowd favorite, but it's not the only game in town - and CaptionForge's whole premise is that a second, independent captioning voice catches what the first one misses. JLC CaptionForge Qwen Caption is that second voice: a Qwen-family vision-language model loaded from Hugging Face, run as a Pass A witness alongside Joy (and optionally Ollama). It's structurally a sibling of the Joy node - same template path, same outputs, same standalone-or-pipeline split - but with a different model and a few different defaults.

The practical argument for it: Joy and Qwen have different blind spots, and when a distiller later merges their accounts, details that appear in both get reinforced while contradictions get flagged. The README is honest that model choices matter a lot and that Qwen's value is mostly as a complementary voice. If you only run one captioner, Joy is the community default; if you run two, this is the natural second.

How it works

It's a Python/Hugging Face engine, so weights load into your ComfyUI process (managed through the CaptionForge model cache, with eviction before Ollama stages run). Models are expected under ComfyUI/models/LLM/JLC_QwenCaption/ and auto-download on first use unless download_probe_only is on. The model dropdown ships five variants, from the lightweight Qwen2.5-VL-3B-Instruct up to Qwen2.5-VL-7B-Instruct, plus community finetunes like the Unredacted and abliterated NSFW-caption variants - which tells you what kind of datasets this pack expects.

qwen_quantization defaults to Balanced (8-bit), bitsandbytes load-time quantization that's genuinely worth keeping for the 7B variants on 16 GB cards. The sampling defaults differ from Joy's too - repetition_penalty sits at 1.08 here versus Joy's 1.0, a small nudge against looped captions.

Inputs and outputs that matter

• model - pick your Qwen variant; bigger isn't automatically better on a 16 GB card.
• qwen_quantization - Balanced (8-bit) vs Default.
• caption_type / caption_length - the template path (Descriptive, LoRA Literal, Taggy, Style Focus, SFW Character Caption…).
• system_prompt / custom_prompt - same structure as Joy; custom_prompt_mode overrides the template path when enabled.
• max_new_tokens / temperature / top_p / top_k - standalone sampling, overridden by the Pipeline Planner when a plan is connected.
• forbidden_phrases / replace_pairs - the shared cleanup filter.

Outputs are the familiar five: caption, resolved_prompt, image_out, pipeline_plan_out, template_options_out. caption is what feeds your dataset or the capstone's JSONL; resolved_prompt shows you the exact prompt that went to the model.

Install

Pack install once:

git clone https://github.com/Damkohler/CaptionForge.git ComfyUI/custom_nodes/CaptionForge

Restart, then pip install -e . in the folder if your ComfyUI env lacks the deps (torch, transformers, accelerate, huggingface-hub, pillow, numpy, safetensors, qwen-vl-utils). pip install bitsandbytes enables the 8-bit path. The model itself auto-downloads into ComfyUI/models/LLM/JLC_QwenCaption/ - no manual weight step.

Common issues

First run is a multi-GB Hugging Face download, so either be patient or dry-run with download_probe_only. If you OOM on a 7B model, check that qwen_quantization is actually set to Balanced (8-bit) - that's the single biggest lever. And expect VRAM handoffs in full-pipeline runs: Qwen and Joy share the process-local model cache and get evicted before Ollama stages take over, which is normal behavior, not a leak.