JoyCaption Ollama Prompter

If you're building a LoRA dataset, you already know the drill: good captions matter more than nearly every training knob, and JoyCaption Beta One is the tool most people use to write them. What this node does is sneakily clever - it takes the exact prompt templates judian17's original llama-cpp GGUF node used and hands them to an Ollama-hosted copy of the same model, so you get the real JoyCaption instructions without fighting llama-cpp-python's GPU build. It doesn't look at the image itself. It builds the two strings that tell the vision model how to describe it.

How it works

The node has two dropdowns and one optional input, and produces two strings: system_prompt and user_prompt. The user prompt comes from a hardcoded template map, one template per caption style, with three length variants each. Pick "Descriptive (Casual)" and length "long," and you get "Write a long descriptive caption for this image in a casual tone." Pick a plain number like 50 and the template becomes "...within 50 words."

The system prompt is where it gets interesting, because it's not generic. Choose a tag mode and the model is told it's a strict tag generator - comma-separated only, no explanatory text, no natural language. Choose "Stable Diffusion Prompt" and it's instructed as an SD prompt engineer. Every mode gets a word-count enforcement line appended: a hard cap for numeric lengths, an approximate guide ("around 100 words") for the named ones. Both outputs wire straight into OllamaGenerateV2's system and prompt inputs.

The inputs that matter

• caption_type - twelve modes. The defaults are the everyday ones: Descriptive (Casual) for natural-language training captions (the right choice for Flux-era datasets, which want prose, not tags), Danbooru tag list / e621 tag list / Rule34 tag list for anime and furry datasets that were trained on exactly that tag vocabulary, and Stable Diffusion Prompt / MidJourney when you want to generate a prompt from an image rather than a caption. The rest - Art Critic, Product Listing, Social Media Post - are fun for a single image and pointless for a 200-image dataset.
• caption_length - "any," "very short" through "very long," or a specific number from 20 to 260. If you want consistent captions across a dataset, pick a number; the model gets a hard word cap in both prompts.
• extra_options_input - the optional bundle from JoyCaptionOllamaExtraOptions. Skip it and you still get a perfectly good caption; plug it in and the extra instruction sentences get appended to the user prompt.

Wiring it in

The pack ships a full working workflow (ollama-joycaption-beta.json) you can load directly, but the chain is simple: LoadImage → OllamaConnectivityV2 (set to aha2025/llama-joycaption-beta-one-hf-llava) → OllamaOptionsV2 → this node → OllamaGenerateV2 → ShowText to inspect the result, or straight into your training captioner. ComfyUI-Ollama (stavsap) is the engine underneath; this pack just feeds it text.

Install and gotchas

ComfyUI Manager (search "JoyCaption-beta-one-hf-llava-Prompt_node") or:

cd ComfyUI/custom_nodes
git clone https://github.com/judian17/ComfyUI-JoyCaption-beta-one-hf-llava-Prompt_node

Restart, then make sure the actual model is present:

ollama pull aha2025/llama-joycaption-beta-one-hf-llava

That pull is a few GB - the workflow example runs the Q6_K quant, a reasonable quality/size tradeoff for captioning. This is a real, lived-in path: people run JoyCaption through Ollama inside ComfyUI precisely because the GGUF route is so fiddly, and report it beating other vision LLMs for prompt generation. Two caveats worth knowing. First, vision LLMs happily drift into describing motion or rambling meta commentary, so the templates' precision is the point - don't second-guess the "avoid meta descriptive phrases" guidance by removing it. Second, if you set a numeric length and the model still overshoots, that's a model quirk, not a node bug; the enforcement is a strong instruction, not a guarantee. Drop the temperature in OllamaOptionsV2 if it keeps running long.