IF LLM🎨

This is the node the whole pack is built around - the one that made r/comfyui sit up when IF-LLM hit the front page. One widget that talks to Ollama, llama.cpp, LM Studio, KoboldCpp, and text-generation-webui running on your box, plus hosted APIs from OpenAI, Anthropic, Google Gemini, Groq, Mistral, xAI, and DeepSeek, and even downloads Transformers models straight from Hugging Face. Thirteen providers, one interface, and it's also a vision model: feed it an image and it'll describe, critique, or turn it into a prompt.

The most popular way people actually use it is as a prompt engineer in the graph. You drop in a rough idea, the LLM writes you a proper prompt, and you route the response output into your CLIP text encode. The pack ships presets specifically for that - profiles, embellishments, style prompts, and negative prompts loaded from JSON files under IF_AI/presets/ - which is why the author also publishes tuned Ollama models like impactframes/llama3_ifai_sd_prompt_mkr_q4km.

How it works

You pick a llm_provider and type the exact llm_model name (no dropdown of your stuff - the pack validates the string against the provider's list at run time). Then user_prompt is the message. The node assembles a system prompt from prime_directives (the field's tooltip just calls it "the system prompt for the LLM") or from a named profiles entry, layers on whatever embellish_prompt, style_prompt, and neg_prompt you chose from the preset dropdowns, applies a stop_string, and fires the request.

The strategy dropdown changes what you get back:

• normal - the text answer you expect.
• omost - regional canvas prompting, paired with an omost model like impactframes/dolphin_llama3_omost. Instead of one prompt it emits a multi-region layout, returned through the omni output.
• create, edit, variations - the provider generates actual images (DALL·E, Gemini, that family), which come back on generated_images.

There's also an auto mode that batch-generates prompts from a folder of images, which is where IF_LLM_LoadImagesS feeds in.

Inputs and outputs that matter

Don't touch most of the optional fields. The ones you'll actually set:

• llm_provider, llm_model, user_prompt - the core of every run.
• max_tokens - caps the response (default 2048).
• random / seed - toggles temperature-based sampling against a fixed seed for reproducible runs.
• keep_alive - keep the model resident between calls (faster, eats VRAM) or unload it.
• images - connect a batch here to use a vision model.
• external_api_key - only if you don't want it picked up from env or .env.

Outputs: question (the fully assembled prompt that was actually sent - great for debugging), response (the answer you wire into your sampler or a display node), negative (the negative-prompt text for image generation), plus omni, generated_images, and mask for the special strategies. Note the tooltip on attention (sdpa / flash_attention_2 / xformers) - that's only relevant on the Transformers provider.

Installing and setting up

Install once for the whole pack - ComfyUI Manager (search "IF_LLM"), or:

cd ComfyUI/custom_nodes
git clone https://github.com/if-ai/ComfyUI-IF_LLM

then restart. The README is explicit about the recommended backend: install Ollama. That's why the default port is 11434. Pull something small to start:

ollama run llama3.2

For hosted APIs, set env vars with the exact names - OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY, GROQ_API_KEY, MISTRAL_API_KEY, XAI_API_KEY - or drop them in a .env file. And before installing: the README warns this pack conflicts with IF_AI_tools, so remove that first or the two collide.

Troubleshooting

• "Invalid LLM model selected" - the model name doesn't match what the provider reports. Use the pack's IF_LLM_ListModels node to get the exact string, tags and all.
• Transformers provider seems to hang the first run - it's downloading the model from Hugging Face, which can be gigabytes. That's expected, not a bug.
• Ollama connections failing - ollama serve running? Right port? Wrong base_ip if it's on another machine?
• API provider returns errors - keys must use those exact env var names; the external_api_key field overrides them when filled.

The honest take: this node is impressively broad, and the price of that breadth is that some knobs are fiddly (the model-name validation being the main one). Once you've got one provider working, the rest of the pack slots in around it.