ComfyUI Node

Ollama

Local LLM captioning with no API key and no cloud

By isekai-sh·Created 9 months ago·Updated 8 months ago· 3
Ollama
    • response
    prompt
    model
    system_prompt
    ollama_urlhttp://localhost:11434

    IsekaiOllamaSummarizer runs a local LLM through Ollama and returns its text as a string. No API key, no cloud, no per-token bill - your images and prompts never leave your machine. If the pack's Claude and Gemini nodes are the "pay for a caption" route, this is the "run it yourself" route, and for batch captioning or prompt-work at scale it's the one you'll want.

    How it works

    It talks to a running Ollama server over HTTP - http://localhost:11434 by default (that's the ollama_url input, and it's overridable if your Ollama lives elsewhere). The prompt and optional system prompt get POSTed to Ollama's /api/generate-style endpoint, and the model's text comes back.

    Two details worth knowing, because they're easy to misread:

    • The model dropdown is live. At import time, the node asks your Ollama instance for its installed models and populates the dropdown from the real list. If Ollama isn't running, or the request times out (it gives it one second), it falls back to a hardcoded default list: llama3, mistral, llama2, clip, llava. So if you see those five and you know you've installed other models, Ollama wasn't reachable when ComfyUI loaded - not necessarily a bug.
    • It's a general LLM node, not just a summarizer. The "Summarizer" in the class name is leftover branding. Any Ollama model you've pulled will show up and can do any text job: captions, titles, tag extraction, prompt rewriting.

    The inputs

    • prompt - your text. Required.
    • model - dropdown from your local Ollama install (or the fallback list).
    • system_prompt - optional instructions.
    • ollama_url - default http://localhost:11434.

    Output is response, a STRING.

    Installing it

    Part of isekai-comfy-node:

    ComfyUI Manager: search "isekai" → install isekai-comfy-node → restart. Manually:

    cd ComfyUI/custom_nodes
    git clone https://github.com/isekai-sh/isekai-comfy-node
    cd isekai-comfy-node
    pip install -r requirements.txt
    

    Restart, then make sure Ollama itself is installed and running with a model pulled (ollama pull llama3.1) before you add the node. It lives under Isekai → LLMs. Only Pillow and requests are needed - Ollama is a separate program, not a Python dependency.

    Common issues

    The classic failure is "I pick a model and get an error." Check, in order: is Ollama actually running (does http://localhost:11434 load in a browser)? Is the model you selected actually pulled locally? Is the ollama_url right? The node returns error text as its output rather than raising, so look at the response for a message starting with "Error:" - the node prints the underlying error to the console too.

    One honest caveat: local LLMs on a machine that's also doing image generation compete for the same VRAM. A 7–8B model captioning alongside a busy sampler is a recipe for slow generations. If you're doing heavy LLM work mid-batch, consider whether the cloud nodes' latency is worth offloading - there's no free lunch, only different lunches.

    CategoryIsekai/LLMs

    Inputs (4)

    NameTypeDefaultDescription
    promptSTRING
    modelCOMBO5 options: llama3, mistral, llama2, clip, llava
    system_promptoptSTRING
    ollama_urloptSTRINGhttp://localhost:11434

    Outputs (1)

    NameTypeDescription
    responseSTRING