Nodes/iwanders/ComfyUI_nodes/IW Ollama Generate String
ComfyUI Node

IW Ollama Generate String

Generate prompts from a local LLM, and keep the recipe in the PNG

By iwanders·Created 2 years ago·Updated 7 months ago· 2
IW Ollama Generate String
    • STRING
    generatetrue
    generate_seed
    generate_prompt
    generate_model
    use_seed0
    use_result
    use_prompt

    IW Ollama Generate String is the pack's one heavyweight: it calls a local Ollama LLM, gets a generated string back, and hands it to your workflow as the prompt - while making sure the exact text that produced an image is recoverable later. This is the node in the pack people actually seek out, and it's more clever than its plain-Jane string cousins.

    How the "off-by-one" trick works

    Here's the design that keeps everything reproducible: the node has two sets of inputs. generate_seed, generate_prompt, and generate_model define the next LLM call. use_seed, use_result, and use_prompt hold what the current image actually used. When you queue a run, the node calls Ollama with the generate values, returns the previous result (use_result) on its STRING output - that's the text your text encoder gets - and sends the new result to the UI. The frontend extension then copies the new values into the use_* widgets. So the prompt feeding your image is always the already-displayed one, and because those use_* values are part of the workflow, they get embedded in the PNG metadata. Later you can look at any image and see exactly what LLM text generated it. That's the reproducibility culture of ComfyUI, done properly.

    There's a generate toggle to flip between "cycle to a new LLM result" and "keep reusing the current one," and a small LRU cache (last 10) keyed by seed+model+prompt, so re-running the same inputs doesn't re-pay the LLM cost. The seed is passed to Ollama as the generation seed, so you can reproduce a prompt deterministically.

    What you set

    • generate - bool; generating new or reusing the shown value.
    • generate_seed - INT seed for the LLM call.
    • generate_prompt - the instruction to the LLM, multiline ("write a cyberpunk scene, 20 words, no quotes").
    • generate_model - dropdown of models installed in your Ollama.
    • The use_* fields are managed by the UI; you mostly don't touch them.

    Output: one STRING - the currently-used result, wired into your CLIP Text Encode or a string-assembly chain.

    What you need installed

    Ollama itself, separately - the node talks to a running server over HTTP, and it does not bundle or download models. Get it from ollama.com, then:

    ollama pull llama3.2        # or whichever model you want
    

    The node reads OLLAMA_HOST (default 127.0.0.1:11434). No pip step for the pack itself - it uses requests and aiohttp, both of which ship with ComfyUI. The node's model dropdown is built once at ComfyUI startup from what Ollama reports, so if you pull a new model, restart ComfyUI to see it in the list.

    Installing and troubleshooting

    cd ComfyUI/custom_nodes
    git clone https://github.com/iwanders/ComfyUI_nodes
    

    Restart ComfyUI, or search iwanders in ComfyUI Manager. Then the classic gotchas: empty model dropdown means Ollama wasn't reachable when ComfyUI started (check it's running, check OLLAMA_HOST), and a slow first queue is normal - generating the prompt adds a few seconds per cycle on top of image gen. Small models like llama3.2 are plenty for prompt-writing and keep the wait short. This is the node that turns ComfyUI into "describe the vibe, the LLM writes the prompt" - and unlike the hosted options, it costs nothing and sends nothing anywhere.

    Categoryiw

    Inputs (7)

    NameTypeDefaultDescription
    generateBOOLEANtrueWhether to cycle the result, or keep it fixed to the currently shown value.
    generate_seedINT
    generate_promptSTRING
    generate_modelCOMBO0 options:
    use_seedINT0
    use_resultSTRING
    use_promptSTRING

    Outputs (1)

    NameTypeDescription
    STRINGSTRING