Transformer LLM Task Runner

The name is the most generic thing about this node, and the pitch is actually good: Transformer LLM Task Runner drops a full, local, causal language model - Qwen, Llama, Mistral, Gemma, whatever fits - into your workflow as a text node. No API key, no Ollama server running in a second process, no cloud calls. It's the kind of thing you bolt on for one narrow job: rewrite a rough idea into a structured prompt, prune Danbooru tags that contradict your new img2img instruction, translate, summarize, coax a clean JSON blob out of a messy caption.

First, a distinction that decides whether this node makes sense at all. The language model inside your checkpoint (T5 on Flux, Qwen3 on Z-Image) is an encoder you never touch - it runs every generation. This node is the other kind: a standalone LLM you drop into the graph as a tool, upstream of the sampler. Different thing, same name, and people conflate them constantly.

How it works

The mechanism is refreshingly boring, which is the point. It uses Hugging Face's AutoModelForCausalLM and AutoTokenizer - the exact libraries ComfyUI already ships - so it doesn't pin, downgrade, or fight your torch/transformers like some LLM nodes do. First run auto-downloads the chosen model into ComfyUI/models/LLM/, applies the model's chat template, generates greedily (do_sample=False), and returns the generated text with your prompt stripped out as a plain STRING you can wire into anything.

Its real selling point is memory discipline. After every run it deletes the model and tokenizer references, forces garbage collection, and clears VRAM via soft_empty_cache - all in a finally block, so an OOM mid-run still frees memory. It's device-agnostic (CUDA, MPS, XPU, NPU, MLU) and has smart fallbacks baked in: OOM flips you to CPU + float32, and attention falls back from sdpa to eager. The author clearly lives in the VRAM-tight end of this hobby.

The inputs that matter

• task - your actual instruction, with optional {arg0}…{arg5} placeholders. This is 90% of the node.
• model - 15 presets: Qwen2.5-7B/1.5B-Instruct, DeepSeek-R1-Distill (Qwen/Llama 8B), Llama 3.1/3.2, Gemma-2, Mistral, down to SmolLM2-135M. Anything in ComfyUI/models/LLM/ auto-appears here.
• custom_model_hf_id - paste any HF repo ID to override the dropdown.
• keep_model_loaded - False (default) unloads everything after each run for max VRAM recovery; True keeps it resident for faster re-runs.
• max_new_tokens - output cap, default 1024, bounded 1–4096.
• device_map - auto offloads layers between GPU and CPU; cpu is your low-VRAM escape hatch.

The single output is the generated STRING. Wire it into a prompt, a text-concat node, a save node, anything.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/mkim87404/ComfyUI-TransformerLLMTaskRunner.git
cd ComfyUI-TransformerLLMTaskRunner
pip install -r requirements.txt

That's accelerate and huggingface_hub - genuinely harmless, no torch/transformers pinning. Restart ComfyUI, double-click anywhere, search "Transformer LLM Task Runner". ComfyUI Manager finds it by pack title too.

Where people get burned

• It will not load GGUF, EXL2, or AWQ. If you're coming from Ollama/llama.cpp thinking "LLM = GGUF," nope - this needs plain safetensors/.bin weights with a config.json and tokenizer files. That's the #1 gotcha. Look for the "Transformers" tag on Hugging Face.
• First run is a multi-gigabyte download. Qwen2.5-7B is ~15GB. On a card that's also holding a diffusion model, budget VRAM or drop to the 1.5B Qwen - or SmolLM2-135M if you want a toy. device_map=auto will offload to CPU and crawl.
• The R1-Distill presets are reasoning models. Fine if you want the thinking chain, but for prompt rewriting they spend tokens deliberating and can leak their scratch-work into your output - the KB's standing advice is small-and-obedient for this job.
• Output isn't a jail. It strips the prompt and skips special tokens, but nothing hard-stops preamble like "Here is your enhanced prompt:" - an LLM asked for JSON may still hand you prose. Scope the task tightly.
• PyTorch's allocator fragments over many runs; a full ComfyUI restart is the only total memory reset, and that's a PyTorch fact, not a node bug.

The author's own note is worth repeating: if you need a vision model like Florence-2 or Qwen-VL, use Kijai's or 1038lab's dedicated nodes instead - this one is strictly for standard decoder-only LMs. It's a narrow, well-built tool from a small developer whose other node (ComfyUI-ControlOrder-FreeMemory) shows the same memory-first obsession. If you need a generic local text brain in the graph without wrecking your environment, it's a solid reach.