LLM Provider: Textgen

LLM Provider: Textgen is the pack's specialist: a dedicated provider for oobabooga text-generation-webui, with the VRAM juggling folded straight in. Where the generic OAI Compatible node auto-detects whatever is at the URL, this one assumes Textgen (default http://localhost:5000), skips the fingerprinting, and - here's the point - can load and unload the model for you around each generation.

That's the feature that makes this the node to reach for if Textgen is your backend, and it matters because of a specific problem: your LLM and your diffusion model are now competing for the same VRAM. The KB's LLM essay (llm-in-comfyui.md) is blunt that this is the load-bearing cost of running a chat model next to a sampler - which is exactly why the good nodes automate unload/reload rather than holding both resident. That's what manage_model_memory does.

The inputs that matter

• url - Textgen's address, default http://localhost:5000.
• model - a dropdown fed from Textgen's own model list. The pack calls Textgen's internal endpoint GET /v1/internal/model/list first (the same list the Textgen UI shows), falling back to GET /v1/models if needed, so what you pick here is what Textgen itself would offer. There's a Refresh Models button and a read-only loaded line showing what's in memory.
• manage_model_memory - the star. ON (default) means the adapter calls Textgen's POST /v1/internal/model/load before the first generation and POST /v1/internal/model/unload after the last one in a chain. You get "queued a generation, model loads itself" with no separate load control.
• model_fallback - a STRING input that overrides the dropdown when connected, handy when the backend is offline.

Output is a provider (LLM_PROVIDER) socket feeding any generation node.

Why this beats the generic provider for Textgen

The README's migration advice is straightforward: if you currently have OAI Compatible + LLM Lifecycle: Textgen wired together, swap them for this one node with manage_model_memory ON. Same VRAM behavior, simpler graph, and the lifecycle node becomes legacy - it still works in old workflows, but the provider now carries that behavior itself. Fewer moving parts to mis-wire.

Keys and the Textgen gotcha

Textgen can enforce two separate secrets: --api-key guards chat completions, and --admin-key guards the internal model-load/unload routes. The pack mirrors that split in config.yaml under providers.text_gen_webui as api_key and admin_key - if you only set one flag in Textgen, put the same value in both fields. No key ever lives in the workflow JSON; it's resolved from config or environment at request time (the security-conscious design external-api-nodes.md argues for).

Install

Pack standard: ComfyUI Manager search "comfyui-llm-bikeshed", or clone into custom_nodes/ and install the (nearly empty) requirements - only pyyaml and requests, both already in ComfyUI core, no model downloads.

Where people get burned

Model doesn't load when you queue the graph? Make sure manage_model_memory is actually ON - the whole load/unload mechanism keys off it, and with it OFF the adapter just sends chat requests at whatever Textgen happens to have loaded. And if Textgen was started with --api-key, don't be surprised by 401s until the key lands in config.yaml; the README's verified notes (in the repo's docs/research/) are the ground truth for which route needs which key.