HF Text Generation

You want a language model in your ComfyUI graph but you don't want to run a 7B model on your machine. That's the whole pitch of HF Text Generation. It's a node from the bitaffinity/ComfyUI_HF_Inference pack that sends your text to Hugging Face's hosted Inference API and gets generated text back - which means the LLM runs on their servers, not yours. No GGUF files, no VRAM, no llama.cpp hunting.

What it does

Two inputs:

• endpoint (STRING) - the model ID, like HuggingFaceH4/zephyr-7b-beta (the README's pick), or a full URL.
• text (STRING, multiline) - your prompt or partial text.

It POSTs {"inputs": text} to https://api-inference.huggingface.co/models/{endpoint}, and the server returns generated text. The node concatenates everything the model produces and hands it out its STRING socket.

That STRING is the useful part. Wire it into a text display node to read it, or - the actually interesting move - into a CLIPTextEncode's text input so the LLM's rewrite becomes the prompt that reaches your sampler. That's prompt refinement without a local LLM: "rewrite this as a vivid scene description," then feed the result into SD. It's the sort of thing the "LLM prompt enhancer" ecosystem does, compressed into one remote call.

What you don't get to control

The node exposes no sampling knobs. No temperature, no max_new_tokens, no stop strings - it sends inputs and takes whatever the endpoint's defaults give it. If you need deterministic, short completions, you're at the mercy of the model's configured defaults, and that's a real limitation for automation. It's also a completion-style call rather than a proper chat-templated session, so treat it as "continue this text," not "hold a conversation."

The size warning is real

The README notes the serverless API only supports models 10GB or below "and fails for random reasons on different models." zephyr-7b-beta is roughly 14GB in bf16 - over that line on paper - but it's the author's own suggestion, and Hugging Face serves it in reduced precision, so it's the endorsed path anyway. Expect it to work, and equally expect occasional odd failures on whatever model you try next. The pack also waits out cold starts: first call to a cold model can take a couple of minutes while HF boots the container, and the node sleeps the reported estimated_time and retries once before it gives up.

Installing it

Identical to the rest of the pack, and it's one of the lightest installs in the ecosystem - the only Python dependency is requests:

git clone https://github.com/bitaffinity/ComfyUI_HF_Inference custom_nodes/ComfyUI_HF_Inference
cd custom_nodes/ComfyUI_HF_Inference
pip install -r requirements.txt

Restart ComfyUI, or search "ComfyUI_HF_Inference" in ComfyUI Manager. Then every node in the pack needs a token in an environment variable - a free Hugging Face token is fine:

HF_AUTH_TOKEN=hf_yourtokenhere python main.py

If you want a working LLM-in-the-graph node without touching this pack's quirks, there are heavier, actively-maintained local options - but those cost you a few gigabytes of VRAM. For a zero-footprint text generation call, this is the one you reach for.