Nodes/fexli-util-node-comfyui/FEGenStringBCDocker
ComfyUI Node

FEGenStringBCDocker

Prompt generation against a local LLM in Docker

By fexli·Created 3 years ago·Updated about a year ago· 3
FEGenStringBCDocker
  • prompt_list
  • prompt
  • generate_ctx
user_promptHello World!
temperature0.50
repetition_penalty1.05
max_new_tokens2048
top_p0.85
top_k5
async_inferNO

FEGenStringBCDocker is a prompt-generation node that turns a user prompt into a richer prompt string using a local LLM - one that runs in a Docker container, "BC" being the author's shorthand for that setup. You type a rough idea, the node streams back a full prompt, and you feed the result straight into your CLIP text encode. It's the "prompt expander" pattern that's been popular in ComfyUI since the first ChatGPT prompt-gen nodes showed up in late 2023, just pointed at a self-hosted model instead of OpenAI.

Mechanically it's straightforward: the node POSTs your prompt to the API in bc_docker_api from config.yaml (default http://127.0.0.1:8009/generate), using HuggingFace TGI's streaming /generate format - inputs plus parameters with max_new_tokens, top_p, top_k, repetition_penalty, do_sample, stream. Responses come back over SSE and the node accumulates the tokens into the final string. If you chain FEBCPrompt conversations into the optional prompt_list, it sends the full history instead of a single line.

The inputs that matter

  • user_prompt - your rough idea. Defaults to "Hello World!", so you'll always type over it.
  • temperature (0.5), top_p (0.85), top_k (5), repetition_penalty (1.05) - the sampling knobs, all standard.
  • max_new_tokens - 2048 max, the ceiling on how long a generated prompt can be.
  • async_infer - YES/NO. This is the one that changes behavior: NO (default) blocks until the LLM finishes streaming and returns the full prompt. YES returns an empty string plus a generate_ctx (type BCD_GEN_CTX) that captures the request for later - that's the hook for the pack's batch node.

Outputs: prompt (STRING) and generate_ctx (BCD_GEN_CTX).

The catch is the same one that shadows everything "BC" in this pack: the node is hard-wired to the author's specific TGI-style container, and the <C_Q>/<C_A> chat markers in the request are a bespoke template. Point it at a generic TGI server and the responses may come back garbled, because the model was tuned for that marker format. In practical terms, unless you're running the BC docker image this node was written against, you'll get connection errors or nonsense.

Where it's not a trap: if you do run that setup, the node is solid - streaming output, retries with backoff on 5xx and 429, progress printed to the console. For everyone else, this is the "written for the author's own infra" corner of the pack. The friendlier cousin is FEGenStringGPT, which points at any OpenAI-compatible endpoint you already run - that's the one I'd reach for.

Categoryfexli/generate

Inputs (8)

NameTypeDefaultDescription
user_promptSTRINGHello World!
temperatureFLOAT0.500–2
repetition_penaltyFLOAT1.050–10
max_new_tokensINT20480–2048
top_pFLOAT0.850.01–2
top_kINT50–100
async_inferCOMBONO2 options: YES, NO
prompt_listoptPROMPTS

Outputs (2)

NameTypeDescription
promptSTRING
generate_ctxBCD_GEN_CTX