ComfyUI Node

Grok Chat

The VLM That Refuses Less Than the Others

By yogurt7771·Created 2 years ago·Updated 6 days ago· 3
Grok Chat
  • images
  • history
  • provider
  • text
  • thought
  • history
◄prompt►
◄model▾►
◄seed0►
◄image_detailauto►
◄extra►
◄retry_count1►
◄system_prompt►

Here's the practical case for this node, and it's the same case that makes Grok relevant in this ecosystem at all: xAI's models are the least restrictive of the closed frontier. If you're captioning a set that makes GPT's filter nervous, or you want a prompt rewritten without the rewrite being moralised at, this is the API you try before you give up and go fully local.

Now the honest half. "Least restrictive" is a relative statement about closed models, and everything about a closed endpoint still applies: no weights to patch, no abliteration, the vendor's policy at the far end of the wire. The absolute floor for anything the hosted services won't touch is still a local model on your own card.

What the node does

Text in, text out, images optionally in. Prompt enhancement, captioning, dialogue writing - the three standard jobs, same shape as the Gemini and GPT chat nodes in this pack. Under the hood it resolves a route from the Grok family for whichever provider you've connected (xAI by default), sends the request, and converts the reply. Nothing local runs.

Inputs

  • prompt - multiline, the user turn.
  • model - dynamic dropdown across the Grok line. Inputs change with your pick.
  • seed - 0 = provider's choice; changing a non-zero value forces a re-run.
  • image_detail - how hard the vision path looks at connected images. Defaults to auto; bump it for fine detail you need read accurately, and remember heavier detail means more tokens.
  • images - autogrow, up to 16, expanded per-image. Vision is the reason to use this node for captioning rather than a plain text model.
  • system_prompt - your format enforcement. "Return only the caption, comma-separated descriptors, no sentence." This field is where caption quality is won or lost.
  • history - previous turns for continuation.
  • extra - JSON merged into the request.
  • retry_count - total attempts, ~3s apart.

Outputs: text, thought, history. thought holds reasoning text where the route reports it and is empty otherwise.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/yogurt7771/ComfyUI-YogurtNodes
cd ComfyUI-YogurtNodes
pip install -r requirements.txt

Manager installs it too - search "YogurtNodes". On the canvas, display names carry a " (Yogurt Nodes)" suffix.

Keys: without a provider connected the node uses the official xAI API, reading xai (or grok) from custom_nodes/ComfyUI-YogurtNodes/yogurt_nodes/llm/api_key.json, then XAI_API_KEY (or GROK_API_KEY). There's also an xai_base_url fallback if you need a non-default host.

{ "xai": "xai-..." }

Note the environment-variable aliasing - both names work, which is a kindness if you already had one exported under the other.

Where it earns its keep

Captioning. A VLM writing descriptions for a LoRA set, especially NSFW material, is the use case this node is genuinely best at. Feed images in, ask for natural language, keep the multi-subject caveat in mind: every VLM of this generation mixes up who is doing what when there are two people in frame, so audit small sets by hand.

Prompt enhancement with attitude. A model that doesn't flinch at "cinematic, sweaty, hyperreal" is a model that writes you a better prompt for that image.

Chained pipelines. history out feeds history in on another chat node, so the describe-then-rewrite pattern - Grok captions, a second model restructures for your chosen encoder - is a two-wire build.

The usual bruises

Preamble and role text in your output. A chat model chats. Constrain with system_prompt, strip with a regex node. Same fix as every other chat node in the pack.

Empty text, full thought. Reasoning went to the reasoning channel. Read the right output or dial the effort down.

An image batch blowing the slot ceiling. Over 16 expanded images and the node errors out. Batches count one per image, not one per wire.

Legacy confusion. The pack also ships Grok Generate Text / Generate Image / Image Understand nodes under YogurtNodes/LLM/Legacy with a "(Legacy)" suffix and their own key handling. Old workflows keep working there; new ones should use this node plus a provider. If a tutorial tells you to paste a key into a Grok node and you can't find the field, you're on the wrong one.

CategoryYogurtNodes/LLM/Text

Inputs (10)

NameTypeDefaultDescription
promptSTRINGPrompt sent to the model.
modelCOMBOModel version; the inputs below change with the selected model.
seedINT00–2147483647Sent to the API when the model supports it; changing it also forces a re-run. 0 lets the API pick a random seed.
image_detailCOMBOautoImage detail level for vision.
extraSTRINGOptional JSON object merged into the request (for parameters not exposed as inputs).
imagesCOMFY_AUTOGROW_V3Optional reference image(s); batches are expanded. Up to 16.
retry_countINT11–10Total attempts on failure.
system_promptoptSTRINGSystem instruction sent with the request.
historyoptHISTORYPrevious turns.
provideroptYOGURT_LLM_PROVIDERProvider node. When unconnected, the official API is used with keys from api_key.json or environment variables.

Outputs (3)

NameTypeDescription
textSTRING—
thoughtSTRING—
historyHISTORY—