Text to Pose

Pose-controlled generation normally works backwards from a picture: find a reference photo, run it through an OpenPose or DWPose preprocessor, feed the skeleton to ControlNet. Text to Pose flips that whole pipeline. You type a dancer jumping with arms raised above their head, it generates the pose map directly from the words. No reference image to hunt for, no preprocessor node, no hoping a random stock photo happens to have the exact pose you want. That's the entire pitch, and it's basically the only thing this pack does - so if that sounds useful, read on.

The trick under the hood is that this isn't an image model at all. It's the T2P Transformer from the "From Text to Pose to Image" paper (NeurIPS 2024 Workshop), which samples human poses conditioned on CLIP text embeddings. The node tokenizes your prompt with CLIP's text encoder, grabs the second-to-last hidden state as the conditioning signal, then calls model.generate() to predict bounding boxes and then keypoints. The result is converted to DWPose format - the same keypoint layout ControlNet's pose preprocessors emit - and drawn onto a canvas. That's why the output drops straight into any OpenPose/DWPose ControlNet without conversion. One genuinely neat behavior: the number of people is inferred from the prompt, up to five. "two people dancing" gives you two skeletons.

The inputs that actually matter

Most of the fields are self-explanatory, but the ones you'll actually touch:

• prompt - the whole game. Describe a person and a pose. The README's examples ("a woman sitting cross-legged on the floor", "a man running to the left") are the right shape.
• width / height - canvas the pose is drawn on, 256–2048, default 1024. Keep it square-ish unless you know your ControlNet wants a ratio.
• seed - set it if you want a reproducible pose.
• bbox_temperature / pose_temperature - sampling randomness. Defaults are 0.1, which is fairly deterministic. Bump to 0.3–0.5 for variety; above 1.0 the README's own warning applies: poses get weird.

The outputs and where they go

pose_image (an IMAGE) is the one you want: it wires into Apply ControlNet → image, alongside any OpenPose/DWPose ControlNet - control_v11p_sd15_openpose for SD 1.5, controlnet-openpose-sdxl-1.0 for SDXL - and that feeds the KSampler. Start with ControlNet strength around 0.8 like the README suggests, then nudge. The other output, pose_keypoints, is a raw keypoints dict (bodies/faces/hands plus canvas size) for advanced or scripted workflows; 95% of users ignore it.

So the minimal graph is: T2P Model Loader → Text to Pose → Apply ControlNet → KSampler → VAE Decode.

Where people get burned

• First run is slow and looks hung. It downloads the ~150 MB T2P model and the CLIP text encoder it tokenizes with. The console shows [T2P] progress lines; the node isn't frozen, it's just fetching.
• Multi-person counting is inferred, not guaranteed. "a group of friends" can guess wrong. Be explicit: "three people standing in a line".
• The pose map is a skeleton, not a finished image. All the shading, clothing, and setting still come from your checkpoint and prompt; the ControlNet just holds the structure.
• A bare git clone isn't a complete install - this pack needs an extra module that install.py sets up. Details in the T2P Model Loader article.

Installing it

Easiest way is ComfyUI Manager → Install Custom Nodes → search "Text-to-Pose" → install → restart. Manager runs the pack's install.py for you, which clones the underlying text-to-pose library and links it up. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/logicalor/comfyui_text_to_pose
cd comfyui_text_to_pose
python install.py      # don't skip this - it sets up the t2p module
pip install -r requirements.txt

Then restart ComfyUI. Requirements are torch ≥2.0, transformers, huggingface_hub, numpy, and Pillow - the usual suspects, almost always already in a working ComfyUI environment.

One caveat to go in with open eyes: this is a research-adjacent, low-traffic pack - its registry entry still carries the author's YOUR_USERNAME placeholder, and there's basically no community discussion to lean on. It's an academic model wrapped in a working thin client. When it works it's genuinely cool; when it misbehaves, you're mostly on your own. For the pose-from-text niche it's still the cleanest way I've seen in ComfyUI.