Kosmos2 Sampler Simple2

This is the pack's showpiece, and it earns it. Feed it an image and it returns three things at once: a plain-text description of the scene, a comma-separated list of the objects it found, and your original image with colored bounding boxes and labels drawn over every detected object. It's the node that closes the loop the pack is named for - "look at this picture, then write a story about what's in it" - and the boxed-up image is why you can see it working.

Mechanically it runs Kosmos-2 (loaded by KosmosLoader2) with its grounding mode switched on: the code prepends the <grounding> token to your prompt, generates up to 128 new tokens, then splits the result into a caption and a list of (entity, bounding box) pairs. The entity names get joined into the keyword output; the boxes get drawn onto a copy of the image with OpenCV, each entity getting its own color and a small label. If you've used Kosmos-2 for training-data captioning you'll recognize the whole thing - it's that, wrapped for ComfyUI.

Inputs that matter

• image (IMAGE) - the picture to analyze. From a LoadImage node.
• model / processor (CUSTOM) - both from KosmosLoader2.
• prompt (STRING, forceInput) - the instruction. Can't be typed in; wire a text node like the pack's Write2 (the example workflow uses Write2 holding describe the image.).
• strip_prompt (BOOLEAN, default true) - strips your prompt back out of the generated text so the description doesn't start by echoing your instruction. Leave it on.

Outputs - this is where the value is

• description (STRING) - Kosmos-2's caption of the image.
• keyword (STRING) - the grounded object names as one comma-separated string. This is the one that feeds the story loop: in the example workflow it plugs straight into StorySamplerSimple's prompt, so the story gets written about the actual contents of your image.
• image (IMAGE) - your original with entity boxes and labels drawn on. Wire it to a PreviewImage and you get a free visual sanity check before you trust the keywords.

You can also lift the whole pipeline as a captioning tool: image in, description out, feed it to a captioner or training set. Kosmos-2's grounded captions were used by real people for SDXL training data precisely because the entity grounding is reliable - the community signal for that is genuine.

Installing it

Pack install plus the dependency catch:

cd ComfyUI/custom_nodes
git clone https://github.com/oztrkoguz/ComfyUI_StoryCreator

then restart - or search "ComfyUI StoryCreator" in ComfyUI Manager. The pack ships no requirements.txt but this module imports transformers and cv2 at load time, so if either is missing from your ComfyUI Python env the entire pack fails to register. Install both:

pip install transformers opencv-python

Common issues

• GPU required. The sampler hardcodes .to("cuda") on its inputs. There is no CPU path no matter what KosmosLoader2's device dropdown says. This is the single most common crash.
• Big first run. Loading Kosmos-2 (about 1.7 GB) plus the story model and the SDXL checkpoint means the full loop's first run is mostly downloads. The console's "loading model" line is your friend.
• Entities can be weird. Kosmos-2 is a 2023 model; small or abstract objects sometimes get named oddly or missed entirely. That's why the annotated image output exists - look at the boxes before you let the keywords drive a story.
• Whole pack missing? Check the cv2/transformers import first (see above). If Write2 shows up but this node doesn't, it's a dependency problem, not an install problem.

The honest take: this is a personal, lightly-maintained pack (last touched spring 2025, basically zero community footprint), so expect rough edges and hardcoded CUDA. But the core idea is a good one - grounded image analysis feeding a story generator is a fun, genuinely different workflow, and seeing the boxes drawn on your image makes it satisfying to debug. Just bring a GPU and a few GB of patience for the first run.