Storyboard Generator (DeepSeek)

This is the pack's headline act. 🎬 Storyboard Image → Prompt takes a storyboard frame (or any reference image) plus the lyric or script section that plays over it, and hands you back seven strings: caption, action, camera, notes, mood, dialogue, and details. The pack's README calls itself "a simple prompter for video," and this node is that pitch made concrete - one shot in, a structured scene breakdown out, ready to be assembled into per-shot prompts for a video model.

How it actually works

Read the source and it's a three-stage pipeline, which is more interesting than the one-line README suggests:

1. BLIP captions the image. It loads Salesforce/blip-image-captioning-base from HuggingFace - roughly a gigabyte, downloaded on first run and cached afterward - and produces a plain-English description. It runs on CUDA if you have it, CPU otherwise. This first load is slow; every later run uses the cache.

2. It extracts the song segment. Your extra_text input is meant to be a full lyric sheet or script with [Section] markers. The node regex-matches the current label against those markers and grabs everything from [YourLabel] up to the next [. If it can't find a match, it silently falls back to the whole extra_text, then to the label itself. Look at the default label - [Intro solo- Ambient, horror Waterphone Swell] - and you're seeing a leftover from the author's own music-video project. You'll replace it with your own section names.

3. Gemma3 turns it into JSON. The caption and the extracted segment are sent to Ollama's gemma3:latest, with instructions to reply only with a JSON object containing the keys action, camera, notes, mood, dialogue, details. The node parses that JSON and routes each key to its output, with caption coming straight from BLIP.

The inputs and outputs that matter

Inputs are simple: image (an IMAGE tensor - wire in a Load Image node or a frame from a video loader), label (the [Section] name, as a string), and extra_text (multiline - paste the full lyric sheet/script with bracketed markers). That's everything.

Outputs: caption plus the six JSON fields. Wire them into prompt assembly however your video workflow likes - action + mood + camera into the positive prompt, dialogue wherever your pipeline wants dialogue, notes/details as scene context. It's flagged as an output node, so it terminates a branch cleanly.

Installing and the gotchas

Installation is identical to the pack's other node: ComfyUI Manager search for "ComfyUI_DreamBoard", or clone and pip install -r requirements.txt. The only new Python package is ollama - torch, transformers, and friends are already in ComfyUI. Then the two things the README never says:

# Ollama server running + the model it hardcodes
ollama pull gemma3

If the node errors on the LLM step, Ollama isn't running or gemma3 isn't pulled. If it errors on the image step, BLIP is still downloading - give it time.

The fragile link is the JSON. The node begs gemma3 to answer with nothing but a JSON object, but small models wander: code fences, trailing prose, extra keys. When that happens, json.loads fails and every field except caption comes back empty. Your first instinct should be to re-run, and your second should be to rephrase extra_text so the segment is cleaner - there's no retry logic in the node. Label-matching is the other quiet failure mode: misspelled or missing brackets and the segmentation silently falls back instead of telling you.

Honest caveat, same as its sibling: the README says "UNDERCONSTRUCTION!", there's a single commit, and the default label is a personal artifact. It's an early experiment, but if you're storyboarding a music video it's also genuinely the fastest way I've seen to turn a frame plus a lyric into six usable prompt parts in one shot.