Nougan Layout LLM Prompt 🧱

The name is slightly misleading in a way that matters: this node doesn't call any LLM, doesn't need an API key, and doesn't produce boxes. It writes the two strings - a system prompt and a user prompt - that make a language model answer with bounding-box JSON that Nougan Bounding Boxes can read. It's a prompt-engineering node with the prompt engineering already done.

You can hand-compose a layout in the Bounding Boxes canvas, but if what you have is a sentence ("a poster with SUMMER SALE at the top and a parrot at the bottom middle"), turning that into coordinates is the annoying part. A small local LLM is genuinely good at that conversion - it's really a text-to-structure problem.

What it actually builds

The system prompt is a template. It states your canvas size and classifies the shape (landscape if width is 15% past height, portrait the other way, otherwise square), then explains that coordinates are integers on a 0–1000 grid, not pixels - 0 is the top/left edge, 1000 the bottom/right. It spells out the box schema:

• type: "obj" for people, animals, objects, scenery; "text" for words that must be written into the image
• bbox: [ymin, xmin, ymax, xmax] - y first, which trips people up
• text: only for text boxes, the exact words with the original capitalisation
• desc: a short visual description

Then it adds composition rules - put "top" at a small ymin, draw boxes tightly, keep text boxes wide and short, list background items before foreground ones - plus "answer with the JSON array only, no markdown, no code fences" and a worked example. Whatever you put in extra_rules is appended as an extra bullet, which is how you get "keep the bottom 15% empty for a caption".

The user prompt is deliberately boring: your scene text plus "Return the JSON array now." All the intelligence is in the system half.

Inputs and outputs

You set four things. scene is the description - be spatial and literal, because that's what it's parsing. width and height should be the same numbers you give Nougan Bounding Boxes and your empty latent; they're baked into the prompt text, so a mismatch between the prompt and the actual canvas slides every box. max_boxes (1–16, default 5) caps the answer; ask for more than four or five and small models start hallucinating overlaps.

extra_rules is the one optional field, multiline, and it's where you actually tune this thing.

Outputs are system_prompt and prompt, both STRING. Wire them to the matching inputs of a TextGenerate (with a CLIP loader for whatever text encoder you run) or to any LLM node you like - LM Studio, Ollama, whatever - and wire that node's text output into Bounding Boxes' bboxes input. That's it.

Install

Part of the Nougan pack:

cd ComfyUI/custom_nodes
git clone https://github.com/Winnougan/Nougan_Nodes
# restart ComfyUI

or ComfyUI Manager, search Nougan. There's nothing else to install here - no model, no dependency. The LLM is your problem, and it doesn't even have to be local.

Where this breaks, and why

The LLM wraps the JSON in a code fence. This is the classic failure of asking a chat model for structure, and it's the single most common reason the boxes never appear - the pack's own docs tell you to look for no JSON found in the console. The system prompt explicitly forbids fences and preambles; smaller models ignore it anyway. Lower the temperature to 0–0.2 and, if it still fences, pick a model that follows instructions better. It's the same problem the whole "LLM in the graph" category fights (llm-in-comfyui.md has the long version: a chat model emits chat, and chat bleeds into your data).

The layout drifts every run. Different answer, different boxes. Fix the seed on the sampler inside your TextGenerate. Once you're happy with a layout, disconnect bboxes so you stop re-rolling it.

The model ignores the boxes. Layout control inside the checkpoint is experimental - some models respond to box-conditioned prompts, most respond loosely, and a prompt that contradicts your boxes wins. ControlNet-style structural control this is not; it's prompt-side hinting (controlnet.md for what real spatial conditioning looks like).

Text boxes render as objects. The box type defaults to object; when you import JSON from an LLM, check that text entries arrive as "type":"text" with the words in text, or you'll get a picture of a sign instead of a sign that says SUMMER SALE.

My honest read: this is the right shape for the problem - prompt assembly as a node rather than a wall of text in a README - and it's a five-minute setup if you already run a local LLM. If you don't, hand-placing boxes in the canvas will be faster than standing up an inference server for one poster layout.