llama.cpp Structured Output

If you've asked a local LLM for JSON and gotten a paragraph with a code fence and a "Sure, here's your JSON:" preamble, you know the pain this node exists to remove. Structured Output builds a decoding constraint - one of three kinds - and hands it to an ADV++ Prompt node, which sends it to llama-server so the model is constrained to produce valid output of the requested shape rather than merely asked to.

This is the difference between "the LLM tried to follow instructions" and "the LLM physically cannot output anything outside the grammar." For a pipeline that has to parse the result - a captioner feeding a JSON-driven workflow, a classifier returning a fixed schema - the constrained version is the only one you can rely on.

How it works

You pick a mode and paste a constraint, and the node produces a structured_output object for an ADV++ Prompt's structured_output input. The three modes map directly to llama-server's constraint types:

• json_schema (default) - you paste a JSON Schema object. With strict on, the model is held to it tightly. This is the right choice when you know the exact shape of what you want back, with typed fields.
• json_object - no schema, just "any valid JSON object." Good for quick experiments where you care that it's JSON, not what keys it contains.
• grammar - a raw GBNF grammar (llama.cpp's grammar format). Maximum control, steepest learning curve. You write the grammar; the model can only emit strings that parse under it.

The node validates its input - grammar mode refuses an empty grammar, schema mode checks the schema - so a bad constraint fails fast at node time rather than halfway through a generation.

Inputs and outputs that matter

• mode - the constraint type. Start with json_schema; reach for grammar only when you need exact control (and know GBNF).
• constraint - the JSON schema or GBNF text.
• enable - bypass the constraint without deleting the node. Handy when debugging a generation: flip this off to confirm the model would otherwise produce the garbage you're trying to fix.
• schema_name - the schema's name in the OpenAI-style json_schema call. Default comfyui_output; only matters if something on the server references it by name.
• strict - demand strict schema adherence. Keep it on.

Output: the structured_output constraint, wired into ADV++.

Wiring it

Structured Output → ADV++ Prompt (structured_output) → Prompt Output. In ADV++ you still pick your model and prompt, and the constraint applies to the generated response. Combine with a Token Ban node for the full reliability stack: a ban stops specific tokens, the grammar forces the overall shape.

Issues to expect

• Generation stalls or returns nothing - a grammar that contradicts what the model wants to say, or a schema that's too restrictive for the prompt. Loosen the constraint or check that mode matches what you pasted.
• Empty schema - json_schema mode with an empty constraint fails validation. Paste an actual schema.
• Schema too clever for the build - json_schema/json_object support varies a bit across llama.cpp builds; an older llama-server may not handle strict schemas. Update llama.cpp, or fall back to a GBNF grammar.