TTP Smart Tile QwenVL Prompt Set Builder

Once Interactive Crop hands you eight or sixteen variable-size tiles, someone still has to write a prompt for every single one of them - "close-up of an eye, sharp detail" for the eye tile, "weathered denim jacket texture" for the clothing tile, and so on. Doing that by hand for every tile on every image gets old fast. This node hands that job to a local QwenVL vision-language model instead: it looks at your tiles and writes the per-tile prompts itself.

Why a VLM instead of one global prompt

Using vision-language models to describe or ground specific regions of an image rather than write one caption for the whole frame has become a standard move across the ComfyUI ecosystem - it's the same idea behind pointing a VLM at an image to name and box a specific object before inpainting it, and Qwen's own vision-language line (the Qwen2.5-VL / Qwen3-VL family) is already doing double duty as the text encoder inside several 2026-era image models. This node applies that pattern to tiling specifically: rather than one prompt trying to describe a face, a hand, and a background all at once, each tile gets its own, generated from what's actually in that crop.

The moving parts

reference_image_mode decides how much visual context the model gets: none (text only), first_message (the full source image shown once, then tiles described relative to it), every_tile (one model call per tile - most accurate, slowest), or contact_sheet (the default - one grid image of every tile at once, a single call). system_prompt, tile_instruction, and global_prompt are three layers of instruction stacked on top of each other - overall behavior, per-tile task, and words that should land in every generated prompt regardless of tile content. prompt_merge_mode controls how those pieces actually get stitched together into the final text: caption_only, global_plus_caption, or global_plus_label_plus_caption (the default, which folds in the tile's semantic label too - so a face tile's prompt carries "face" plus your global style words plus what the model actually saw).

output_language is english/chinese/bilingual. max_new_tokens and temperature are standard generation controls - temperature defaults to a low 0.2, which is deliberate: you want the same tile to caption the same way on a rerun, not a different flavor of prose every time. prompt_preset swaps in a whole different instruction template - tile_img2img_prompt is the one you want for this loop, tile_caption_only for a plainer description, tile_json_strict for structured output, bbox_detect for bounding-box detection instead of captioning, style_material_lighting for a narrower focus. qwen_max_side/qwen_max_pixels cap the image size fed to the model - useful for VRAM and speed. use_tile_cache (on by default) is worth knowing about: it caches results by model file, tile hash, prompt, and seed, so rerolling a workflow doesn't re-run the VLM on tiles that haven't changed. global_negative appends to every tile's negative, qwen_seed seeds the generation.

Two optional inputs actually make this node function: tile_set (from Interactive Crop) and qwen_vl_model, which has to come from a separate TTP QwenVL3 Local Loader node - Interactive Crop and this node don't read .safetensors files directly. reference_image is optional too, for supplying your own context image. Outputs: tile_set (now carrying a prompt per tile), prompt_set_json (the full dump), summary.

Installing it - and the model file you'll need

Pack install is the usual ComfyUI Manager search for "Comfyui_TTP_Toolset", or:

cd ComfyUI/custom_nodes
git clone https://github.com/TTPlanetPig/Comfyui_TTP_Toolset

This is the one node in the pack with a real dependency: it needs a QwenVL model file dropped in ComfyUI/models/text_encoders (the README's own example is qwen3vl_4b_fp8_scaled.safetensors), loaded through TTP QwenVL3 Local Loader and fed into this node's qwen_vl_model input. Budget the VRAM and disk space for a multi-gigabyte local vision-language model - it's not optional if you want this node to do anything.

Where people get tripped up

If nothing gets captioned, the first thing to check is that qwen_vl_model is actually connected - the loader has to sit between the file on disk and this node, always. If reruns feel painfully slow, confirm use_tile_cache is on; it's the difference between re-captioning your whole tile set every reroll and reusing what hasn't changed. And if output_language doesn't match what came out, double-check the enum rather than assuming it inferred from your prompt text - it doesn't.