Nodes/ComfyUI-Henjicc · 痕继痕迹/Qwen3-VL Prompt Optimization Assistant
ComfyUI Node

Qwen3-VL Prompt Optimization Assistant

Qwen3-VL rewrites your prompt off a contact sheet, and tells you what it can't see

By henjicc·Created 23 days ago·Updated 21 days ago· 1
Qwen3-VL Prompt Optimization Assistant
  • media
  • clip
  • text
  • visuals
  • media
◄taskauto►
◄frames_per_second2►
◄segment_seconds10►
◄analysis_seconds0►
◄max_tokens512►
◄pixel_budget_mp16►
◄custom_instruction►

Prompt enhancers are the easiest node to build badly. You hand a chat model your rough idea, it hands back three paragraphs of preamble and a synonym for "cinematic," and that garbage goes straight into your conditioning. The good ones constrain the job, strip the chat scaffolding, and keep your reference tags intact.

HenjiccVisualPrompt - Qwen3-VL Prompt Optimization Assistant, Henjicc/Text - is in the good category, and it does one thing I wish every assistant did: its second output is the actual images it looked at. Preview them and a bad rewrite shows itself in ten seconds.

The contact-sheet mechanism

Qwen3-VL is an image encoder here, not a video model, so video gets sampled into sheets. Each video is decoded at 2 or 4 frames per second, split into segment_seconds chunks (5–10), and each chunk is composed into a grid with timestamps drawn outside the frames - never over a face - plus a caption telling the model the segment number, the real sampled range in seconds, and that different sheets from one video are different moments, not simultaneous views. The last frame is always included. Total pixels across all sheets and images are held under pixel_budget_mp (default 16) by downscaling the sheets, not by dropping them.

Then it calls ComfyUI's own CLIP object directly: clip.tokenize(template, images=..., skip_template=True) with a hand-built chat template, and clip.generate(..., do_sample=False, repetition_penalty=1.05). Greedy decode, </think> blocks stripped, and if generation hits max_tokens without an end token it raises "Output truncated" rather than returning a half sentence.

The model has to be a full Qwen3-VL - a Qwen3VLTokenizer behind the CLIP, which the host's Load CLIP loader gives you for stable_diffusion-style Qwen3-VL or for Qwen-Image 2.1's encoder (the two can share a loader). Point it at MiniMax H3's 32B encoder and it refuses, with the reason spelled out: that encoder is layers 0–50 of 63 with no final norm and no lm_head, so there is nothing to generate with. That error message alone is worth the install.

The inputs that matter

  • media and clip - the bundle and the vision model. Nothing runs without both.
  • task - auto by default, and auto is clever: it walks the downstream graph and picks. A Qwen-Image text encoder → qwen_auto (edit prompt if you have references, text-to-image prompt if you don't). An H3 role-replacement node → h3_swap, with a specialist instruction set that knows to write <Picture N> instead of <imageN>, to keep the source video's action and camera, and to describe 2–4 concrete facial features rather than "same face." Ambiguous graph → general.
  • frames_per_second (2 or 4) and segment_seconds - sheet density. More fps and shorter segments means more sheets means more pixels; the budget decides how legible they end up.
  • custom_instruction - required when task is custom, up to 12,000 characters.

The outputs

text is the rewritten prompt. visuals is the list of images and video sheets actually sent - preview them when output looks hallucinated. media is your same asset bundle with the prompt replaced by the rewrite, so it can loop back into Super Input-derived nodes without you copying text by hand.

One genuinely thoughtful guard: in the h3_swap task, if the rewrite drops any <Picture N>/<Video N> reference your original had, the node logs a warning and returns your original prompt instead of the mangled one. Silent reference loss is the top way these workflows break.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/henjicc/ComfyUI-Henjicc.git ComfyUI-Henjicc
python -m pip install aiohttp av numpy Pillow

Manager: comfyui-henjicc. Restart and refresh. The pack ships no weights; the Qwen3-VL comes from your CLIP loader, which means you need a ComfyUI whose comfy.text_encoders.qwen3vl exists - the node imports it at execution time.

Limits worth knowing before you rely on it

It cannot hear. Audio items are skipped, video soundtracks are never opened, and the system prompt says so explicitly. If the audio matters, use HenjiccOmniPrompt instead.

Frames are not motion. The prompt tells the model it can only judge motion from sampled frames and should say "static" when nothing changes - honest, but it means fast action gets summarised, not understood.

analysis_seconds does nothing now. It's kept for workflow compatibility, min 0, and the sheet builder ignores it in favour of whole-video coverage from segment_seconds. Don't tune it expecting a change.

Read the output. Multi-subject attribution is the standing weakness of every VLM in this ecosystem, and an enhancer that "improves" your prompt can invent detail you never asked for. It's a first draft, not an authority.

CategoryHenjicc/Text

Inputs (9)

NameTypeDefaultDescription
mediaHENJICC_MEDIA—
clipCLIP通过原生 CLIP 加载器选择完整 Qwen3-VL;可共享千问 2.1 编码器。H3 专用 50 层编码器不能直接生成文本。 / Select a full Qwen3-VL with Load CLIP; Qwen-Image 2.1 can share it.
taskCOMBOauto7 options: auto, describe, qwen_auto, h3_swap, h3_reference, general, +1
frames_per_secondCOMBO22 options: 2, 4
segment_secondsINT105–10—
analysis_secondsINT00–60—
max_tokensINT51264–2048—
pixel_budget_mpoptFLOAT164–24—
custom_instructionoptSTRING—

Outputs (3)

NameTypeDescription
textSTRING—
visualsIMAGE—
mediaHENJICC_MEDIA—