ComfyUI-Bernini-PromptEnhancer

ComfyUI-Bernini-PromptEnhancer

中文文档 | Installation | Tutorial

ComfyUI custom nodes for local prompt enhancement via GGUF multimodal models. Includes Bernini VLLM semantic planning, Qwen3.5 text enhancement, and MiniMax H3 video prompt planning (powered by Gemma 4 E4B) with multi-material context analysis. Zero cloud API.

Based on Bernini: Latent Semantic Planning for Video Diffusion (Bernini node) and the MiniMax H3 prompt specification (H3 node).

Features

• 13+n task types — 13 core tasks + specialized lightweight tasks
• Smart frame selection — smart_frames toggle uses frame-difference sampling instead of uniform spacing, giving the MLLM higher information density
• First/Last frame transition — fl2v task generates a prompt describing the video between a start frame and end frame
• GGUF local inference — runs entirely offline on consumer GPUs via llama.cpp subprocess
• Zero VRAM residue — subprocess-isolated, VRAM fully released after each run
• Dual output — enhanced_prompt + structured_plan
• Image passthrough — source_video, reference_video, reference_image_0/1/2 pass through unchanged
• Dual template mode — structured (five-section RULES) or official (single-pass), per the Bernini node
• Thinking-mode toggle — Qwen3.5 node with thinking_mode switch (off by default)
• H3 video prompt planning — multi-material context (video + image + audio), H3-standard three-section output, direct passthrough to official MiniMax H3 node

Tasks

| Task | Inputs used | Description | |------|-------------|-------------| | t2v | prompt | Text to video | | t2i | prompt | Text to image | | v2v | source_video + prompt | Enhance/edit source video | | mv2v | source_video + prompt | Multi-video edit (alias) | | i2i | prompt | Image to image | | i2v | ref_images + prompt | Image(s) to video | | ads2v | source_video + prompt | Ad placement in video | | vi2v | source_video + ref_images + prompt | Video + ref images edit | | r2v | ref_images + prompt | Reference images to video | | r2i | ref_images + prompt | Reference images to image | | rv2v | ref_video + ref_images + prompt | Ref video + ref images edit | | vrc2v | source_video + ref_images + prompt | Video + ref images edit (alias) | | fl2v | ref_image_0 + ref_image_1 + prompt | First-to-last frame transition |

Specialized tasks — for specific mixed-source scenarios, with explicit role assignment per input:

| Task | Inputs used | Role assignment | Description | |------|-------------|-----------------|-------------| | r2v_motion | source_video + ref_images + prompt | ref=character, source=motion | Ref character performs source motion in described setting |

All tasks accept an optional user prompt. When left empty, a default instruction is used.

H3 node tasks: t2v / i2v / h3_multi_ref — see H3 Prompt Planner section below for details.

Smart frame sampling

Default frame selection is uniform (evenly spaced). Enable smart_frames to use thumbnail-based frame-diff sampling: the first frame is always kept, remaining slots are filled by the highest inter-frame change magnitude. Thumbnail downscale (64x64) naturally resists noise and compression artifacts.

Uniform: 30 frames, pick 5 → frames 0, 7, 15, 22, 29
Smart:   30 frames, pick 5 → frame 0 (fixed) + the 4 frames with highest change

Controlled by video_frames (1–16).

fl2v — First/Last frame transition

Generates a prompt describing the continuous video between a start frame (reference_image_0) and an end frame (reference_image_1). Reference images are labeled START-FRAME and END-FRAME explicitly in the model input.

Installation

cd ComfyUI/custom_nodes
git clone https://github.com/djdzzzz/ComfyUI-Bernini-PromptEnhancer.git
cd ComfyUI-Bernini-PromptEnhancer
pip install -r requirements.txt

Model setup

Download GGUF quantized models into ComfyUI/models/clip/:

• Bernini-MLLM-Qwen2.5-VL-7B GGUF — *.Q4_K_M.gguf recommended
• Matching *.mmproj-*.gguf from the same repo

Qwen3.5 node (optional)

• Yusiko/qwen3.5-prompter — Qwen3.5-4B, prompt-engineering tuned, multimodal

H3 node

• Gemma 4 E4B GGUF (4.2 GB); mmproj-F32.gguf in same repo — recommended for H3 video prompt planning

Nodes

Bernini MLLM Prompt Enhancer (GGUF)

Category: Bernini

| Input | Type | Default | Notes | |-------|------|---------|-------| | model | dropdown | — | .gguf file | | mmproj | dropdown | — | .mmproj.gguf file | | task_type | dropdown | v2v | 13 tasks | | template_mode | dropdown | structured | structured or official | | prompt | STRING | "" | multiline | | temperature | FLOAT | 0.6 | 0.0–2.0 | | video_frames | INT | 3 | 1–16, frames per video | | smart_frames | BOOLEAN | false | frame-diff sampling | | image_max_side | INT | 512 | 0=original, else max side | | source_video | IMAGE | — | optional | | reference_video | IMAGE | — | optional | | reference_image_0/1/2 | IMAGE | — | optional |

| Output | Type | Notes | |--------|------|-------| | enhanced_prompt | STRING | final prompt | | structured_plan | STRING | RULES reasoning trace | | source_video | IMAGE | passthrough | | reference_video | IMAGE | passthrough | | reference_image_0/1/2 | IMAGE | passthrough |

Qwen3.5 Prompt Enhancer (GGUF)

Category: Bernini

Same inputs and outputs as the Bernini node, plus:

| Input | Type | Default | Notes | |-------|------|---------|-------| | thinking_mode | BOOLEAN | false | Qwen3.5 think mode |

H3 Prompt Planner (GGUF)

Video prompt planner for MiniMax H3. Accepts video, image, and audio as context, produces H3-standard three-section English prompts with shot-by-shot structure and integrated soundscape. Powered by Gemma 4 E4B GGUF.

Tasks: t2v / i2v / h3_multi_ref

Materials: Drag & drop videos, images, audio. Reference as <Video 1>, <Picture 1>, <Audio 1> in prompt.

FINAL_PROMPT format (H3 three-section standard):

integrated_multimodal_description: Live-action cinematic...
[Shot 1] ...camera pushes in with small amplitude at slow speed...
At 00:03.500, the camera cuts to... (S1) says: <d>[English]...</d>...

overall_soundscape: Steady rain on cobblestone, distant traffic hum...

non_diegetic_music: N/A

Passthrough: media_0–media_14 (JS renames to ref_video_0, ref_image_0, ref_audio_0 in ComfyUI). Wire directly into official MiniMax H3 Reference to Video node inputs.

| Input | Type | Default | Notes | |-------|------|---------|-------| | model | dropdown | — | GGUF file, auto-scanned | | mmproj | dropdown | <none> | Vision projector (required for image/video) | | task_type | dropdown | h3_multi_ref | t2v / i2v / h3_multi_ref | | prompt | STRING | "" | Multiline; use <Video 1> etc. | | temperature | FLOAT | 0.6 | 0.0–2.0 | | repeat_penalty | FLOAT | 1.15 | 1.0–2.0 | | seed | INT | 0 | 0 = random | | n_ctx | INT | 8192 | Context window | | n_gpu_layers | INT | -1 | -1 = all GPU | | max_tokens | INT | 2048 | Max output tokens | | frame_mode | dropdown | uniform | uniform / smart | | video_frames | INT | 5 | 1–64, frames per video | | sample_fps | INT | 0 | 0 = use video_frames | | sample_seconds | FLOAT | 5.0 | Max duration for sampling | | image_max_side | INT | 512 | 64–4096 | | thinking_mode | BOOLEAN | false | Qwen3.5 think mode |

| Output | Type | Notes | |--------|------|-------| | enhanced_prompt | STRING | H3-standard three-section prompt (FINAL_PROMPT only) | | structured_plan | STRING | OBSERVATION–PRESERVE planning trace | | media_0–14 | IMAGE/AUDIO | Material passthrough (JS renames to ref_video_N etc.) |

Models

| Quant | Size | VRAM | |-------|------|------| | Q4_K_M | ~4GB | ~5GB | | Q8_0 | ~8GB | ~9GB | | bf16 | ~14GB | ~14GB |

Acknowledgements

• Bernini-Diffusers — semantic planner
• Bernini MLLM GGUF — GGUF quantization
• MiniMax H3 — omni-modal video generation model (open weights)
• Gemma 4 E4B GGUF — H3 planner backbone
• ComfyUI
• llama-cpp-python

License

Apache-2.0 (see LICENSE)