comfyui-krea2-visualref

ComfyUI-Krea2-VisualRef

Experimental style-reference conditioning for Krea2 in ComfyUI.

This plugin does not implement Krea's official product-side style reference system. It uses Krea2's Qwen3-VL-4B conditioning path to test whether reference images can influence Krea2 generations through a combination of extracted style text and visual tokens.

Nodes

Krea2 Visual Reference Encode

Inputs:

• clip: a Krea2 text encoder loaded with ComfyUI CLIPLoader type krea2
• prompt: the generation prompt
• style_image: optional reference image. If omitted, the node falls back to ComfyUI native clip.tokenize(prompt) text conditioning.
• strength: visual-reference strength from 0.0 to 1.0, with decimal values supported. 0.0 uses native text-only conditioning; 1.0 is full visual-reference weight. visual_only uses a 2.5 power response curve (effective_strength = strength^2.5) so mid values reduce visual dominance. auto_style_text_and_visual keeps the text style lock at strength=1.0 and applies the direct strength value to the visual-token conditioning.
• style_text_mode: auto_style_text_only, auto_style_text_and_visual, or visual_only. The default auto_style_text_only first splits a batched style_image into individual reference images, asks Qwen3-VL to describe each transferable style with fixed structured fields, adds algorithmic palette hints computed from each image tensor, then runs a second text-only Qwen3-VL pass to synthesize one stronger TRANSFERABLE STYLE LOCK for conditioning. The style-description length and seed are fixed internally for deterministic behavior. auto_style_text_and_visual also keeps all split reference images in the final visual-token conditioning path. visual_only keeps the older image-token behavior and does not generate a style description. It returns only the visual-token conditioning instead of merging an extra native text conditioning item, so strength=1.0 is not averaged with a separate text-only condition. Its visual-token guidance is intentionally medium-neutral, uses a style-only system prompt instead of Krea2 native image-description system prompt, adds stronger wording at high strength, and does not enumerate cartoon, anime, comic, or other unrelated style words. For photographic references it explicitly asks for real-camera photographic realism, natural optics, material realism, and physically plausible light.
• placement: before_prompt or after_prompt

Output:

• CONDITIONING

When style_image is connected and style_text_mode is auto_style_text_only, the node extracts style text and uses text-only conditioning. When style_text_mode is auto_style_text_and_visual, the node does these things:

1. splits batched style_image input into individual reference images;
2. calls clip.tokenize(style_analysis_prompt, images=[single_reference]) and clip.generate(...) once per reference image to extract structured style notes with signature cues, palette, line shape, lighting, texture, rendering medium, and negative subject categories;
3. computes an algorithmic palette hint from each reference image tensor;
4. for multiple references, runs a text-only fusion pass that creates one style lock with a hierarchy and approximate recipe weights, preserving at least one non-subject cue from every reference instead of averaging them into generic neutral wording;
5. encodes the original prompt plus the style lock as native text conditioning at strength=1.0;
6. in auto_style_text_and_visual, constructs a full Krea2 chat template with one Qwen3-VL image block per split reference image, calls clip.tokenize(..., images=style_images), and encodes that visual-reference conditioning at direct strength.

When style_text_mode is auto_style_text_only, the final visual-token step is skipped, which is useful when the image token leaks reference subjects or objects into the output. When style_text_mode is visual_only, the node skips step 1 and returns only the visual-token conditioning for comparison with the older high-visual-weight behavior. When style_image is connected and strength is above 0.0, the generation prompt must be non-empty because it is embedded inside the visual-token template. When style_image is not connected, or strength is 0.0, the node behaves like native text encoding by calling clip.tokenize(prompt) directly.

Krea2 Style Prompt Encode

This helper node extracts a text style description from a reference image and returns text-only conditioning plus the final style prompt string. It is useful for inspecting whether Qwen3-VL is describing style or accidentally describing the reference subject. The main Krea2 Visual Reference Encode node now includes the same extraction path by default.

Krea2 Visual Reference Debug

Reports:

• conditioning item count
• tensor shape
• sequence length
• feature width
• whether the width matches Krea2's expected 30720
• attention-mask shape
• metadata written by Krea2 Visual Reference Encode, including style_text_mode, final unified style_text, raw per-reference aggregate style_text_raw, per-reference style_text_raw_items, computed palette_hint, per-reference palette_hint_items, style_text_fusion, style_image_count, strength, and effective_strength

If the conditioning did not come from this plugin, the debug node still reports shape information, but it cannot reliably infer the image-token span.

Install

Place this folder here:

E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Krea2-VisualRef

Restart ComfyUI after installation.

Basic Workflow

1. Load Krea2 model weights as usual.
2. Load Qwen3-VL-4B text encoder using CLIPLoader with type krea2.
3. Connect the Krea2 CLIP to Krea2 Visual Reference Encode.
4. Connect a reference image to style_image.
5. Start with style_text_mode=auto_style_text_only when the reference image subject is leaking into the output. Use auto_style_text_and_visual only when you still want the final visual-token influence, and visual_only when comparing against the older behavior.
6. Start with strength=0.5 or 0.75; lower it if the reference image subject overwhelms the prompt or causes collage/panel-like output.
7. Use placement=before_prompt first, then compare after_prompt.
8. Use the output conditioning as the positive conditioning for Krea2 sampling.
9. Add Krea2 Visual Reference Debug and check whether style_text_raw describes style rather than subject, text, logos, pose, or layout, and whether palette_hint matches the reference image palette.

Development Checks

Run from this plugin directory:

E:\ComfyUI_windows_portable\python_embeded\python.exe -B -m pytest -q --assert=plain

See VALIDATION.md for the full manual ComfyUI acceptance checklist.

Validation Suggestions

Use a fixed prompt, seed, sampler, steps, CFG, resolution, and Krea2 checkpoint. Change only the reference image or visual-reference settings.

Recommended comparisons:

• text-only baseline, generated either with the normal Krea2 text-conditioning path or this VisualRef node with style_image unconnected
• style_text_mode=auto_style_text_only versus style_text_mode=auto_style_text_and_visual versus style_text_mode=visual_only
• strong style reference image with several strength values such as 0.25, 0.5, 0.75, and 1.0
• before_prompt versus after_prompt
• Debug style_text_raw and palette_hint; if style_text_raw mentions the reference subject, objects, text, logos, pose, or layout, the generated style prompt can still leak reference content

Expected first milestone:

style image changes Krea2 conditioning
conditioning feature width is 30720
Debug reports a style_text in auto_style_text_only or auto_style_text_and_visual mode
generation changes under fixed seed

Image quality and true style fidelity are experimental and not guaranteed.

Limitations

• The auto style-text path uses the loaded Qwen3-VL CLIP wrapper's generate(...) and decode(...); it depends on the local ComfyUI/Krea2 implementation exposing those methods.
• The extracted style_text_raw_items may still mention subject matter or objects. If they do, even auto_style_text_only can leak reference content into the final generation. The algorithmic palette_hint_items can reinforce palette and contrast, but they do not understand semantic style by themselves. Multi-image fusion applies to automatic style text extraction, and auto_style_text_and_visual now passes one final visual-token placeholder per split batch image.
• strength is experimental and uses ComfyUI conditioning metadata; it is not an official Krea reference-strength control. visual_only uses strength^2.5 to avoid overpowering the prompt at mid values, while auto_style_text_and_visual uses the direct strength value for its final visual conditioning so the visual branch is not weakened twice.
• The plugin depends on ComfyUI's current Krea2/Qwen3-VL image-token support.
• Krea2 may not have been trained to interpret an image token as a style reference.
• Subject leakage, noisy conditioning, no visible effect, or fallback toward Krea2's default style bias are all possible. If style_text looks clean but the output still copies the reference subject, compare auto_style_text_only against auto_style_text_and_visual; the visual-token path is the likely leakage source.