H3 Director — Compact Multimodal Edit

What it is

This is the centre of the pack. You give it a rough instruction - "swap the background to a rainy Tokyo alley, keep her face and jacket" - connect whatever source images or video plates you have, and it returns a finished MiniMax H3 prompt written in H3's own grammar, with your sources named as references, the visible evidence it actually saw in the pixels, and the change stated precisely.

H3 is the 33B omni-modal video model whose whole selling point is a unified text/image/video/audio context, which means it will honour a well-written reference instruction and will blend your sources into mush if the instruction is vague. That gap is the entire reason this node exists. It also writes prompts that respect H3's native structure: a bracketed mode header plus, in the richer modes, the four-headed prompt grammar H3 expects (subject_definitions:, integrated_multimodal_description:, overall_soundscape:, non_diegetic_music:).

Be clear about what it is not: it doesn't touch the generation model, sampler recipe or LoRAs, and it has no built-in API. You must wire an external LLM provider into its llm input - OpenRouter, Ollama, a compatible chat API, or a local Qwen3-VL text encoder, all in the same pack. Connect nothing and it raises rather than faking it.

How it works

The node assembles one large request: your edit_request, the selected mode's rule set appended to the system prompt, the names of every connected source (<Picture 1>–<Picture 4>, <Video 1>/<Video 2>, <Mood Image 1>, <Mood Video 1>), and the images themselves. Stills get resized to fit image_max_dimension and JPEG-encoded; video plates are sampled at video_samples (3, 5 or 10 frames) and sent as frames tagged with which video they came from. The provider then has to return JSON - edit_type, source_roles, visual_evidence, edit_prompt - or, in the sequence modes, a scene_prompts array.

After the reply comes back, the node compiles and audits it. It strips any header the model wrote itself and prefixes the correct one: [reference generation] for stills-only, [video editing] for video without pictures, [video editing + reference generation] when you connect both. Then it checks for the failure modes it knows about and reports them in an output rather than hiding them: a source tag you named that never made it into the final prompt, weak overlap between the stated visual evidence and the prompt, missing H3 sections in the elaborate modes, a missing I2V alignment phrase. It retries once if the JSON is malformed.

The modes are the real decision. compact is the short one. edit adds a strict source ledger and change/preserve analysis. Elaborate writes one long cinematic prompt with expanded setting, action, mood, camera and sound, sized to your seconds_per_scene. Enhance writes one compact prompt per scene. Continuous Edit and Continuous Elaborate keep that sequence structure with progressively evolving edit rules.

The inputs that matter

• edit_request - the prompt you write. Name each connected source and give it a role ("use <Picture 2> only for the wardrobe"). The system prompt explicitly forbids blending sources just because they're connected.
• edit_mode - start on compact; move to Elaborate when you want a long, cinematic description.
• reference_image_1..4, source_video_1/2, mood_image, mood_video - all optional. Mood slots are direction-only; the model is told to take the feeling, not the content.
• llm - required unless bypass is on.
• continuous_scene_count (0–12, local plan; 0 is normalised to 1) and seconds_per_scene (FLOAT, only used by the elaborate modes, so it must be fed by a Primitive or similar).
• i2v_mode, hold_prompt, bypass, debug_request - the four behaviour switches worth reading the tooltips for. bypass returns your prompt verbatim and skips the LLM entirely, which is a good way to check the rest of your graph.

Outputs: edit_prompt goes into H3 conditioning. validation is the readiness-and-warnings string - read it. usage_stats gives tokens plus provider and model. continuous_scene_count matches the number of prompts actually returned; debug_request and raw_response are diagnostics.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/elgalardi/ComfyUI-VisionPromptAssistant
# restart ComfyUI; ComfyUI Manager works too - search "Vision Prompt Assistant"

No pip dependencies (the old faster-whisper requirement was dropped in 2.0), but ComfyUI 0.30+ is required. Version 2.0 was a breaking cleanup: the story/LTX directors and transcription nodes are gone, and old workflows referencing those node IDs need an earlier release.

Common issues

Connect an external provider to llm... No provider wired. It says clearly that no internal API or fallback exists.

Prompt count keeps changing between runs. That's the LLM, not a bug. Reusing the same seed on director and sampler does not guarantee identical external output - the README says so outright. If you need exact recall, enable hold_prompt, which caches the plan under output/Sexy AI Studio/director_hold/plans.json and reuses it.

I2V Mode requires reference_image_1 as the opening frame. The mode treats Picture 1 as the literal first frame; you must actually connect one.

Empty edit_request. Only the elaborate modes can work from direction controls alone - everywhere else the request is mandatory.

JSON repair loops or truncated prompts. Raise max_tokens, and for sequence work raise timeout_seconds too. If you're on a local model, expect hard failures instead of quiet degradation: their contract checks the JSON and refuses to send anything malformed downstream.

Before you install anything that gets your API key. This category of node - LLM/VLM wrappers that reach the network by design - includes the one pack that shipped credential-stealing malware into a Disney machine. Prefer the env-var key path, and read what a fresh node pack does before its first run.