DiffusionGemma Context Hub

If you hand-wire only one node in this pack, it's this one. DiffusionGemma Context Hub is the input-assembly station: it takes your plain-text brief, an optional image, an optional video, and an optional visual_description, and packs them into a single DG_CONTEXT bundle that the CoT Generator feeds to the model. Everything upstream of it is your own; everything downstream is the pipeline's business.

It's also the only node in the pack that touches your pixels before the LLM does - which is why it's where the "how much is this going to cost me" dials live.

How it works

Under the hood the hub delegates to the pack's internal media sampler (kept in source, not exported as its own node). It does three useful things to your inputs:

• Samples the video. At sample_fps frames per second, up to max_frames total. This is the trick that keeps a 10-second clip affordable - and the author's own advice is that 2–3 frames per second is plenty. Sampling 241 frames for a 10s video is not better than sampling 2–3 per second; it just burns VRAM and generation time.
• Trims or rejects overlong clips. If the video runs past max_duration_seconds, overlong_policy decides between trimming it down (default) or refusing outright.
• Prepares the identity reference. In image_identity_video_control mode it aspect-fits the reference image to match the sampled video frame dimensions and prepends it to the tensor, so the model receives one coherent stack: identity image first, then ordered control frames.

All the context - fps, sampled frame indices, timecodes, duration, warnings, synthesis contract - travels inside DG_CONTEXT as metadata, so the model knows exactly what it's looking at and when.

The inputs that matter

• user_prompt - your creative brief. This is supposed to be short and high-level: "a knight walking through rain toward a castle." DiffusionGemma fills in the generator-ready detail (lighting, camera, pacing) itself; you're not writing the final prompt here.
• sample_fps and max_frames - your cost dial. Start at 1–3 fps / 60 frames and raise only if the prompt quality suffers.
• max_duration_seconds + overlong_policy - how much of a long clip gets analyzed. reject throws an error on overlong input; trim just chops it (and tags a warning).
• media_synthesis_mode - the important one. video_recreation (default) treats the video as the thing to recreate. image_identity_video_control is the mode behind the packaged character motion-transfer workflow: the reference image owns the subject's identity and appearance (face, hair, wardrobe, style), while the video supplies pose, motion, camera choreography, depth, and Canny/edge structure. If you want a different person performing the source clip's actions, this is the mode.

The optional visual_description is a text caption you can supply instead of (or alongside) pixels - handy when your backend can't inspect image/video directly.

Outputs are gemma_context (wire it into the CoT Generator), plus context_json and context_preview - both just for eyeballing what got packed before you burn a generation on it.

Install

This is one of five nodes in the DiffusionGemma Prompt Builder pack, so install is the same for all of them: ComfyUI Manager (search DiffusionGemma Prompt Builder) or

cd ComfyUI/custom_nodes
git clone https://github.com/exportAnything/ComfyUI-DiffusionGemmaPromptBuilder

then restart. The real setup cost is the model: nvidia/diffusiongemma-26B-A4B-it-NVFP4 (a multi-GB download) goes whole-folder into ComfyUI/models/LLM/, plus transformers>=5.12.1, comfy-kitchen, and a ComfyUI build exposing comfy.quant_ops. This node itself does nothing until the model is in place.

Common issues

• "Reference image was aspect-fitted and padded to match sampled video frame dimensions" - that's a warning, not an error. Expected in image_identity_video_control mode when the reference image's aspect ratio differs from the video's.
• Overlong video + reject policy throws. If you're processing long clips, keep the default trim unless you have a reason not to.
• Identity isn't sticking - you're probably in video_recreation mode. Switch media_synthesis_mode to image_identity_video_control so the model treats the image as the appearance source and the video purely as control structure. Mixing the two is the #1 way to get the video subject's face back in your output.