Prism Native Video Audio Sampler

What it is

This is the honest Prism experience. One node takes your assembled seven-component pipeline and runs the original sampler: 50 steps, CFG 5, 1280×720 at 205 frames, video and audio denoised together with a paired flow-match scheduler. It's the reference path the accelerated FreeVideo sampler is measured against, and it's the one to use when you actually care what Prism can do rather than how fast it can do it.

Reach for it when you want the full sparse-attention control surface (via sparse_options), when you want the official 50-step geometry, or when you're comparing against something and don't want a distilled student in the way.

The two modes, and the white-image trick

mode has two values, and it changes what the reference image means:

• i2va - image-to-video-with-audio. You connect reference_image and it conditions on it. If you don't, it raises Connect a reference IMAGE for native I2VA.
• t2va_white_reference - the author's tooltip says it plainly: T2VA uses MOVA's experimental white reference image convention. Prism's video tower was trained to expect a first frame, so "text to video" is really "generate from a pure white frame". Don't connect an image in this mode - it errors with T2VA white-reference mode expects no image.

The reference image gets cropped to your target width/height rather than letterboxed, so frame your input roughly at your output aspect ratio.

Inputs worth touching

prompt and audio_prompt are independent. The main workflow also understands Prism's audio tags - <music>, <sfx>, <speech> - which is how you get dialogue instead of generic room tone. An empty audio_prompt falls back to the video prompt.

negative_prompt defaults to the official Chinese negative string the pack ships with (the over-saturated, blurry, worst-quality boilerplate). It's tuned for this model; replacing it with your favourite SDXL negative is a downgrade, not a personality choice.

width / height must be multiples of 16, num_frames at least 5 with (num_frames - 1) % 4 == 0. The validation is strict and tells you exactly which rule you broke. frame_policy decides what happens when your number doesn't fit: strict refuses, snap rounds down to the nearest legal count. Use snap while experimenting, strict for reproducibility.

steps (50), cfg (5), visual_shift and audio_shift (7 each) are the sampling knobs. The shifts are the flow-match scheduler's time-shift per modality - that they're separate is the nice part of a layered video+audio model, and the defaults are the upstream ones.

offload is the memory lever: block streams transformer blocks between CPU and GPU, cpu keeps only the frozen encoders and codecs off the card, none keeps everything resident if you have the VRAM. Block offload's speed depends on your PCIe and system RAM, and INT8 does not shrink high-resolution activations, so it doesn't rescue you from 720p/205-frame decoding.

attention picks sdpa (default, no Triton needed) or auto. int8_backend picks portable (W8A16, the safe default) or kitchen (dynamic W8A8 on the video towers). The author's tooltip is the important bit: the audio tower, the bridge and the text encoder keep floating-point activations either way, and the text encoder can be either BF16 or INT8 weights.

vae_tiling, tile_size (256) and tile_stride (192) exist for decoding, because the video VAE is where tiling presets live in the 720p example. Tile size and stride must be divisible by 8 and stride must be smaller than size.

sparse_options is optional and takes a Prism Native Sparse Options node - blend-shaped block-sparse attention, if you want it and have Triton.

Outputs

frames (IMAGE), audio (AUDIO) and fps (FLOAT). Frames plus audio plus fps go into Prism Save Video with Audio and you're done; the audio comes out at 48 kHz and is trimmed to exactly match the video length, since the DAC codec rounds up to whole codec frames.

Install

Manager → Comfyui-Prism-T8, registry t8star/prism-t8, or:

cd ComfyUI/custom_nodes && git clone https://github.com/T8mars/Comfyui-Prism-T8.git
cd Comfyui-Prism-T8 && python -m pip install -r requirements.txt

Then the seven standalone model files from t8star/Prism-Comfy into models/diffusion_models/, models/text_encoders/ and models/vae/, and restart. Python 3.10+, BF16-capable NVIDIA CUDA, and ffmpeg on PATH for the saving node. SDPA needs no Triton; the native BSA path does.

Realistic expectations

The pack's own validation run - 848×480, 49 frames, 50 steps, INT8, block offload - took about 47 minutes on an RTX 5090 Laptop 24 GB with a 7.76 GiB peak allocation. That's the floor, not the reference size: 720p/205 frames is roughly four times the pixels and four times the frames. There's mild composition drift and slightly soft fine texture; quantisation isn't free.

And the caveat the author repeats more than once: Prism's base model degrades audio for some prompt/seed/reference combinations, a limitation reproduced with the FreeVideo author. Run it, then actually listen. Automated checks that the waveform is finite and decodable prove nothing about whether the speech is intelligible.

One community note: there is essentially zero English-language discussion of Prism to lean on. The Bilibili demos and the pack's Chinese docs are the whole trail, so treat the README's numbers as the best available evidence rather than consensus.