Prism Audio DiT

What it is

The audio transformer. Prism's whole selling point is that sound and picture come out of the same denoising run - speech, music and effects generated alongside the frames, rather than bolted on afterwards the way most local workflows do it (generate silent video, then run a foley model like MMAudio at the result). This node loads the half of that system that produces the waveform.

You reach for it because the audio_dit socket on Prism Native Pipeline needs filling, and without it there's no pipeline. It's a two-second node and a several-gigabyte file, which is a fair summary of Prism generally.

How it fits, and how it doesn't

Worth being precise about the division of labour, because it isn't obvious from the node browser:

• Prism Audio DiT loads the audio denoiser - the transformer that generates audio latents.
• Prism DAC Audio VAE loads the codec that turns those latents into a waveform. That's the Descript Audio Codec side of the stack, and it's why the pack's requirements pull in descript-audiotools.
• The Prism Audio Video Bridge is what lets the two towers see each other at all.

Under the hood you get a descriptor, not a loaded model: the node reads the safetensors header, verifies the component kind is audio_dit, and hands a typed output downstream. Weights materialise when a sampler runs. One caveat that's easy to miss - the audio tower keeps floating-point activations in both samplers. The native sampler's int8_backend choice (portable vs kitchen) applies to the video towers; the bridge, the text encoder and this audio tower stay in float. So "I picked kitchen and it's still slow" is expected behaviour, not a bug.

The one input and one output

model_name - an enum of what's in your models/diffusion_models/ folders. Yes, the audio DiT sits in the diffusion models directory, which reads oddly for an audio component; that's the layout the pack documents and the enum follows it.

Output is audio_dit, which wires into the matching socket on Prism Native Pipeline.

The file: prism_alpha_audio_dit_int8_convrot.safetensors, from t8star/Prism-Comfy.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/T8mars/Comfyui-Prism-T8.git
cd Comfyui-Prism-T8
python -m pip install -r requirements.txt   # with the Python that runs ComfyUI

ComfyUI Manager → Comfyui-Prism-T8 (or registry t8star/prism-t8) does the same thing with less typing. Then drop the audio DiT into models/diffusion_models/ and restart. Requirements include diffusers, transformers, accelerate, safetensors, einops, ftfy, descript-audiotools, sentencepiece, filelock and psutil - install them with ComfyUI's interpreter, and leave its CUDA PyTorch alone. Python 3.10+, NVIDIA CUDA with BF16 support.

Where people get burned

Wrong socket. Dropping this file into the video DiT loader gives you Incorrect component in video_dit: audio_dit. Genuinely useful error, but only if you read it.

Mixed bundles. All seven components carry a bundle id, and the pipeline refuses to assemble a set from different conversions: Components are from different Prism variants/conversion bundles.

Sharp prompt, muddy audio, and the model gets the blame. Fairly often it should. The author is unusually direct about this: Prism's base model produces degraded audio for some prompt, seed and reference-image combinations, a limitation reproduced together with the FreeVideo author, and it affects both the native and accelerated samplers. The audio_prompt and the <speech> / <music> / <sfx> tags in the prompt steer it, but they don't guarantee a clean result. Always listen to the output before you ship it - decoded, finite and speech-recognised are not the same thing as sounds right.

Expecting this node to be enough on its own. It isn't an audio generator in the ACE-Step or Chatterbox sense. It makes the soundtrack that belongs to this clip. If you want a specific voice or song, generate it elsewhere and keep Prism for the picture.