MonarchRT Generate (Self-Forcing T2V)

Here's the honest framing first, because the name oversells it. This node does not run the MonarchRT models. Those were never released. What it runs is the training-free trick: take the public Self-Forcing DMD checkpoint (dense attention, Wan2.1-T2V-1.3B base), swap its self-attention for Monarch attention at inference time, and see what you get. The upstream repo explicitly supports this route, and this pack is a careful ComfyUI wrapper around it - with a dense baseline in the same workflow so you can compare on identical weights, prompt, noise and seed.

If you've read about MonarchRT and assumed it's "1.4-11.8x faster video in ComfyUI", pump the brakes. That's the paper's trained-checkpoint numbers. Community reaction to the paper was mostly one "will this come to ComfyUI?" thread, and the demo threads got picked apart on quality. Nobody in the indexed corpus has even mentioned this pack. Treat it as an experiment with real instrument readings, not a production speedup.

How it works

ComfyUI writes a schema-checked job file and starts runtime/monarchrt_job.py with the runtime's Python (via wsl.exe -d <distro> --exec ... on Windows). The runner imports the pinned upstream code unmodified, applies the profile's Monarch settings, and generates.

The loop is specific: 21 latent frames in 7 blocks of 3, each block running 4 denoising forwards plus 1 clean-context forward that updates the KV cache - 35 generator forwards per video. torch.manual_seed(seed) runs before the initial noise, so the dense and Monarch runs share the same noise and re-noising draws. The VAE decodes 81 RGB frames to an H.264 MP4 at 16 fps. And every one of the 1050 self-attention calls per video is counted: the job fails if any call took a different path than the profile asks for, so there is no silent fallback to dense attention when a kernel doesn't compile. That's unusually good instrumentation for a ComfyUI node.

Inputs and outputs

Required, all of them:

• runtime - wire it from MonarchRT Runtime.
• prompt - used as written, no rewriting, no magic. Self-Forcing is CFG-1 territory (the KB's distillation writeup explains why guidance-distilled video wants it), so don't expect a negative prompt to do anything.
• seed - integer, and the seeds for a batch are seed, seed+1, ....
• attention - the profile. monarch_h2 is the default: h_reduce=2, roughly 90% effective sparsity, the setting the paper pairs with training-free use. monarch_h1 is the upstream config default at ~95% sparsity, and without trained weights the quality drop is visible. dense disables Monarch entirely and gives you the flash-attn baseline.
• videos - 1 to 4, generated one after another by the same process. Use more than one: the first video of a process eats the Triton autotune and the 11 GB T5 read, and you want that amortised.

Outputs: video is a list of real ComfyUI VIDEO objects - wire them to Save Video, not to a latent decoder. report is a JSON string; pipe it to Preview Any and read it. It carries the profile used, wall and model-load seconds, the effective upstream config, per-video generator_forwards, attention_dispatch, CUDA peak allocation, frame counts, and hashes of the latents and RGB frames. It's the one place you can confirm you actually got Monarch kernels and not something else.

Install

Grab the node first:

cd ComfyUI/custom_nodes
git clone https://github.com/hiroki-abe-58/ComfyUI-MonarchRT

No Python dependencies land in ComfyUI. Everything else is the runtime - pinned upstream checkout, torch 2.8.0+cu128 / triton 3.4.0 / flash-attn 2.8.3 / flashinfer 0.6.3, roughly 23 GB of pinned weights, and the monarchrt.runtimes.json registration. docs/SETUP.md has the exact hf download lines with revisions. Then load workflows/monarchrt_vs_dense.json, pick your runtime, and queue: it runs both profiles as an A/B pair.

Where people get burned

The first video costs minutes even when everything is fine. Triton retunes the Monarch kernels for each of the 7 KV-cache lengths (about 140 s total), and the first prompt encoding streams the 11 GB text encoder. Later jobs in a warm worker are ~10-12 s. If you judge this pack on job one, you'll judge it wrong.

Monarch uses more VRAM than dense here, not less. On the reference 5090, monarch_h2 peaked at 22.3 GB whole-GPU versus 19.1 GB for dense. The win is 6.60 s → 4.91 s in the generator (1.34x), not a memory saving. Under 32 GB, set offload_text_encoder: true.

Quality. Read the README before posting a bug. Same noise gives PSNR 12.9 dB / SSIM 0.41 against dense for monarch_h2 - the autoregressive rollout diverges, so these are not the same video with a speed knob. The author's own eye test: coherent, often less camera motion, one duplicated subject; monarch_h1 clearly worse. There is no parity claim.

Seed overflow and empty prompts. A blank prompt errors out, and seed + videos - 1 over a 2^31-1 seed errors too.

Timeouts and cancelling. Jobs are bounded by timeout_minutes in the runtime config, and a cancel interrupts through to the runtime process. If a job dies, run the Doctor node before re-queueing.