Spark-H3
Adaptive Block Sparse Attention for MiniMax-H3
Nodes (9)
Loading a Cached MiniMax-H3 Latent
MiniMax H3 BSA All-Exact (Scheduled Dense Control)
MiniMax-H3 DMAD Sampler
MiniMax H3 Sol Attention (Official Compatibility)
Spark Attention (SM120)
Spark Attention (SM86)
Spark Attention (SM89) on 4090s and 4080s
MiniMax-H3 AV Latent Saver
Save Video Lossless (Ultrafast)
✨ Spark-Attn
Spark-Attn is a block sparse attention method for MiniMax-H3 video generation. It combines two operations:
- 🧩 Reblocking groups similar tokens into blocks so sparse attention can select more relevant interactions.
- ⚖️ Reweighting assigns different weights to tokens within each tail block when building key/value summaries, with a correction for attention mass.
Spark-H3 provides the Spark attention kernels and MiniMax-H3 inference integration.
<a id="demo"></a>
🎬 Demo
<a id="quick-start"></a>
🚀 Quick Start
Set up your H3 pipeline using the upstream MiniMax-H3 instructions, then install Spark in the same environment with a compatible PyTorch/CUDA stack:
python -m pip install -e '.[cuda]'
Wrap an already loaded pipeline with the Spark installer:
from h3_sparse_attention import install_h3_spark_attn
# MiniMaxH3Scheduler includes the terminal sigma=0 in num_inference_steps,
# but that terminal point does not run the transformer.
num_denoise_steps = 19
num_inference_steps = num_denoise_steps + 1
with install_h3_spark_attn(
pipe.transformer,
num_denoise_steps=num_denoise_steps,
warmup_mode="warmup_steps",
warmup_steps=4,
dense_layers=0,
topk_ratio=0.1,
):
result = pipe(**inputs, num_inference_steps=num_inference_steps)
<a id="comfyui-installation"></a>
🧩 ComfyUI Installation
Spark-H3 provides standalone nodes for ComfyUI 0.38.x and 0.39.x on Linux and Windows x86_64. Release packages support GeForce RTX 30-series GPUs (SM86), GeForce RTX 40-series GPUs (SM89), GeForce RTX 50-series and RTX PRO 5000/6000 Blackwell GPUs (SM120). Choose the CUDA 13.0 package matching your operating system and GPU architecture. ComfyUI and model weights are not included.
See the ComfyUI guide or 中文安装指南 for downloads, installation, and example workflows.
Step-count convention
MiniMaxH3Scheduler includes the final σ=0 endpoint in
num_inference_steps, so the default num_inference_steps=50 runs 49 denoising steps. See the
Hugging Face scheduler documentation.
<a id="todo"></a> <a id="project-status"></a>
🧭 Project Status & Roadmap
| Status | Item | Resources |
|---|---|---|
| ✅ Available | Fused Spark kernels for SM80, SM86, SM89, and SM120 | Kernel documentation · Performance matrix |
| ✅ Available | Standalone ComfyUI node packaging with wheel-first installation and source fallback | ComfyUI guide |
| ✅ Available | Release Ref2VA inference examples | Spark-Ref2VA Preview |
| 🚧 In progress | Test and validate the ComfyUI implementation on SM120 GPUs, including GeForce RTX 50 series and RTX PRO 5000/6000 Blackwell | — |
| 🗓️ Planned | Standalone stable ComfyUI implementation without a comfy-kitchen dependency | — |
| 🗓️ Planned | Fused Spark kernels for SM90 and SM100 | Pending resources |
| 🗓️ Planned | Release the technical report | — |
🤝 Acknowledgments
Thanks to MiniMax-H3 and Sol-Attn / Sol-Engine for the model and attention infrastructure. See the third-party notices for kernel attributions and included licenses.
<a id="special-thanks"></a>
❤️ Special Thanks
- Thanks to Smthem for support and help during the development of Spark-H3's ComfyUI plugin.
- Thanks to AI绘画KK for relaying the community's requests for Windows support and providing valuable workflow cases for plugin testing.
- Thanks to AIwood爱屋研究室 for identifying a bug when Spark-H3 was used in a two-pass sampling workflow and suggesting a switch to Kitchen as the dense backend, which further improved the plugin's speedup ratio.
📄 License
Spark-Attn’s original code and documentation are licensed under the Apache License 2.0. Third-party components retain their original licenses and notices; see third-party notices. MiniMax-H3 model weights remain subject to their upstream license.