Extensions/ComfyUI-DiT360
ComfyUI Extension

ComfyUI-DiT360

Implement circular padding VAE in ComfyUI from DiT 360 Paper

By cedarconnor·Created 9 months ago·Updated 8 months ago· 3
cedarconnor/ComfyUI-DiT360
Nodes8
On cloudLocal install
CategoryDiT360/vae, DiT360
Stars3
Updated8 months ago
Readme

⚠️ EXPERIMENTAL - Active Development

Major bugs recently fixed (circular RoPE, padding, sampling). Test thoroughly before production use.

ComfyUI-DiT360

360° Panorama Generation Enhancement Nodes for ComfyUI

Generate seamless equirectangular panoramic images using FLUX.1-dev with the DiT360 LoRA adapter.


🌟 Features

  • Circular Padding: Seamless wraparound at panorama edges
  • 2:1 Aspect Ratio Enforcement: Automatic equirectangular format
  • Edge Blending: Perfect continuity at boundaries
  • Interactive 360° Viewer: Three.js-based panorama navigation
  • Geometric Losses: Yaw loss (rotational consistency) & Cube loss (pole distortion reduction)

📦 What is This?

DiT360 is a LoRA adapter (~2-5GB) for FLUX.1-dev that enables high-quality 360° panorama generation. This node pack provides enhancement nodes that add the necessary circular padding and post-processing to make panoramas seamless.

This is NOT a full model loader—you use standard ComfyUI nodes to load FLUX and apply the DiT360 LoRA, then use our enhancement nodes for 360° specific features.


🚀 Installation

1. Install ComfyUI

If you haven't already, install ComfyUI.

2. Clone This Repository

cd ComfyUI/custom_nodes
git clone https://github.com/yourusername/ComfyUI-DiT360.git
cd ComfyUI-DiT360
pip install -r requirements.txt

3. Download Models

  • FLUX.1-dev: Place in ComfyUI/models/checkpoints/ Download from Hugging Face

  • DiT360 LoRA: Place in ComfyUI/models/loras/ Download from Hugging Face

4. Restart ComfyUI

python main.py

You should see: ✅ ComfyUI-DiT360 v2.0.0 loaded


📖 Usage

Basic Workflow

  1. Load Checkpoint → Select FLUX.1-dev (standard node)
  2. Load LoRA → Select dit360.safetensors, strength 1.0 (standard node)
  3. CLIP Text Encode → Enter your prompt (standard node)
  4. Equirect360EmptyLatent → Creates 2048×1024 latent (our node)
  5. Equirect360KSampler → Sample with circular padding (our node)
  6. Equirect360VAEDecode → Decode with circular padding (our node)
  7. Equirect360EdgeBlender → Blend edges for perfect wraparound (our node)
  8. Equirect360Viewer → Interactive 360° preview (our node, coming soon)
  9. Save Image → Standard save (standard node)

Example Workflow

Load any of these JSONs in ComfyUI and update the model/LoRA/VAE filenames to match your setup:

  • examples/dit360_flux_01_baseline_no_rope_no_padding.json (stable baseline; seam via EdgeBlender)
  • examples/dit360_flux_02_ksampler_and_vae_padding.json (uses sampler + decode circular padding)
  • examples/dit360_flux_03_vae_conv2d_circular_padding.json (uses Apply Circular Padding VAE; set VAEDecode padding to 0)
  • examples/dit360_flux_04_rope_shift_plus_vae_padding.json (experimental RoPE shift + VAE circular padding)

🎨 Node Descriptions

1. Equirect360EmptyLatent

Creates empty latent with enforced 2:1 aspect ratio.

  • Input: Width (e.g., 2048), Batch size
  • Output: Latent (auto-calculated height = width/2)
  • Use instead of: EmptyLatentImage

2. Equirect360KSampler

Standard KSampler with circular padding for seamless edges.

  • Inputs: Model, conditioning, latent, sampler settings
  • Key Parameter: circular_padding (16-32 recommended)
  • Optional Geometric Losses:
    • enable_yaw_loss + yaw_loss_weight (0.05-0.2) - Ensures rotational consistency, ~2x slower
    • enable_cube_loss + cube_loss_weight (0.05-0.2) - Reduces pole distortion, ~1.5x slower
  • Use instead of: KSampler

Note: Geometric losses improve quality but increase generation time. Start with just circular padding, then add losses if you need extra quality.

3. Equirect360VAEDecode

VAE decode with circular padding for smooth edges.

  • Inputs: Samples (latent), VAE, circular_padding
  • Use instead of: VAEDecode

4. Equirect360EdgeBlender

Post-processing to ensure perfect wraparound.

  • Inputs: Image, blend_width (10-20 px), blend_mode (cosine/linear/smooth)
  • Highly recommended for best results!

5. Equirect360Viewer

Interactive Three.js viewer for 360° navigation inside ComfyUI.

  • Input: Image
  • Features: Mouse drag, scroll zoom, fullscreen

Viewer dependency: If the 360° viewer button does nothing, install Three.js locally:

  • Run python install.py in this repo (downloads into web/js/lib), then restart ComfyUI

🌀 Optional: Circular RoPE (Experimental)

These nodes patch a model’s Rotary Position Embeddings (RoPE) so the panorama wraps horizontally at the attention level (not just padding/blending). Results are model-dependent and still experimental.

Apply Circular RoPE

  • mode:
    • shift (default): wraps only a seam band near the right edge (safer for planar-trained models like FLUX)
    • angle: maps X positions to a circle in radians (more aggressive)
  • seam_width: token columns near the right edge to wrap (0 = auto)

Apply Circular Panorama (All-in-One)

Applies both:

  • Conv2d X-only circular padding (where applicable)
  • Circular RoPE (same options as above)

Apply Circular Padding VAE

Patches the VAE Conv2d layers to use X-only circular padding (helps reduce seam after decode).

  • If you use this, set Equirect360VAEDecode.circular_padding to 0 to avoid “double padding”.

💡 Prompting Tips

Describe the full 360° environment in your prompts:

Good Examples:

"A cozy mountain cabin interior, large windows showing snowy peaks,
warm fireplace, wooden furniture, morning light, 360 degree panorama"

"Standing in a futuristic city plaza, skyscrapers all around,
neon signs, rain-slicked streets, night time, cyberpunk, 360 panorama"

"Ancient library with towering bookshelves on all sides, spiral staircases,
warm lighting from chandeliers, dusty atmosphere, 360 degree view"

⚙️ System Requirements

Minimum

  • GPU: NVIDIA with 12GB VRAM (RTX 3060 12GB, 3080, 4070)
  • RAM: 16GB system memory
  • Storage: 30GB free space (FLUX + LoRA)
  • OS: Windows 10/11 or Linux

Recommended

  • GPU: NVIDIA with 16GB+ VRAM (RTX 4080, 4090)
  • RAM: 32GB system memory
  • Storage: 50GB NVMe SSD

Resolution Guide

| Resolution | VRAM | Speed | Quality | |-------------|-------|--------|-------------| | 1024×512 | 12GB | Fast | Good | | 2048×1024 | 16GB | Medium | Excellent | | 4096×2048 | 24GB+ | Slow | Outstanding |


🐛 Troubleshooting

Visible Seam at Edges

  • Solution: Increase circular_padding to 24-32
  • Solution: Increase blend_width to 20+
  • Solution: Enable enable_yaw_loss (slower)

Out of Memory

  • Solution: Lower resolution (1024×512)
  • Solution: Use fp8 precision for FLUX
  • Solution: Disable yaw/cube losses

Not Seamless in Viewer

  • Check: DiT360 LoRA is loaded and strength = 1.0
  • Check: circular_padding > 0
  • Check: Using Equirect360EdgeBlender

Washed Out / Broken Output (RoPE)

  • Solution: Prefer full FLUX dev over fp8 when testing RoPE
  • Solution: Use rope_mode=shift (start with rope_seam_width=0 or 4-16)
  • Solution: Turn patch_rope OFF first and rely on padding + EdgeBlender
  • Check: Avoid double padding (if using Apply Circular Padding VAE, set VAEDecode padding to 0)

📚 Documentation


🗺️ Roadmap

v2.0.0 (Current - Complete!)

  • [x] Documentation cleanup
  • [x] Core utilities (circular padding, edge blending)
  • [x] 5 clean nodes
  • [x] Three.js 360° viewer
  • [x] Example workflows
  • [x] Yaw loss (rotational consistency)
  • [x] Cube loss (pole distortion reduction)

v2.1.0 (Future)

  • [ ] ControlNet integration
  • [ ] Inpainting support
  • [ ] Img2img workflow improvements
  • [ ] Advanced loss visualization

v2.2.0 (Future)

  • [ ] Performance optimizations (xFormers, attention slicing)
  • [ ] Batch generation support
  • [ ] Video panorama generation
  • [ ] Custom loss functions API

📄 License

Apache License 2.0

Models:

  • FLUX.1-dev: Subject to Black Forest Labs license
  • DiT360 LoRA: Subject to Insta360 Research license

🙏 Credits


🤝 Contributing

Contributions welcome! Please:

  1. Fork the repository
  2. Create a feature branch
  3. Submit a pull request

📞 Support


⭐ If you find this useful, please star the repository!