DiffusionLight-ComfyUI

DiffusionLight (Turbo) implemented in ComfyUI

This repository is recommended for non-researchers who want to extract lighting information from any image.

Unlike the main DiffusionLight repository, which requires 16GB of VRAM to run, this ComfyUI implementation enables machines with lower VRAM to operate the model efficiently.

If you are looking for more in-depth details, please visit the DiffusionLight's Main repository

How to use

1. Select Input Image
2. Click Run
3. The output HDR Environment map DiffusionLight-0001.hdr will appear at ComfyUI/output

Installation

1. Get ComfyUI
2. Install ComfyUI-Manager
3. Install comfyui_controlnet_aux
4. Download the Stable Diffusion-releated .safetensors:

• 4.1. SDXL Based model (With VAE fix) and put into ComfyUI/models/checkpoints
• 4.2. SDXL Depth ControlNet and put into ComfyUI/models/controlnet
• 4.3. DiffusionLight's Exposure LoRA and put into ComfyUI/models/loras
• 4.4. DiffusionLight's Turbo LoRA and put into ComfyUI/models/loras

5. Clone this repository and place it in ComfyUI/custom_nodes
6. Run ComfyUI and import the workflow file diffusionlight-workflow.json from this repository

Citation

@inproceedings{Phongthawee2023DiffusionLight,
    author = {Phongthawee, Pakkapon and Chinchuthakun, Worameth and Sinsunthithet, Nontaphat and Raj, Amit and Jampani, Varun and Khungurn, Pramook and Suwajanakorn, Supasorn},
    title = {DiffusionLight: Light Probes for Free by Painting a Chrome Ball},
    booktitle = {ArXiv},
    year = {2023},
}

Visit us 🦉