ComfyUI Extension: vae-decode-hdr

Authored by sumitchatterjee13

Created

Updated

40 stars

A custom ComfyUI node that preserves High Dynamic Range (HDR) data during VAE decoding, bypassing the default 0-1 range clamping. It also includes a companion 'Linear EXR Export' node for creating professional-grade linear EXR files.

Custom Nodes (0)

    README

    ComfyUI HDR VAE Decode Node

    GitHub License: MIT ComfyUI

    A custom ComfyUI node that intelligently preserves HDR data from VAE models for professional VFX workflows.

    šŸ“‹ Table of Contents

    Overview

    After extensive research and development, I've created a node that solves the fundamental HDR preservation problem in VAE decoding. Instead of blindly bypassing layers, this implementation uses a scientific approach to understand and work with the VAE's natural behavior:

    HDR VAE Workflow Professional HDR workflow: HDR VAE Decode ā†’ Linear EXR Export for true HDR preservation

    šŸŽÆ Important: This HDR VAE Decode is specifically built and tested for Flux.1 VAE models. While it may work with other VAE architectures, it has been optimized for Flux.1's specific decoder structure and may not perform as expected with different models.

    • Intelligent Analysis: Automatically analyzes the VAE's conv_out transformation to understand how HDR data is processed
    • Smart HDR Expansion: Preserves the VAE's excellent tone mapping while selectively expanding highlight regions
    • Multiple HDR Modes: Six distinct modes including advanced inverse sigmoid recovery techniques
    • Nuke-Style Clamping: Professional clamp controls to manage extreme values and outliers
    • Professional Quality: Maintains perceptual image quality while extending dynamic range where needed
    • VFX Ready: 32-bit float pipeline with proper HDR preservation for compositing workflows

    Problem Statement

    ComfyUI's default image processing pipeline has several HDR-breaking limitations that result in significant quality loss:

    Standard VAE Decode Issues āŒ Standard Processing: Clipped highlights, limited color range, lost detail in bright areas

    VAE Decode Issues:

    1. Range Clamping: VAE outputs are constrained to 0-1 pixel values
    2. Lost Dynamic Range: High and low luminance information is compressed/clipped
    3. 8-bit Pipeline: Effective output is 8-bit despite higher precision formats
    4. Color Space Constraints: Limited to sRGB-like color spaces

    Save Image Issues:

    1. šŸšØ Critical: ComfyUI's built-in Save Image node normalizes ALL pixel values to 0-1 range
    2. HDR Data Destruction: Even if you preserve HDR in processing, Save Image destroys it during export
    3. False EXR Support: Claims to save EXR but actually saves 8-bit data in EXR container
    4. VFX Incompatible: Produces files that appear HDR but contain no extended range data

    The result: Beautiful AI-generated content gets degraded during processing, losing the very highlight details and color richness that make images compelling for professional use.

    Technical Innovation

    Core Breakthrough

    Through systematic analysis, I discovered that the VAE's conv_out layer applies sigmoid-like normalization that clamps HDR values to [0,1]. Rather than simply bypassing this layer, I developed an intelligent approach that:

    1. Analyzes the transformation using forward hooks to capture pre/post conv_out data
    2. Detects the normalization pattern (sigmoid, tanh, or custom)
    3. Preserves base image quality by using the VAE's excellent tone mapping as foundation
    4. Selectively expands highlights only in regions that exceed the standard range

    Technical Features

    • Scientific Analysis: Real-time conv_out transformation analysis with statistical profiling
    • MAX Pooling Channel Conversion: Preserves HDR peaks when converting 128ā†’3 channels (vs averaging which destroys brightness)
    • Smart Device Management: Automatic CUDA/CPU synchronization across all processing stages
    • Multiple HDR Modes:
      • Conservative: Gentle 1.5x expansion, safest for general use
      • Moderate: 3x smart expansion, balanced quality/range
      • Exposure: Natural exposure-based HDR for compositing workflows (default)
      • Luminance Recovery: Advanced exposure-based HDR using full inverse sigmoid mathematical recovery
      • Aggressive: Full mathematical recovery for maximum range
      • Highlight Fusion: Advanced multi-pass blending using inverse sigmoid techniques for maximum highlight detail preservation
    • Nuke-Style Clamping: Professional clamp controls (min/max) to manage extreme values and remove outliers, matching industry-standard compositing workflows
    • Robust Fallback System: Smart bypass ā†’ Simple bypass if intelligent methods fail
    • Professional Pipeline: Float32 throughout with proper tensor formatting for ComfyUI

    šŸŽÆ The HDR Solution

    HDR VAE Decode Success
    āœ… HDR VAE Decode: Extended highlights preserved, natural color depth, professional VFX quality

    Dramatic improvements achieved:

    • Highlight Detail Recovery: Bright bokeh retains structure instead of clipping to pure white
    • Color Richness: Full spectrum preservation with enhanced vibrancy and depth
    • Natural Light Falloff: Smooth gradients in bright areas maintain photographic realism
    • VFX Compositing Ready: Extended range enables professional color grading and exposure control

    Installation

    Method 1: Git Clone (Recommended)

    1. Clone this repository into your ComfyUI custom_nodes directory:
    cd ComfyUI/custom_nodes
git clone https://github.com/sumitchatterjee13/vae-decode-hdr.git
    1. Install minimal additional dependencies (ComfyUI already provides torch, torchvision, numpy):
    cd vae-decode-hdr
pip install -r requirements.txt

    āš ļø Important for Professional EXR Export: For VFX-grade EXR file writing with full bit depth control, install OpenImageIO (industry standard):

    pip install OpenImageIO

    OpenImageIO provides professional-grade EXR support with true 32-bit/16-bit float precision and full compression options. If not installed, the node will automatically fall back to OpenCV (which still preserves HDR data, but with limited bit depth control).

    1. Restart ComfyUI

    Method 2: Direct Download

    1. Download ZIP from GitHub
    2. Extract to ComfyUI/custom_nodes/vae-decode-hdr/
    3. Install dependencies: pip install -r requirements.txt
    4. (Optional but recommended) Install OpenImageIO for professional EXR support: pip install OpenImageIO
    5. Restart ComfyUI

    The nodes should appear in the latent category as "HDR VAE Decode" and in the image category as "Linear EXR Export".

    HDR VAE Workflow What you'll create: HDR VAE Decode (latent category) ā†’ Linear EXR Export (image category)

    Usage

    The HDR VAE Decode node will appear in the "latent" category in ComfyUI.

    Parameters:

    • samples: Latent samples from your pipeline
    • vae: The VAE model (tested with Flux.1 Dev, compatible with other VAE models)
    • hdr_mode (optional): HDR processing mode:
      • conservative: Gentle 1.5x expansion, maintains natural appearance
      • moderate: 3x smart expansion, balanced for most VFX work
      • exposure: Natural exposure-based HDR using exposure stops (default)
      • luminance_recovery: Advanced exposure-based HDR with full inverse sigmoid recovery for maximum highlight detail
      • aggressive: Full mathematical recovery, maximum dynamic range
      • highlight_fusion: Advanced multi-pass blending with inverse sigmoid for extreme highlight preservation
    • clamp_maximum (optional): Maximum clamp value (0.0-65504.0, default: 4.0) - removes extreme outliers from inverse sigmoid processing
    • clamp_minimum (optional): Minimum clamp value (-65504.0 to 65504.0, default: 0.0) - sets lower bound
    • enable_clamping (optional): Enable/disable Nuke-style clamping (default: true) - disable to preserve full dynamic range
    • scale_factor (optional): Additional scaling multiplier (0.1-10x, default: 1.0)

    Expected Results:

    The node first attempts intelligent HDR decode using scientific analysis of the VAE's behavior. If this succeeds, you'll get natural-looking images with selectively expanded highlights. If it fails, the system falls back to the robust bypass method that directly processes VAE decoder layers.

    Output:

    • image: Professional-quality HDR image tensor with preserved dynamic range, ready for EXR export or further processing

    Example Workflow

    HDR VAE Workflow Example ComfyUI workflow showing HDR VAE Decode connected to Linear EXR Export

    For typical VFX work:

    1. Connect your Flux.1 latents to the HDR VAE Decode node
    2. Choose your HDR mode:
      • Start with "exposure" (default) for natural, reliable results
      • Try "moderate" or "conservative" for gentler HDR expansion
      • Use "luminance_recovery" or "highlight_fusion" for maximum highlight detail (uses inverse sigmoid recovery)
    3. Adjust clamping if needed:
      • Default clamp maximum of 4.0 removes outliers from inverse sigmoid modes
      • Disable clamping for full unconstrained dynamic range
      • Adjust maximum/minimum values to match your compositing workflow
    4. āš ļø CRITICAL: Connect output to the Linear EXR Export node - DO NOT use ComfyUI's built-in Save Image node
    5. Configure output path (e.g., /Test for output/Test/ subfolder)
    6. Use in your compositing software (Nuke, After Effects, etc.) with proper HDR handling

    šŸšØ IMPORTANT: Why You Must Use Linear EXR Export

    āŒ ComfyUI's Built-in Save Image Node:

    • Automatically normalizes all pixel values to 0-1 range
    • Destroys HDR data above 1.0
    • Clips highlights and bright regions
    • Converts to 8-bit even when saving as EXR
    • Completely defeats the purpose of HDR processing

    āœ… Our Linear EXR Export Node:

    • Preserves full HDR range (values above 1.0)
    • True 32-bit float precision
    • Linear color space maintained
    • Professional VFX quality output
    • Smart path handling (/Test ā†’ output/Test/)

    The HDR VAE Decode node will automatically:

    • Analyze the VAE's behavior
    • Apply intelligent HDR expansion to highlight regions
    • Preserve natural image appearance
    • Output HDR data for professional workflows

    āš ļø Remember: The magic happens in the combination - HDR VAE Decode generates the HDR data, but you need Linear EXR Export to save it properly!

    Built-in HDR Export Solution

    This package now includes a Linear EXR Export node for professional HDR output:

    šŸšØ Critical Difference from Built-in Save Image

    | Feature | ComfyUI Save Image | Our Linear EXR Export | |---------|-------------------|------------------------| | HDR Values | āŒ Normalizes to 0-1 | āœ… Preserves >1.0 values | | Bit Depth | āŒ 8-bit effective | āœ… True 32-bit float | | Color Space | āŒ sRGB/Gamma | āœ… Linear maintained | | VFX Ready | āŒ No | āœ… Professional quality | | HDR Workflow | āŒ Destroys HDR data | āœ… Preserves all HDR data |

    Linear EXR Export Node Features:

    • OpenImageIO-powered: Uses industry-standard VFX library for professional-grade EXR writing
    • True 32-bit EXR export with preserved HDR values above 1.0
    • Professional VFX quality - maintains linear color space with full OpenEXR specification compliance
    • Smart path handling:
      • Empty path ā†’ saves to ComfyUI/output/
      • /subfolder ā†’ saves to ComfyUI/output/subfolder/
      • Full path ā†’ uses custom absolute/relative path
    • Multiple bit depths: 32-bit float or 16-bit half precision (smaller files)
    • Compression options: ZIP, PIZ, RLE, PXR24, or none (via OpenImageIO)
    • Auto-incrementing filenames: Never overwrites existing files - perfect for image sequences
    • Clean file naming with customizable prefixes and counters
    • HDR verification: Automatically verifies HDR values are preserved in saved files
    • Seamless integration: Designed specifically for HDR VAE Decode output
    • Automatic fallback: Uses OpenCV if OpenImageIO is not installed (still preserves HDR data)

    šŸ”„ Auto-Incrementing Filename Feature

    The Linear EXR Export node now includes intelligent filename management that prevents accidental overwrites:

    How it works:

    • If Test_00001.exr exists, automatically saves as Test_00002.exr
    • If HDR_VAE_00005.exr exists, automatically saves as HDR_VAE_00006.exr
    • Perfect for image sequences: Generate multiple variations without losing previous outputs
    • Safe iterations: Experiment with different settings without fear of overwriting good results

    Examples:

    Generation 1: MyScene_00001.exr
Generation 2: MyScene_00002.exr (automatically incremented)
Generation 3: MyScene_00003.exr (automatically incremented)
...and so on

    Complete HDR Workflow:

    HDR VAE Decode ā†’ Linear EXR Export ā†’ Professional EXR files ready for compositing in Nuke, After Effects, or other VFX software.

    HDR VAE Workflow Correct node setup: Both nodes are essential for professional HDR output

    The Linear EXR Export node will appear in the image category in ComfyUI.

    āš ļø Warning: Do NOT use ComfyUI's built-in Save Image node with HDR data - it will destroy all values above 1.0!

    Additional HDR Export Option

    For advanced HDR processing workflows with multi-exposure fusion, you can also use the HDR Export node from the Luminance Stack Processor package available through ComfyUI Manager.

    Results & Performance

    The intelligent HDR approach successfully preserves HDR data while maintaining professional image quality:

    • HDR Preservation: Maintains hundreds of thousands of HDR pixels (>1.0 values) through the entire pipeline
    • Dynamic Range: Conservative mode (default): 1.5x range, Moderate: up to 9.0x, Exposure/Aggressive: 10+ range
    • Image Quality: Natural appearance with selective highlight expansion, not false color artifacts
    • Processing Speed: ~40-42 seconds for 752Ɨ1328 images (similar to standard VAE decode)
    • Memory Efficiency: Float32 pipeline with smart device management (CUDA/CPU synchronization)

    šŸ” Quality Comparison Examples

    The HDR VAE Decode delivers dramatically superior results with preserved highlight details and enhanced color reproduction:

    Standard VAE Decode Standard VAE decode: Limited dynamic range, clipped highlights, reduced color vibrancy

    HDR VAE Decode
    HDR VAE Decode: Extended highlights preserved, enhanced color depth, natural bokeh detail

    Key Improvements Visible:

    • āœ… Highlight Preservation: Bright bokeh elements retain detail instead of clipping to white
    • āœ… Enhanced Color Range: Richer, more vibrant colors throughout the image
    • āœ… Natural Gradients: Smooth transitions in bright areas instead of hard clipping
    • āœ… Professional Quality: VFX-ready output suitable for compositing and grading

    For direct comparison of output quality, reference examples are also provided in the /images folder:

    • Sample_builtin_vae_decode.png - Output from ComfyUI's built-in VAE decode (0-1 range, limited dynamic range)
    • Sample_hdr_vae_decode.exr - Output from HDR VAE Decode (extended range, preserved highlights and shadows)

    Technical Achievements

    Through this project, several challenging problems have been solved:

    1. Identified the root cause: VAE's conv_out layer applies sigmoid normalization, not simple clamping
    2. Developed MAX pooling: Preserves HDR peaks during 128ā†’3 channel conversion (vs averaging)
    3. Created intelligent expansion: Uses VAE's tone mapping + selective highlight extension
    4. Implemented inverse sigmoid recovery: Advanced mathematical techniques to recover extreme highlight details (contributed by Antonio Lisboa M. Neto)
    5. Added professional clamping: Nuke-style clamp controls to manage outliers while preserving detail
    6. Solved device synchronization: Proper CUDA/CPU tensor management across processing stages
    7. Built robust fallbacks: Multiple processing paths ensure reliability

    Compatibility

    šŸŽÆ VAE Model Support

    • āœ… Fully Supported: Flux.1 Dev VAE model (primary target)
    • āš ļø Experimental: Other VAE architectures (may work but not guaranteed)
    • āŒ Not Recommended: SD 1.5, SDXL, or other non-Flux VAE models without testing

    Important: This node was specifically engineered for Flux.1's VAE architecture. The intelligent analysis, bypass methods, and channel reduction logic are optimized for Flux.1's specific:

    • 128-channel intermediate representations
    • Up-block structure with ResNet + transposed convolution
    • conv_out sigmoid normalization behavior

    While the fallback methods may work with other models, optimal HDR preservation is only guaranteed with Flux.1.

    šŸ–„ļø System Requirements

    • ComfyUI: Stable versions (tested with latest releases)
    • Python: 3.8+ required
    • Hardware: CUDA-capable GPU strongly recommended for performance
    • Memory: Sufficient VRAM for your target resolution + model

    šŸ“ Output Compatibility

    • Formats: Professional EXR with Linear EXR Export node
    • Software: Nuke, After Effects, DaVinci Resolve, Blender
    • Workflow: VFX compositing and color grading pipelines

    šŸ” How to Verify Your Workflow is Correct

    HDR VAE Workflow āœ… Correct workflow: HDR VAE Decode connected to Linear EXR Export

    1. Check Node Connection: HDR VAE Decode ā†’ Linear EXR Export (not Save Image)
    2. Look for Console Output: Should show "HDR pixels: [number] >0" in Linear EXR Export logs
    3. Verify File Size: True HDR EXR files are typically 2-5MB+ for 1K images (vs <1MB for fake HDR)
    4. Test in VFX Software: Load in Nuke/After Effects and adjust exposure - should see extended highlights
    5. Compare with Examples: Check /images/Sample_builtin_vae_decode.png vs /images/Sample_hdr_vae_decode.exr to see the quality difference

    Troubleshooting

    OpenImageIO Installation Issues

    Windows Users:

    pip install OpenImageIO

    If you encounter build errors, you can try:

    • Download pre-built wheels from PyPI
    • Use conda: conda install -c conda-forge openimageio

    Linux Users:

    pip install OpenImageIO
# Or use your package manager:
sudo apt-get install python3-openimageio  # Debian/Ubuntu
sudo yum install python3-OpenImageIO      # RHEL/CentOS

    Mac Users:

    pip install OpenImageIO
# Or use Homebrew:
brew install openimageio
pip install OpenImageIO

    If OpenImageIO fails to install:

    • The node will automatically fall back to OpenCV for EXR writing
    • HDR data is still preserved with the OpenCV fallback
    • You just won't have access to advanced compression options and precise bit depth control
    • Check console logs for fallback messages

    Common Issues

    Problem: "OpenImageIO not available" warning in console
    Solution: Install OpenImageIO using the commands above, or ignore if using OpenCV fallback

    Problem: EXR files don't show HDR values in Nuke/After Effects
    Solution: Make sure you're using the Linear EXR Export node, not ComfyUI's built-in Save Image node

    Problem: Memory errors during processing
    Solution: Reduce resolution or adjust batch size, HDR processing requires more VRAM

    Problem: Colors look different than standard output
    Solution: This is expected - the node preserves linear color space. Apply proper color management in your compositing software

    Development Notes

    This project represents months of research into VAE architecture and HDR processing. The approach evolved from simple layer bypassing to sophisticated analysis-based processing. While the current implementation works well with Flux.1 Dev, different VAE models may require adjustments to the analysis logic.

    Key learnings:

    • Simple bypassing often breaks image quality - the VAE's processing pipeline exists for good reasons
    • Averaging destroys HDR data - MAX pooling is essential for preserving brightness peaks
    • Device synchronization is critical - mixed CPU/CUDA operations cause subtle failures
    • Multiple fallback paths improve reliability - complex pipelines need robust error handling

    Contributors

    Special thanks to those who have contributed to advancing HDR processing in ComfyUI:

    šŸŒŸ Top Contributors

    • Antonio Lisboa M. Neto - Contributed inverse sigmoid recovery techniques that enabled the advanced Luminance Recovery and Highlight Fusion modes, significantly enhancing highlight detail preservation capabilities.

    Contributing

    This project welcomes contributions! If you encounter issues or have improvements:

    How to Contribute:

    1. Fork the repository
    2. Create a feature branch (git checkout -b feature/amazing-feature)
    3. Commit your changes (git commit -m 'Add amazing feature')
    4. Push to the branch (git push origin feature/amazing-feature)
    5. Open a Pull Request

    This is an experimental node that pushes the boundaries of what's possible with VAE decoding - your contributions help advance HDR processing in ComfyUI!

    License

    This project is licensed under the MIT License - see the LICENSE file for details.

    MIT License Summary:

    āœ… Commercial use
    āœ… Modification
    āœ… Distribution
    āœ… Private use

    Repository: https://github.com/sumitchatterjee13/vae-decode-hdr
    Author: Sumit Chatterjee
    Created: 2025

    ā­ If this project helps you, please consider giving it a star on GitHub!