ComfyUI Extension: StereoVision Plugin for ComfyUI

Authored by DrMWeigand

Created

Updated

9 stars

A ComfyUI node for producing stereoscopic and autostereogram (magic eye) images and videos.

Custom Nodes (2)

README

StereoVision Plugin for ComfyUI

The StereoVision plugin for ComfyUI enables the creation of stereoscopic and autostereoscopic images and videos using depth maps. It supports both traditional stereoscopic image generation and autostereogram (Magic Eye) creation.

Components

The plugin provides four primary nodes:

  1. Stereoscopic Generator: Basic side-by-side stereoscopic images from base images and depth maps
  2. Stereoscopic Generator Pro: Advanced stereoscopic generation with sub-pixel accuracy and occlusion handling
  3. Stereoscopic Generator Max: Maximum quality stereoscopic generation with forward warping and multi-scale inpainting
  4. Autostereogram Generator: Produces autostereograms (Magic Eye images) from depth maps

Stereoscopic Generator

Usage

  1. Input Requirements

    • Base Image: The source image for conversion
    • Depth Map: Corresponding depth information (lighter = closer, darker = further)

  2. Node Setup

    • Add the Stereoscopic Generator node
    • Connect base image to base_image input
    • Connect depth map to depth_map input

  3. Parameters

    • depth_scale: Controls 3D effect intensity (default: 80)
      • Higher values increase depth separation
      • Lower values produce subtler effects

Technical Considerations

  • Image Selection

    • Clear foreground/background separation improves results
    • Consistent depth mapping enhances effect quality
    • High-contrast scenes may require lower depth scale values

  • Processing Requirements

    • Higher resolution inputs require more processing time
    • Video processing may need significant computational resources

Example Output

Seattle (Use cross-eyed technique to view 3D)

<img src="example_images/Seattle.png" alt="Seattle (Use cross-eyed technique to view 3D)" width="300"/>

Video Demonstration

Stereoscopic Generator Demo

Stereoscopic Generator Pro

Overview

The Stereoscopic Generator Pro offers advanced stereoscopic image generation using symmetric Depth Image Based Rendering (DIBR) with sub-pixel accuracy. This node provides superior occlusion handling and high-quality hole filling, making it ideal for professional-quality stereoscopic content.

Key Features

  • Symmetric DIBR: Creates balanced left and right views with proper depth warping
  • Sub-pixel Accuracy: Smooth interpolation for high-quality output
  • Advanced Occlusion Handling: Detects and properly handles occluded regions
  • Multiple Inpainting Options: Choose between fast and high-quality hole filling methods
  • Edge-Preserving Depth Filtering: Maintains sharp object boundaries
  • Temporal Consistency: Reduces flicker in video sequences
  • 4K+ Optimization: Stripe-based processing for memory-efficient handling of ultra-high resolution content
  • GPU Acceleration: Optional GPU processing with fp16 precision for faster 4K rendering

Usage

  1. Input Requirements

    • Base Image: Source image for conversion
    • Depth Map: Corresponding depth information

  2. Parameters

    • max_disparity_pct (0.5-5.0, default: 1.5)
      • Maximum disparity as percentage of image width
      • Higher values create stronger 3D separation
    • depth_gamma (0.3-2.0, default: 0.8)
      • Gamma correction for depth values
      • Adjusts the depth response curve
    • inpaint_radius (1-15, default: 3)
      • Radius for hole filling operations
      • Larger values for smoother fills
    • mask_dilate (0-10, default: 2)
      • Dilation amount for occlusion masks
      • Helps catch edge artifacts
    • swap_for_cross (boolean, default: False)
      • Swaps left/right images for cross-eyed viewing
    • use_patchmatch (boolean, default: False)
      • Enables advanced PatchMatch-inspired inpainting
      • Better for large holes but slower
    • edge_filter (boolean, default: True)
      • Applies edge-preserving filter to depth map
      • Improves object boundaries
    • temporal_seed (boolean, default: False)
      • Uses previous frame data to reduce flicker
      • Essential for smooth video sequences
    • fast_mode (boolean, default: False)
      • Enables performance optimizations for faster processing
      • Trades some quality for significant speed improvements
      • Recommended for real-time preview or draft renders

  3. Optional Parameters (for 4K+ optimization)

    • stripe_width (256-4096, default: 1024)
      • Width of processing stripes for memory efficiency
      • Smaller values use less memory but may be slower
      • Larger values are faster but use more memory
    • overlap (1-16, default: 2)
      • Overlap between stripes to avoid seams
      • Higher values provide smoother transitions
    • gpu (boolean, default: True)
      • Enable GPU acceleration with fp16 precision
      • Significantly faster for 4K+ content
      • Falls back to CPU if CUDA unavailable
    • stretch_thresh (0.2-1.5, default: 0.5)
      • Threshold for stretch-based occlusion detection
      • Lower values detect more occlusions
      • Higher values are more conservative

When to Use

Choose Stereoscopic Generator Pro when you need:

  • High-quality results with minimal artifacts
  • Better handling of complex scenes with occlusions
  • Smooth video sequences with temporal consistency
  • Fine control over depth processing
  • 4K+ resolution support with memory-efficient processing
  • GPU acceleration for faster rendering of high-resolution content

Stereoscopic Generator Max

Overview

The Stereoscopic Generator Max represents the highest quality stereoscopic generation available, utilizing forward warping with z-buffer, multi-scale inpainting, and optional GPU acceleration. This node is designed for maximum quality output when processing time is not a primary concern.

Key Features

  • Forward Warping with Z-Buffer: Most accurate depth-based rendering
  • Multi-Scale Inpainting: Sophisticated hole filling for large occlusions
  • GPU Acceleration: Optional CUDA support for faster processing
  • Advanced Temporal Blending: Superior video stability
  • Adaptive Inpainting: Automatically selects best method based on hole size
  • Guided Depth Filtering: Preserves fine details while smoothing depth

Usage

  1. Input Requirements

    • Base Image: Source image for conversion
    • Depth Map: Corresponding depth information

  2. Parameters

    • max_disparity_pct (0.5-5.0, default: 1.5)
      • Maximum disparity as percentage of image width
    • depth_gamma (0.3-2.0, default: 0.8)
      • Gamma correction for depth values
    • inpaint_method (options: adaptive, telea, ns, mixed, default: adaptive)
      • adaptive: Automatically chooses method based on hole size
      • telea: Fast method, good for small holes
      • ns: Navier-Stokes, better for medium holes
      • mixed: Two-pass approach for best quality
    • edge_smoothing (0-10, default: 3)
      • Smoothing level for depth edges
      • Higher values reduce depth noise
    • temporal_weight (0.0-1.0, default: 0.7)
      • Blending weight with previous frame
      • Higher values increase temporal stability
    • swap_for_cross (boolean, default: False)
      • Swaps left/right images for cross-eyed viewing
    • use_gpu (boolean, default: False)
      • Enables GPU acceleration if CUDA is available

When to Use

Choose Stereoscopic Generator Max when you need:

  • Absolute highest quality output
  • Processing of high-resolution content
  • Complex scenes with significant occlusions
  • Professional production requirements

Choosing the Right Stereoscopic Generator

Comparison Guide

| Feature | Basic | Pro | Pro (Fast Mode) | Max | |---------|-------|-----|-----------------|-----| | Speed | Fast | Medium | Fast | Slow | | Quality | Good | Excellent | Very Good | Best | | Occlusion Handling | Basic | Advanced | Good | Superior | | Temporal Stability | None | Good | Good | Excellent | | GPU Support | No | Yes (fp16) | No | Yes | | Memory Usage | Low | Low* | Low-Medium | High | | 4K+ Support | Limited | Excellent | Good | Good | | Max Resolution | 2K | 8K+ | 4K | 4K |

*With stripe-based processing enabled

Recommendations

  • Stereoscopic Generator (Basic)

    • Quick previews and drafts
    • Real-time or near real-time processing
    • Simple scenes with minimal occlusions
    • Lower resolution content

  • Stereoscopic Generator Pro

    • Production-quality output
    • Complex scenes with multiple depth layers
    • Video sequences requiring temporal consistency
    • Balance between quality and processing time
    • Use fast_mode=True for preview renders with near-Basic speed
    • Recommended for 4K-8K content with GPU enabled
    • Adjust stripe_width based on available memory (smaller for limited VRAM)

  • Stereoscopic Generator Max

    • Final renders and master copies
    • High-resolution content (4K+)
    • Scenes with complex occlusions
    • When processing time is not a constraint

Autostereogram Generator

Usage

  1. Input Requirements

    • Depth Map: Defines the hidden 3D image
    • Optional Texture: Custom pattern for the autostereogram

  2. Core Parameters

    • pattern_div: Pattern width (integer)

      • Low values: Larger patterns, easier viewing
      • High values: Smaller patterns, more detail

    • depth_multiplier: Depth effect intensity

    • invert: Reverses depth perception

    • x_tiles, y_tiles: Pattern repetition count

  3. Pattern Configuration

    • Pattern Types:

      • Random
      • Perlin
      • Dots
      • Lines
      • Checkers
      • Waves

    • Noise Parameters:

      • noise_scale: Pattern size
      • noise_octaves: Pattern complexity

    • Color Options:

      • Grayscale
      • RGB
      • Complementary

  4. Batch Processing Controls

    • clamp_depth_to_first: When enabled, uses the first depth map frame for all outputs
    • clamp_texture_to_first: When enabled, uses the first texture frame for all outputs
    • These options are useful for:
      • Creating consistent autostereograms across multiple frames
      • Maintaining pattern continuity in animations
      • Reducing visual noise in batch processing

  5. Output Settings

    • Custom width/height support
    • Default: matches depth map dimensions

Technical Parameters

Pattern Division Effects:

  • 4: Large patterns, basic depth perception
  • 16: Fine patterns, detailed hidden images

Noise Configuration:

  • Scale Range: 0.5 (fine) to 2.0 (coarse)
  • Octaves: 2-6 (higher = more complex)

Example Output

Harold (texture based)

Use parallel viewing technique to see the hidden 3D image <img src="example_images/Harold_Image.png" alt="Harold (texture based)" width="60%"/>

Harold (noise based)

Use parallel viewing technique to see the hidden 3D image <img src="example_images/Harold_Noise.png" alt="Harold (noise based)" width="60%"/>

Video Demonstration

Autostereogram Generator Demo (Max Resolution: 1920x1080)

Implementation Notes

  1. Performance Optimization

    • Match output resolution to viewing requirements
    • Consider computational load for video processing
    • Monitor memory usage with high-resolution inputs

  2. Quality Control

    • Verify depth map accuracy
    • Test various pattern settings for optimal viewing
    • Validate output across different display sizes

  3. Error Prevention

    • Ensure consistent input dimensions
    • Validate depth map grayscale values
    • Monitor processing resource usage

Technical Specifications

  • Input Formats: Standard image formats
  • Output Resolution: Configurable or source-matched
  • Processing: CPU-based computation
  • Memory Requirements: Scales with input resolution

Example Implementation

For detailed implementation examples and advanced usage scenarios, refer to the plugin's source code and example workflows.