ComfyUI Extension: MiniMax Video Object Remover Suite

Authored by casterpollux

Created

Updated

41 stars

Professional video object removal suite using MiniMax optimization. Includes BMO-enhanced nodes with VAE normalization, temporal preservation, and 6-step inference. Complete video inpainting solution for ComfyUI.

Custom Nodes (0)

    README

    ๐ŸŽฏ MiniMax-Remover for ComfyUI

    High-quality video object removal with automatic model management and universal resolution support

    <p align="center"> <a href="https://huggingface.co/zibojia/minimax-remover"><img alt="Huggingface Model" src="https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-Model-brightgreen"></a> <a href="https://github.com/zibojia/MiniMax-Remover"><img alt="Github" src="https://img.shields.io/badge/MiniMaxRemover-github-black"></a> <a href="https://arxiv.org/abs/2505.24873"><img alt="arXiv" src="https://img.shields.io/badge/MiniMaxRemover-arXiv-b31b1b"></a> </p>

    demo 1

    demo 2

    demo 3

    demo 4


    ๐Ÿš€ Major Improvements & Features

    โœจ Latest Enhancements

    • โœ… Auto-Download Models: Zero-configuration setup - models download automatically on first use
    • โœ… Universal Resolution Support: Works with ANY resolution input (480p to 4K+)
    • โœ… Smart VAE Compatibility: Automatic dimension adjustment for perfect compatibility
    • โœ… OpenCV Conflict Resolution: No more OpenCV reinstallation issues
    • โœ… DW Preprocessor Compatibility: Works seamlessly with DWPose and other preprocessors
    • โœ… Tensor Dimension Fixes: Eliminates all dimension mismatch errors
    • โœ… Professional Quality: Proper VAE normalization for natural, high-quality results

    ๐ŸŽฎ Core Features

    • Fast: Only 6-12 inference steps, highly optimized
    • Robust: Handles any mask/video resolution combination automatically
    • Plug-and-Play: Install and use immediately - no manual setup required
    • Production Ready: Comprehensive error handling and diagnostics

    ๐Ÿ“ฅ Installation

    Method 1: ComfyUI Manager (Recommended)

    1. Open ComfyUI Manager
    2. Click "Install Custom Nodes"
    3. Search for "MiniMax-bmo"
    4. Click Install and restart ComfyUI

    Method 2: Manual Installation

    cd ComfyUI/custom_nodes
    git clone https://github.com/CasterPollux/MiniMax-bmo.git
    cd MiniMax-Remover
    # Install with CUDA support (recommended)
    pip install -r requirements.txt
    

    Method 3: Automatic Setup Script

    # Run the setup script for automatic ComfyUI integration
    python setup_comfyui_integration_bmo.py
    

    ๐Ÿค– Model Management - Auto-Download System

    Zero-Configuration Experience (Default)

    When you first use the node:

    ๐Ÿ” Checking for MiniMax-Remover models...
    ๐Ÿ“ฅ Models not found locally. Starting automatic download...
    ๐ŸŒ Downloading MiniMax-Remover models to: ./models
    ๐Ÿ“Š Expected download size: ~25-30 GB
    โณ This may take several minutes depending on your connection...
    ๐ŸŽ‰ Models downloaded successfully!
    โœ… Processing your video...
    

    Model Download Details

    • Total Size: ~2-3 GB (one-time download)
    • Components:
      • VAE (~500mb) - Video encoding/decoding
      • Transformer (~2GB) - Main diffusion model
      • Scheduler (~1KB) - Denoising control (json only)
    • Source: HuggingFace: zibojia/minimax-remover

    Storage Locations (Auto-Detected)

    The system automatically tries these locations in priority order:

    Option 1: Custom Node Directory (Default)

    ComfyUI/custom_nodes/MiniMax-Remover/models/
    โ”œโ”€โ”€ minimax_vae/          (VAE encoder/decoder)
    โ”œโ”€โ”€ minimax_transformer/  (main diffusion model)
    โ””โ”€โ”€ minimax_scheduler/    (denoising scheduler)
    

    Option 2: ComfyUI Models Directory

    ComfyUI/models/
    โ”œโ”€โ”€ minimax_vae/          (MiniMax VAE models)
    โ”œโ”€โ”€ minimax_transformer/  (MiniMax Transformer models)
    โ””โ”€โ”€ minimax_scheduler/    (MiniMax Scheduler configs)
    

    Option 3: User Cache Directory (Fallback)

    ~/.cache/minimax-remover/models/
    โ”œโ”€โ”€ minimax_vae/
    โ”œโ”€โ”€ minimax_transformer/
    โ””โ”€โ”€ minimax_scheduler/
    

    Windows: C:\Users\[Username]\.cache\minimax-remover\models\

    Manual Download Options

    If auto-download fails or you prefer manual control:

    # Option 1: Use built-in script
    python download_models.py
    
    # Option 2: Direct HuggingFace download
    huggingface-cli download zibojia/minimax-remover --local-dir ./models --local-dir-use-symlinks False
    # Then rename folders for clarity:
    mv ./models/vae ./models/minimax_vae
    mv ./models/transformer ./models/minimax_transformer  
    mv ./models/scheduler ./models/minimax_scheduler
    
    # Option 3: Download to ComfyUI models directory  
    huggingface-cli download zibojia/minimax-remover --local-dir ComfyUI/models --local-dir-use-symlinks False
    # Then rename folders for clarity:
    mv ComfyUI/models/vae ComfyUI/models/minimax_vae
    mv ComfyUI/models/transformer ComfyUI/models/minimax_transformer
    mv ComfyUI/models/scheduler ComfyUI/models/minimax_scheduler
    

    ๐Ÿ“‹ For detailed setup instructions, see AUTO_DOWNLOAD_GUIDE.md


    ๐ŸŽฏ Universal Resolution Support

    ANY Resolution Now Supported!

    The node now handles any resolution input with automatic compatibility:

    • โœ… Any input resolution - 480p, 720p, 1080p, 4K, custom resolutions
    • โœ… Mixed mask/image sizes - automatically handles different resolution inputs
    • โœ… VAE compatibility - dimensions automatically adjusted for perfect compatibility
    • โœ… No configuration needed - works automatically

    Example Resolution Fixes

    | Input Resolution | Problem | Auto-Fixed To | Result | |------------------|---------|---------------|--------| | 1000x1778 | 125x222.25 latent | 1008x1784 | 126x223 โœ… | | 720x480 | No issue | 720x480 | 90x60 โœ… | | 1920x1080 | 240x135 | 1920x1088 | 240x136 โœ… |

    ๐Ÿ“‹ For technical details, see TENSOR_DIMENSION_FIX_GUIDE.md


    ๐Ÿ› ๏ธ Compatibility Fixes

    OpenCV Conflict Resolution โœ…

    Problem Solved: Users were experiencing OpenCV reinstallation prompts and conflicts.

    Root Cause: Multiple conflicting OpenCV version requirements across different files:

    • requirements.txt: opencv-python==4.5.5.64 (exact old version)
    • requirements_bmo.txt: opencv-python>=4.8.0 (newer minimum)
    • pyproject.toml: opencv-python>=4.7.0 (different minimum)

    Solution Applied:

    • โœ… Standardized OpenCV requirements to opencv-python>=4.5.0,<5.0.0 across all files
    • โœ… Safe installation procedures that don't break existing OpenCV installations
    • โœ… Recovery scripts for users with broken installations

    Result: No more OpenCV reinstallation prompts or conflicts!

    DW Preprocessor Compatibility โœ…

    Problem Solved: Conflicts between MiniMax-Remover and DWPose/DW preprocessors.

    Root Cause: Multiple compatibility issues:

    • PyTorch version conflicts (torch==2.6 vs DWPose requirement <2.4)
    • Segment Anything model loading conflicts
    • ONNX runtime conflicts
    • CUDA memory issues from simultaneous GPU usage

    Solution Applied:

    • โœ… Compatible PyTorch range: torch>=2.0.0,<2.5.0 for DWPose compatibility
    • โœ… TorchVision compatibility: Proper version alignment
    • โœ… Memory management: Smart GPU memory handling
    • โœ… Model isolation: Prevents loading conflicts

    Result: MiniMax-Remover now works seamlessly with DWPose and other preprocessors!

    ๐Ÿ“‹ For detailed compatibility info, see DW_PREPROCESSOR_COMPATIBILITY.md


    ๐Ÿš€ Usage

    In ComfyUI

    1. Add the Node: Look for "MiniMax-Remover (BMO)" in the MiniMax-Remover category

    2. Connect Inputs:

      • images: Your video frames as IMAGE type
      • masks: Your binary masks as MASK type
    3. Configure (Optional):

      • num_inference_steps: 12 (default, optimal quality/speed balance)
      • iterations: 6 (mask expansion, default works for most cases)
      • seed: Any number for reproducible results
      • auto_download: True (default, enables automatic model management)
    4. Run: Perfect results with any resolution combination!

    Node Parameters

    Core Settings

    • auto_download: True (default) - Automatic model management
    • num_inference_steps: 12 (default) - Quality/speed balance
    • iterations: 6 (default) - Mask expansion iterations
    • seed: 42 (default) - Random seed for reproducible results

    Advanced Settings (Auto-Detected)

    • vae_path: "auto" (default) - Auto-detected VAE path
    • transformer_path: "auto" (default) - Auto-detected Transformer path
    • scheduler_path: "auto" (default) - Auto-detected Scheduler path

    What You'll See

    First Use (Auto-Download)

    ๐Ÿ”ง Using BMO MiniMax-Remover Pipeline (Flexible Resolution)
    ๐Ÿ“ Input Analysis:
       Images: torch.Size([165, 1920, 1080, 3])
       Masks: torch.Size([165, 1024, 576, 1])
       Using input dimensions as target: 1920x1080
    
    ๐Ÿ”ง Auto-resizing for compatibility to 1920x1080
       VAE-compatible target: 1920x1088
       Resizing masks: 1024x576 -> 1920x1088
    
    ๐ŸŽ‰ Models downloaded successfully with descriptive names!
    ๐Ÿ“ MiniMax VAE: D:\ComfyUI\custom_nodes\MiniMax-Remover\models\minimax_vae
    ๐Ÿ“ MiniMax Transformer: D:\ComfyUI\custom_nodes\MiniMax-Remover\models\minimax_transformer
    ๐Ÿ“ MiniMax Scheduler: D:\ComfyUI\custom_nodes\MiniMax-Remover\models\minimax_scheduler
    โœ… Processing complete - perfect results!
    

    Subsequent Uses (Instant)

    ๐Ÿ” Checking for MiniMax-Remover models...
    โœ… Found existing models at: D:\ComfyUI\custom_nodes\MiniMax-Remover\models (descriptive names)
       VAE: D:\ComfyUI\custom_nodes\MiniMax-Remover\models\minimax_vae
       Transformer: D:\ComfyUI\custom_nodes\MiniMax-Remover\models\minimax_transformer
       Scheduler: D:\ComfyUI\custom_nodes\MiniMax-Remover\models\minimax_scheduler
    โœ… BMO MiniMax-Remover models loaded successfully!
    ๐Ÿš€ Processing your video...
    

    ๐Ÿ”ง Troubleshooting

    Common Issues & Solutions

    Models Not Downloading

    # Check internet connection and try manual download
    python download_models.py
    
    # Or use HuggingFace CLI
    huggingface-cli download zibojia/minimax-remover --local-dir ./models
    

    PyTorch CUDA Issues

    # If PyTorch gets downgraded to CPU version during installation:
    # 1. Reinstall PyTorch with CUDA support first
    pip install torch>=2.0.0,<2.8.0 torchvision>=0.15.0,<0.20.0 --index-url https://download.pytorch.org/whl/cu121
    
    # 2. Then install other dependencies
    pip install -r requirements.txt --no-deps --force-reinstall diffusers transformers accelerate
    

    OpenCV Conflicts

    # Install with compatible versions
    pip install -r requirements.txt
    
    # If issues persist, see INSTALLATION_GUIDE_OPENCV_FIX.md
    

    DW Preprocessor Conflicts

    # Use compatible PyTorch version
    pip install "torch>=2.0.0,<2.5.0" "torchvision>=0.15.0,<0.20.0"
    
    # See DW_PREPROCESSOR_COMPATIBILITY.md for details
    

    Memory Issues

    • Use smaller input resolutions
    • Enable model offloading in ComfyUI settings
    • Close other GPU-intensive applications

    Poor Quality Results

    • Ensure you're using the latest BMO version
    • Try different seeds (use seed parameter)
    • Adjust mask expansion (iterations parameter: 6-10)
    • Check that masks are clean and binary

    ๐Ÿ“ Project Structure

    The MiniMax-Remover models can be stored in two different locations. Check both locations to find your models:

    MiniMax-Remover/
    โ”œโ”€โ”€ ๐Ÿ“„ README.md (this file)
    โ”œโ”€โ”€ ๐Ÿ”ง minimax_mask_node_bmo.py (main ComfyUI node)
    โ”œโ”€โ”€ ๐Ÿ”ง pipeline_minimax_remover_bmo.py (processing pipeline)
    โ”œโ”€โ”€ ๐Ÿ”ง transformer_minimax_remover.py (model architecture)
    โ”œโ”€โ”€ ๐Ÿ“ฆ requirements.txt (dependencies)
    โ”œโ”€โ”€ ๐Ÿ“ models/ (auto-downloaded models with descriptive names)
    โ”‚   โ”œโ”€โ”€ minimax_vae/ (VAE encoder/decoder)
    โ”‚   โ”œโ”€โ”€ minimax_transformer/ (main diffusion model)
    โ”‚   โ””โ”€โ”€ minimax_scheduler/ (denoising scheduler)
    โ”œโ”€โ”€ ๐Ÿ“‹ Documentation/
    โ”‚   โ”œโ”€โ”€ AUTO_DOWNLOAD_GUIDE.md
    โ”‚   โ”œโ”€โ”€ TENSOR_DIMENSION_FIX_GUIDE.md
    โ”‚   โ”œโ”€โ”€ DW_PREPROCESSOR_COMPATIBILITY.md
    โ”‚   โ”œโ”€โ”€ INSTALLATION_GUIDE_OPENCV_FIX.md
    โ”‚   โ””โ”€โ”€ OPENCV_FIX_SUMMARY.md
    โ””โ”€โ”€ ๐Ÿ› ๏ธ Scripts/
        โ”œโ”€โ”€ download_models.py
        โ”œโ”€โ”€ setup_comfyui_integration_bmo.py
        โ””โ”€โ”€ fix_comfyui_diffusers.py
    

    Option 2: ComfyUI Models Directory Structure (Alternative)

    ComfyUI/
    โ”œโ”€โ”€ models/
    โ”‚   โ”œโ”€โ”€ checkpoints/ (Stable Diffusion models)
    โ”‚   โ”œโ”€โ”€ vae/ (Standard VAE models)  
    โ”‚   โ”œโ”€โ”€ loras/ (LoRA models)
    โ”‚   โ”œโ”€โ”€ minimax_vae/ (๐ŸŽฏ MiniMax VAE - auto-downloaded here)
    โ”‚   โ”œโ”€โ”€ minimax_transformer/ (๐ŸŽฏ MiniMax Transformer - auto-downloaded here)
    โ”‚   โ””โ”€โ”€ minimax_scheduler/ (๐ŸŽฏ MiniMax Scheduler - auto-downloaded here)
    โ””โ”€โ”€ custom_nodes/
        โ””โ”€โ”€ MiniMax-Remover/
            โ”œโ”€โ”€ ๐Ÿ“„ README.md
            โ”œโ”€โ”€ ๐Ÿ”ง minimax_mask_node_bmo.py
            โ”œโ”€โ”€ ๐Ÿ”ง pipeline_minimax_remover_bmo.py
            โ””โ”€โ”€ ๐Ÿ“ฆ requirements.txt
    

    ๐Ÿ’ก Tip: When the node runs, it will display the actual local paths where your models are found, so you'll know exactly which location is being used.


    ๐Ÿ“š Documentation

    Complete Guides Available

    Support Resources

    • GitHub Issues: Report bugs and request features
    • Documentation: Comprehensive guides for all features
    • Community: Share workflows and tips

    Before vs After

    โŒ Old Experience

    - Manual model download required (25GB)
    - Resolution errors with non-standard sizes
    - OpenCV conflicts breaking installations  
    - DW preprocessor incompatibility
    - Complex setup and configuration
    

    โœ… New Experience

    - Plug-and-play: install and use immediately
    - Works with ANY resolution automatically
    - No OpenCV conflicts or reinstallations
    - Perfect DW preprocessor compatibility
    - Professional-quality results every time
    

    ๐Ÿ”— Links & Resources

    Official Project

    ComfyUI Integration


    ๐Ÿ“ง Contact & Support


    ๐Ÿ“œ License

    This project is licensed under the terms specified in the LICENSE file.


    ๐Ÿ™ Credits

    Original Authors

    Bojia Zi*, Weixuan Peng*, Xianbiao Qiโ€ , Jianan Wang, Shihao Zhao, Rong Xiao, Kam-Fai Wong
    *Equal contribution. โ€ Corresponding author.

    ComfyUI Integration & Enhancements

    • Auto-download system implementation
    • Universal resolution support
    • OpenCV & DW preprocessor compatibility fixes
    • Comprehensive documentation and guides

    ๐ŸŽจ Happy Video Inpainting! โœจ

    Experience professional-quality video object removal with zero configuration required.