ComfyUI-MVAdapter

This extension integrates MV-Adapter into ComfyUI, allowing users to generate multi-view consistent images from text prompts or single images directly within the ComfyUI interface.

🔥 Feature Updates

• [2024-12-09] Support integration with SDXL LoRA
• [2024-12-02] Generate multi-view consistent images from text prompts or a single image

Installation

From Source

• Clone or download this repository into your ComfyUI/custom_nodes/ directory.
• Install the required dependencies by running pip install -r requirements.txt.

Notes

Workflows

We provide the example workflows in workflows directory.

Note that our code depends on diffusers, and will automatically download the model weights from huggingface to the hf cache path at the first time. The ckpt_name in the node corresponds to the model name in huggingface, such as stabilityai/stable-diffusion-xl-base-1.0.

We also provide the nodes Ldm**Loader to support loading text-to-image models in ldm format. Please see the workflow files with the suffix _ldm.json.

GPU Memory

If your GPU resources are limited, we recommend using the following configuration:

• Use madebyollin/sdxl-vae-fp16-fix as VAE. If using ldm-format pipeline, remember to set upcast_fp32 to False.

• Set enable_vae_slicing in the Diffusers Model Makeup node to True.

However, since SDXL is used as the base model, it still requires about 13G to 14G GPU memory.

Usage

Text to Multi-view Images

With SDXL or other base models

• workflows/t2mv_sdxl_diffusers.json for loading diffusers-format models
• workflows/t2mv_sdxl_ldm.json for loading ldm-format models

With LoRA

workflows/t2mv_sdxl_ldm_lora.json for loading ldm-format models with LoRA for text-to-multi-view generation

Image to Multi-view Images

With SDXL or other base models

• workflows/i2mv_sdxl_diffusers.json for loading diffusers-format models
• workflows/i2mv_sdxl_ldm.json for loading ldm-format models

With LoRA

workflows/i2mv_sdxl_ldm_lora.json for loading ldm-format models with LoRA for image-to-multi-view generation