ComfyUI-Qwen2-VL-Nodes

ComfyUI-Qwen2-VL-Nodes

A complete set of custom nodes for ComfyUI that integrates the Qwen2-VL vision-language model. This package features the all-in-one QWEN VLM MACHINE which supports both Transformers (Safetensors) and GGUF (llama.cpp) loading, along with full image and video support!

Included Nodes

1. QWEN VLM MACHINE (Safetensors): Loads the model efficiently with transformers (supports 4-bit, 8-bit, fp16). Handles IMAGE, VIDEO (lists of frames), and custom instructions.
2. QWEN VLM MACHINE (GGUF): Connects to optimized GGUF versions of Qwen2-VL using llama-cpp-python. Great for users with lower VRAM or those on Mac/CPU setups!
3. Qwen2-VL Text Preview: Display generated text directly in your ComfyUI workflow.
4. Qwen2-VL Prompt Generator: Analyze an image and generate positive and negative prompts formatted in detailed, booru, simple, or artistic styles.

Installation

1. Navigate to your ComfyUI custom nodes directory:

   cd ComfyUI/custom_nodes
   

2. Clone this repository (or copy the ComfyUI-Qwen2-VL-Nodes folder):

   git clone <repository_url>
   

3. Install the requirements:

   cd ComfyUI-Qwen2-VL-Nodes
   pip install -r requirements.txt
   

   (Note: llama-cpp-python is required if you plan to use the GGUF node. You might need C++ build tools installed on Windows to install it successfully).

How to use GGUF Models

To use the QWEN VLM MACHINE (GGUF) node, you need two files from huggingface:

1. The actual language model (e.g. Qwen2-VL-7B-Instruct-Q4_K_M.gguf)
2. The vision connector/mmproj model (e.g. qwen2-vl-7b-vision.gguf)

Place both of these in your models/LLM or models/checkpoints directory, and point the node to them!

Enjoy!