Qwen2.5-VL GGUF Nodes
ComfyUI nodes for running GGUF quantized Qwen2.5-VL models using llama.cpp
Nodes (30)
Caption a whole folder without babysitting a single-image node 500 times
A captioning instruction your VLM will actually obey (and your dataset will thank you)
Eight optional image ports, one clean batch — for VLM comparisons that don't fall over
It Doesn't Call Any API — It Just Turns Your JSON Into a Prompt
Stop trusting VLM JSON with your life — check it instead
Run a real LLM inside ComfyUI without ever leaving the graph
The node that un-wedges your VRAM when a loaded GGUF model won't let go
Compare one video and three images side by side in a single analysis call
A diagnostic dashboard for your Nexa SDK server and the models it can see
Make a vision model read your screenshots like it means it
Reverse-prompt any image into the prompt that (probably) made it
Point ComfyUI at Ollama, LM Studio, or Nexa without learning three different APIs
Ask a remote vision model 'what is this?' and get an answer your workflow can use
Give ComfyUI eyes through LM Studio or Ollama, without installing a single GGUF locally
Caption, OCR, reverse-prompt, JSON, or anything you want
Stop retyping the same captioner persona into every node
NodeGPT's bare-bones single-shot LLM call
The same collector, repackaged for the v2 menu — know what you're grabbing
A pure text LLM in ComfyUI — no vision, no server, just GGUF
Describe images and video on your own GPU, with sampling knobs that make it behave
Load a Qwen vision GGUF (and its mmproj) without a server in sight
Ask a local server's vision model anything, and get the answer as a string
Point a VLM at your local server without touching a config file
The pack's bread-and-butter task runner, now under the v2 menu
A local LLM that's actually inside your graph — with a thinking-mode output
Your VLM doesn't need all 600 frames — give it the 8 that matter
Actually asking your local Qwen2.5-VL what an image or video shows
The trickiest part of local vision models is finding the mmproj file. This node handles it.
HF Transformers instead of llama-cpp, abliterated models included
Don't know which vision model fits your VRAM? Ask the pack to think for you
ComfyUI-GGUF-VLM
<a id="中文"></a>
中文
ComfyUI-GGUF-VLM 是一个 GGUF-first 的 ComfyUI 多模态 VLM/LLM 扩展，聚焦 Qwen/GGUF、本地优先推理、LM Studio / Ollama / llama.cpp server，以及实用的 Caption、OCR、Prompt Reverse、JSON 和批量图像工作流。
说明：本项目不是云 API 大聚合，也不是模型 marketplace。OpenAI-compatible 是协议层，核心仍是本地/GGUF/VLM 工作流。
安装
cd ComfyUI/custom_nodes
git clone https://github.com/walke2019/ComfyUI-GGUF-VLM.git
cd ComfyUI-GGUF-VLM
pip install -r requirements.txt
ComfyUI Portable 用户请使用 ComfyUI 自带 Python 安装依赖。
快速使用
远程模式（推荐）
Remote Vision Model Config → Remote VLM Business Task ← IMAGE
推荐服务：
| 服务 | 默认端口 | 用途 | |---|---:|---| | LM Studio | 1234 | Windows / GGUF 图形化管理 | | Ollama | 11434 | 本地模型服务 | | llama.cpp server | 自定义 | 最新 GGUF 多模态服务 | | OpenAI-compatible | 自定义 | 兼容协议层 |
本地 GGUF 模式
Local Vision Model Loader (GGUF) → Local Image Analysis (GGUF) ← IMAGE
推荐模型目录：
ComfyUI/models/LLM/GGUF/
ComfyUI/models/text_encoders/
ComfyUI/models/clip/
ComfyUI/models/clip/gguf/
Nodes
远程 / Remote
| Node | 说明 | |---|---| | Remote Text Model Config | 远程文本模型配置 | | Remote Vision Model Config | 远程视觉模型配置 | | Text Generation | 文本生成 | | Remote Vision Analysis | 远程视觉/纯文本分析 |
本地 / Local
| Node | 说明 | |---|---| | Local Text Model Loader (GGUF) | 本地 GGUF 文本模型加载 | | Local Text Generation (GGUF) | 本地文本生成 | | Local Vision Model Loader (GGUF) | 本地 GGUF 视觉模型加载 | | Local Image Analysis (GGUF) | 本地图像/视频/纯文本分析 | | Vision Model Loader (Transformers) | 可选 Transformers 视觉模型 | | Multi-Image Analysis | 多图/视频帧分析 |
业务工作流 / Workflow
| Node | 说明 | |---|---| | Caption Prompt Builder | Caption / tags / dataset 提示词 | | OCR Prompt Builder | OCR 提示词 | | Prompt Reverse Builder | SDXL / Flux / Midjourney 反推提示词 | | JSON Extraction Prompt Builder | 结构化 JSON 提取提示词 | | Remote VLM Business Task | 单图 Caption/OCR/Prompt/JSON/Custom | | Batch Remote VLM Task | IMAGE 批量 Caption/OCR/JSON/Custom | | Dynamic Multi-Image Collector | 多图输入收集为 batch | | Video Frame Sampler | 视频帧抽样 | | VLM Model Advisor | 根据任务/显存/偏好推荐后端和模型 | | JSON Result Validator | 校验 JSON 输出字段 |
多语言 / i18n
本项目支持 ComfyUI 官方自定义节点多语言结构：
locales/en/nodeDefs.json
locales/zh/nodeDefs.json
当前提供英文和简体中文节点定义翻译。ComfyUI 会根据界面语言加载对应的 nodeDefs.json。
示例工作流
workflows/Example.jsonworkflows/V2_Business_Example.json
开发与测试
python -m unittest discover -s tests -v
python -m compileall __init__.py api_routes.py core nodes utils models config tests
CI 配置位于 .github/workflows/ci.yml。
License
MIT
<a id="english"></a>
English
ComfyUI-GGUF-VLM is a GGUF-first multimodal VLM/LLM extension for ComfyUI. It focuses on Qwen/GGUF, local-first inference, LM Studio / Ollama / llama.cpp server, and practical workflows such as captioning, OCR, prompt reverse, JSON extraction, and image batches.
Note: This project is not a cloud API aggregator or a model marketplace. OpenAI-compatible is only the protocol layer; the core remains local/GGUF/VLM workflows.
Installation
cd ComfyUI/custom_nodes
git clone https://github.com/walke2019/ComfyUI-GGUF-VLM.git
cd ComfyUI-GGUF-VLM
pip install -r requirements.txt
For ComfyUI Portable, install dependencies with ComfyUI's bundled Python.
Quick Start
Remote Mode (recommended)
Remote Vision Model Config → Remote VLM Business Task ← IMAGE
Recommended services:
| Service | Default Port | Use | |---|---:|---| | LM Studio | 1234 | Windows / GUI GGUF management | | Ollama | 11434 | Local model service | | llama.cpp server | custom | Latest multimodal GGUF service | | OpenAI-compatible | custom | Protocol compatibility layer |
Local GGUF Mode
Local Vision Model Loader (GGUF) → Local Image Analysis (GGUF) ← IMAGE
Recommended model directories:
ComfyUI/models/LLM/GGUF/
ComfyUI/models/text_encoders/
ComfyUI/models/clip/
ComfyUI/models/clip/gguf/
i18n
This project uses ComfyUI's official custom node i18n structure:
locales/en/nodeDefs.json
locales/zh/nodeDefs.json
English and Simplified Chinese node definition translations are included.
Development
python -m unittest discover -s tests -v
python -m compileall __init__.py api_routes.py core nodes utils models config tests
License
MIT