ComfyUI Node
ialhabbal VLLM Advanced
A ComfyUI node in 🧪ialhabbal_VLLM with 17 inputs and 1 output.
ialhabbal VLLM Advanced
- image
- video
- RESPONSE
◄model_nameQwen3-VL-2B-Instruct►
◄quantizationNone (FP16)►
◄attention_modeauto►
◄use_torch_compilefalse►
◄deviceauto►
◄preset_prompt🖼️ Detailed Description►
◄custom_prompt►
◄max_tokens512►
◄temperature0.60►
◄top_p0.90►
◄num_beams1►
◄repetition_penalty1.20►
◄frame_count16►
◄keep_model_loadedtrue►
◄seed1►
Category🧪ialhabbal_VLLM
Inputs (17)
| Name | Type | Default | Description |
|---|---|---|---|
| model_name | COMBO | Qwen3-VL-2B-Instruct | Pick the Qwen-VL checkpoint. First run downloads weights into models/LLM/Qwen-VL, so leave disk space. |
| quantization | COMBO | None (FP16) | Precision vs VRAM. FP16 gives the best quality if memory allows; 8-bit suits 8–16 GB GPUs; 4-bit fits 6 GB or lower but is slower. |
| attention_mode | COMBO | auto | auto tries flash-attn v2 when installed and falls back to SDPA. Only override when debugging attention backends. |
| use_torch_compile | BOOLEAN | false | Enable torch.compile('reduce-overhead') on supported CUDA/Torch 2.1+ builds for extra throughput after the first compile. |
| device | COMBO | auto | Choose where to run the model: auto, cpu, mps, or cuda:x for multi-GPU systems. |
| preset_prompt | COMBO | 🖼️ Detailed Description | Built-in instruction describing how Qwen-VL should analyze the media input. |
| custom_prompt | STRING | Optional override—when filled it completely replaces the preset template. | |
| max_tokens | INT | 51264–4096 | Maximum number of new tokens to decode. Larger values yield longer answers but consume more time and memory. |
| temperature | FLOAT | 0.600.1–1 | Sampling randomness when num_beams == 1. 0.2–0.4 is focused, 0.7+ is creative. |
| top_p | FLOAT | 0.900–1 | Nucleus sampling cutoff when num_beams == 1. Lower values keep only top tokens; 0.9–0.95 allows more variety. |
| num_beams | INT | 11–8 | Beam-search width. Values >1 disable temperature/top_p and trade speed for more stable answers. |
| repetition_penalty | FLOAT | 1.200.5–2 | Values >1 (e.g., 1.1–1.3) penalize repeated phrases; 1.0 leaves logits untouched. |
| frame_count | INT | 161–64 | Number of frames extracted from video inputs before prompting Qwen-VL. More frames provide context but cost time. |
| keep_model_loaded | BOOLEAN | true | Keeps the model resident in VRAM/RAM after the run so the next prompt skips loading. |
| seed | INT | 11–4294967295 | Seed controlling sampling and frame picking; reuse it to reproduce results. |
| imageopt | IMAGE | — | |
| videoopt | IMAGE | — |
Outputs (1)
| Name | Type | Description |
|---|---|---|
| RESPONSE | STRING | — |