ComfyUI Node
SID LLM Local
Local Vision Models [No GPU] - No API needed
SID LLM Local
- LLM_MODEL
◄modelQwen3-VL-2B-Instruct►
◄quantizationAuto (Detect Best)►
◄deviceauto►
◄temperature0.3►
◄max_tokens_presetMedium (1024)►
◄custom_max_tokens1024►
◄keep_model_loadedtrue►
◄attention_modeauto►
◄repetition_penalty1.2►
◄top_p0.90►
◄use_torch_compilefalse►
CategorySID Photography Toolkit/LLM Providers
Inputs (11)
| Name | Type | Default | Description |
|---|---|---|---|
| model | COMBO | Qwen3-VL-2B-Instruct | Select model: Name | Max Tokens | VRAM (4-bit) |
| quantization | COMBO | Auto (Detect Best) | Auto: detects best based on VRAM, 4-bit: lowest VRAM, FP16: best quality |
| device | COMBO | auto | Device to run on (auto recommended) |
| temperature | FLOAT | 0.30–1 | Creativity (0=deterministic, 0.3=balanced, 1=creative) |
| max_tokens_preset | COMBO | Medium (1024) | Output length: Low=512, Medium=1024, High=2048, Very High=Model Max |
| custom_max_tokens | INT | 1024128–4096 | Custom max tokens (only used when preset is 'Custom') |
| keep_model_loaded | BOOLEAN | true | Keep model in VRAM between runs (faster repeat inference) |
| attention_mode | COMBO | auto | Attention implementation: auto (recommended), flash_attention_2 (Ampere+), sdpa, eager |
| repetition_penalty | FLOAT | 1.20.8–2 | Penalize repeated tokens (1.0=off, 1.2=recommended, 2.0=strong) |
| top_p | FLOAT | 0.900.1–1 | Nucleus sampling (0.9=recommended, lower=more focused) |
| use_torch_compile | BOOLEAN | false | Enable torch.compile for faster inference (CUDA + Torch 2.1+ only, first run slower) |
Outputs (1)
| Name | Type | Description |
|---|---|---|
| LLM_MODEL | LLM_MODEL | LLM configuration to connect to prompt generator |