VLM Prompt HF Advanced (PyTorch)

"VLM Prompt HF Advanced (PyTorch)" is the full-panel version of the HF prompt node - the same transformers backend as "VLM Prompt HF (PyTorch)", with quantization, device, frame sampling, and the sampler dials exposed. It's what you move to when the simple HF node works but you need it to fit your VRAM or behave differently on a loop.

It's a leaner advanced panel than the GGUF Advanced node, and that's by design: the HF backend manages the context window and layer placement itself, so there's no ctx, gpu_layers, or pool_size to fiddle with. You get the settings that actually matter for this backend and not much noise.

What the extra knobs actually do

• quantization - None (FP16/BF16) (default), 8-bit, or 4-bit via BitsAndBytes. This is the knob that decides whether a 9.5 GB fp16 model becomes a 2.5 GB 4-bit model. Requires a working bitsandbytes; if it's missing, the node falls back to fp16 and logs it rather than dying.
• device - auto or cpu. auto lets transformers place things across your GPU; only touch it if you're deliberately forcing CPU.
• frame_count - how many evenly sampled frames from the video input get analyzed (1–64, default 16). More frames, better motion description, slower run.
• temperature / top_p / repetition_penalty - the sampler set (defaults 0.6 / 0.9 / 1.2). Lower temperature for consistent describing; raise the repetition penalty if output loops.
• model_name - Qwen3.5-2B-abliterated (default) or Qwen3.5-4B-Claude-HighIQ-THINKING, from models_hf.json.

Everything else is inherited: the same 27 presets, system_prompt, custom_prompt (overrides the preset when filled), max_tokens, keep_model_loaded, seed, and optional image/video inputs. The output is the single prompt STRING for a Text Encode or video model.

Install

Same as the pack:

cd ComfyUI/custom_nodes
git clone https://github.com/mexxmillion/ComfyUI-VLMPrompt.git
pip install -r ComfyUI-VLMPrompt/requirements.txt

Restart ComfyUI. Snapshots land in ComfyUI/models/LLM/HF on first use. Add a platform-appropriate bitsandbytes if you want the 4/8-bit options to actually engage.

Common issues

• 4-bit option does nothing - the classic BitsAndBytes fallback. Check the console log for "falling back to FP16" before troubleshooting anything else.
• Long first run - downloading a full-precision snapshot is slow; the quantization options are the fix for small cards.
• Video prompt too generic - raise frame_count rather than the temperature.