FastVLM 7B (Apple)

If you want to ask an image a question and get a plain-text answer without leaving ComfyUI, this is one of the simplest doors in. FastVLM 7B (Apple) - a single node from the Ltamann/ComfyUI-FastVLM-7B pack - feeds a ComfyUI image plus your instruction into Apple's FastVLM-7B vision-language model and returns the generated text as a STRING. The name is a hint at the pitch: Apple markets FastVLM as a fast VLM, optimized for cheap deployment rather than for topping benchmark leaderboards.

So set expectations now. The author of this pack - TBG, the same person behind the TBG-ETUR upscaler - was upfront about it when he shipped this on r/comfyui: it's "a fast alternative to Florence" for quick captions, and when he needs quality he switches to Janus or Qwen. The community's take landed about there too ("Apple's models were kinda butt," in the immortal words of one commenter). Reach for this node when you want speed and simplicity - a local, free, keyless captioner for auto-tagging or a lightweight image-QA in a workflow. Reach for something like Janus or Qwen if the caption quality is going into a training dataset.

How it works

Under the hood it's a thin wrapper over Hugging Face transformers. The node converts your ComfyUI image tensor (B,H,W,C floats) to a PIL image, builds a chat prompt from your instruction, slots the image in with the model's special <image> token, and runs model.generate(). Nothing exotic - no API, no key, all inference is on your machine.

The interesting bit is the model management. On first run the node auto-downloads apple/FastVLM-7B from Hugging Face into ComfyUI/models/LLM/FastVLM, and it loads in fp16 on CUDA or fp32 on CPU via device_map="auto". That means no manual model download step in the README - but it also means your first run is a multi-gigabyte download that can look like a frozen ComfyUI.

The inputs that matter

The node has three required inputs, and two of them are self-explanatory:

• image - any ComfyUI IMAGE; feed it from Load Image.
• instruction - your question or caption request, multiline text. "Describe this image in one sentence" or "What color is the car?" both work.
• max_new_tokens - default 128, up to 1024. This caps the length of the answer. Bump it to 512+ if you're getting truncated captions; leave it low for short Q&A so it doesn't ramble.

The single output is a STRING - the model's answer. Wire it into a Show Text node to see it, Save Text to write it to disk, or loop it straight back into a prompt string for a self-captioning pipeline.

Installing it

Easiest path: ComfyUI Manager → install custom nodes → search "FastVLM-7B". Or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/Ltamann/ComfyUI-FastVLM-7B
# restart ComfyUI

The pack ships no requirements.txt - it leans on ComfyUI's bundled torch plus transformers. If you get an ImportError: No module named transformers at load time, that's the missing piece:

pip install transformers

Then just add the node and run it once. The model (roughly 14GB in fp16, more like 28GB in fp32 on CPU) downloads itself into models/LLM/FastVLM - watch the console for the [FastVLM7BNode] Loading ... line and be patient.

Where people get burned

• The first run is not hung, it's downloading. A 7B model over your connection takes a while, and there's no progress bar in the UI.
• It only looks at one image. tensor_to_pil grabs batch index 0, so a batch of images in gets you a caption for the first one only. Batch-captioning isn't this node's job.
• VRAM. fp16 7B is ~14GB of weights plus activation overhead. Below that, device_map="auto" will offload to CPU and "fast" stops being the operative word. On a CPU-only box this is usable but slow - it's an LLM, after all.
• trust_remote_code=True. The node loads Apple's model with remote code execution, meaning the trust boundary is really Apple's model card. It's a legit, widely-used HF model, but in the post-LLMVISION era it's worth knowing what you're letting run. This node itself is about as thin as a node gets - a handful of imports, no sneaky network calls.

Want quality captions for LoRA training? Go with a better VLM. Want a free, local, no-key "ask your image anything" node that just works? This is your fast, slightly dumb friend.