APNext Phi Model Loader

PhiModelLoader (APNext Phi Model Loader) is the setup node for this pack's Phi vision pipeline. On its own it does nothing visible - it loads Microsoft's Phi-3.5-vision-instruct model onto your GPU and hands out a phi_pipeline object that the two Phi inference nodes consume. It's the classic ComfyUI loader/runner split: load once, run many times, so you're not re-loading a multi-gigabyte model on every queue.

Phi-3.5-vision is Microsoft's small multimodal model - a lightweight, fully local captioner and image-QA model in the same lane as MiniCPM-V and LLaVA. You'd use the Phi nodes when you want a compact local VLM that fits on a modest GPU and doesn't phone home.

How it works

This node instantiates the model and returns a phi_pipeline (type PHI_MODEL_PIPELINE). You wire that single output into PhiModelInference or PhiCustomModelInference, which do the actual talking. The model downloads from HuggingFace on first use.

Only three inputs, and two of them are worth understanding:

• model_version - currently just Phi-3.5-vision-instruct. One option; nothing to decide.
• image_crops (4 or 16) - how many crops Phi slices each image into. 4 is the fast, low-VRAM default; 16 gives the model more detail to work with (better for dense or high-resolution images) at the cost of more memory and time. Start at 4, bump to 16 only if captions are missing fine detail.
• attention_mechanism (flash_attention_2 / sdpa / eager, default eager) - how attention is computed. flash_attention_2 is fastest but only if you have the flash-attn package built and installed, which is a notorious pain. sdpa is a good middle ground on recent PyTorch. eager is the slow-but-always-works default, which is why it's the default.

Installing it

ComfyUI Manager: search comfyui_dagthomas, install, restart. Or clone and pip install -r requirements.txt. This node relies on transformers from that requirements file. No API key - the weights pull from HuggingFace on first load, so budget time and disk for that.

Common issues

flash_attention_2 crashes on load. This is the classic. flash-attn isn't installed, or the prebuilt wheel doesn't match your CUDA/PyTorch, and it fails at model init. Switch attention_mechanism to sdpa (fast, no extra install) or eager (bulletproof). Don't fight flash-attn unless you specifically need the speed and know your build works.

Out of memory at load, or when the image model is also resident. image_crops: 16 roughly quadruples the visual tokens versus 4. Drop to 4, and remember Phi shares the GPU with your checkpoint - the inference nodes are where you manage that.

Nothing happens. That's expected. The loader only produces a pipeline object; you won't see output until you connect it to a PhiModelInference node and give that an image and a prompt.

First run stalls. Model downloading from HuggingFace. One-time, then cached.