APNext Phi Model Inference

PhiModelInference (APNext Phi Model Inference) is the node that actually does something with Microsoft's Phi-3.5-vision. You feed it a loaded pipeline plus an image and a question, and it returns text - a caption, a description, a prompt written from the picture. It's the straightforward, no-frills runner in this pack's Phi trio; if you just want "look at this image and tell me about it" from a local model, this is it.

It pairs with PhiModelLoader, which handles the heavy lifting of loading the model. This node is deliberately light: it borrows the already-loaded pipeline and runs inference, so you can queue it repeatedly without re-loading gigabytes each time.

How it works

You give it a phi_pipeline (from PhiModelLoader), your input_images, and a user_prompt, and it runs Phi locally on your GPU to generate an answer. Everything stays on your machine - no key, no API.

The inputs:

• phi_pipeline (PHI_MODEL_PIPELINE) - wire this from a PhiModelLoader node. This is the required link; without it the node has no model.
• input_images (IMAGE) - what Phi looks at.
• user_prompt - your instruction. "Describe this image as a detailed prompt for an image generator" works well; a blank prompt gives you a generic description.
• generation_temperature (default 0.7) - creativity vs. determinism. Lower (0.2-0.4) for tight, factual captions; higher for looser, more varied prose.
• max_output_tokens (default 2048) - caption length ceiling. Drop it if you want short captions and don't want the model to ramble.

One output: generated_text (STRING), which you wire into your positive text encode or wherever you need the description.

If you want the fancier version - dynamic LoRA-trigger substitution, image blending, a separate CLIP-L output - that's the sibling node, PhiCustomModelInference. This one keeps it simple.

Installing it

ComfyUI Manager: search comfyui_dagthomas, install, restart. Or clone and pip install -r requirements.txt. Relies on transformers. No API key; the Phi weights download via the loader on first use.

Common issues

"Missing phi_pipeline" / red input. You didn't connect a PhiModelLoader, or it errored on load. This node can't run standalone - it needs the loader's pipeline output wired into it.

Captions are too long or too rambly. Lower max_output_tokens and drop generation_temperature toward 0.3. Long prompts also hurt you on LLM-encoded image models, so tight is usually better anyway.

Out of memory. Phi is sharing the GPU with your image checkpoint. If you OOM, that's a memory-management problem at the loader/generation boundary - consider loading Phi, running it, then loading your checkpoint, rather than holding both at once.

Output is generic. A vague user_prompt gets a vague answer. Tell it exactly what you want described - subject, lighting, lens, mood - and it'll write a far more usable prompt.