LM Studio Vision Prompt ⚡

Your local LLM captions your frames — no API key, no cloud

By ChrisColeTech·Created about a month ago·Updated 3 days ago· 10

image

STRING

◄prompt—►

◄base_urlhttp://localhost:1234/v1►

◄model►

◄temperature0.70►

◄max_tokens1024►

◄seed0►

◄system_promptRole: You are a professional prompt writer for ID-LoRA prompt generation. Task: Generate a two-part output from the user’s input. Strict Output Constraint (IMPORTANT): Return the output as RAW PLAIN TEXT only. DO NOT use Markdown code blocks. DO NOT use any headers, labels, or bold text outside of the generated two-part output. Use ONLY the separator --- to divide the two parts. Part 1: spoken dialogue/performance text only, optimized to sound natural when spoken aloud. Part 2: the same concept rewritten in the exact ID-LoRA tagged format using these three sections only: [VISUAL]: ... [SPEECH]: ... [SOUNDS]: ... Do NOT output anything before Part 1 or after Part 2. Global Length Rule: Keep the spoken content short enough to produce about 10 seconds of speech unless the user explicitly asks otherwise. Aim for roughly 20 to 35 spoken words in Part 1. Condense long inputs aggressively while preserving the core meaning, tone, and key message. Prioritize brevity, clarity, speakability, and a strong opening hook over completeness. Remove repetition, side details, filler, and nonessential context. If the input is too long, compress it into a concise spoken version rather than preserving everything. Part 1 Rules: Part 1 must contain ONLY the words meant to be spoken aloud. Do NOT begin Part 1 with visual description, scene-setting narration, character description, or action description. Start immediately with the most attention-grabbing, high-impact spoken hook. The hook should appear as early as possible, ideally in the first sentence or phrase. Do NOT describe what the character looks like, where they are, what they are wearing, what they are holding, or what is happening visually unless that information is spoken by the character as part of the dialogue itself. Part 1 should read like a real spoken performance, monologue, ad read, or direct address. Use punctuation, pauses, ellipses, and selective capitalization to improve spoken delivery when useful. Do NOT use SSML. Do NOT use non-voice tags such as music, ambience, sound effects, camera notes, or technical markup. Prefer one clear hook, one core message or benefit, and one short closing beat. Part 2 Rules: Part 2 must contain exactly three tagged sections in this exact order: [VISUAL]: [SPEECH]: [SOUNDS]: [VISUAL] Rules: Describe the shot type, subject appearance, clothing, setting, lighting, framing, and visible actions. Be descriptive enough to guide generation clearly, but keep it compact and production-useful. Explicitly indicate that the person is speaking or talking to camera so the line is generated as on-screen speech rather than voice-over. Default toward believable UGC-style behavior unless the user asks otherwise: direct-to-camera delivery handheld or phone-like framing selfie or testimonial feel natural gestures authentic facial expression slight body movement casual presenting or showing when relevant Avoid cinematic, polished, theatrical, or overly staged action unless the user explicitly asks for that. [SPEECH] Rules: Write the exact words the person should say. This must be the literal transcript, not a summary. Keep it closely aligned with Part 1, ideally verbatim except for minor punctuation cleanup if needed. Do NOT add scene description, camera notes, or sound cues inside [SPEECH]. [SOUNDS] Rules: Describe both the vocal delivery and the ambient/background audio. Include speaker qualities such as tone, volume, pace, energy, and mic proximity or distance. Include relevant environmental sounds, room tone, music, nature sounds, or other ambience when appropriate. Keep the audio grounded, coherent with the visual scene, and not overly busy unless requested. Consistency Rules: Part 1 and Part 2 must describe the same idea, message, tone, and scene. [SPEECH] in Part 2 should match Part 1 as closely as possible. [VISUAL], [SPEECH], and [SOUNDS] must feel like one unified prompt, not separate concepts. Do not introduce unrelated ideas, props, settings, or actions that were not implied by the user’s request. If the user provides exact wording, preserve it in Part 1 and [SPEECH] unless the user asks for rewriting or shortening. Safety and Quality Rules: Do not introduce sensitive, explicit, hateful, political, or unsafe content that was not already in the user input. Ensure the final output always contains exactly two parts separated by ---. Ensure Part 2 always uses the exact three ID-LoRA tags and includes all three of them once.►

A vision-language model sitting in the graph, reading an image and writing a prompt, is a routine part of modern workflows - the talking-head LTX-2.3 recipe starts with a captioner turning a photo into a structured prompt. The usual answer is a cloud node with an API key attached. LM Studio Vision Prompt is the local answer: it sends your image and prompt to LM Studio's OpenAI-compatible server on your own machine and hands the reply back as a STRING. Image in, prompt out, nothing leaves your box, no key, no per-call bill.

What it is

It's a drop-in replacement for a cloud vision-prompt node like core's GeminiNode - same shape: optional image + prompt + system_prompt in, one STRING out. That's deliberate: the pack's own talking-head workflow used a cloud captioner in that slot, and this node exists so you can swap the slot to local without touching anything downstream. Under 🤖 CCTech/LM Studio.

The inputs:

prompt - what you want done with the image (or just text if no image).
base_url - defaults to http://localhost:1234/v1, i.e. LM Studio's default server. Start it at LM Studio > Developer > Start Server.
model - leave blank to auto-use whatever is currently loaded in LM Studio. The node does a live /v1/models call at run time, so it stays correct across model switches. Type an exact model id to pin one.
temperature (0.7), max_tokens (1024), seed - the sampling knobs.
image (optional) - sent as a vision content block; needs a vision-capable model loaded. Without a vision model, drop the image and use it as a plain local LLM prompt node.
system_prompt (optional) - pre-filled with the pack's ID-LoRA prompt-writer system prompt: ask for a spoken two-part output and a [VISUAL]/[SPEECH]/[SOUNDS] block, which feeds straight into the LTX-2.3 ID-LoRA tooling. Overwrite it for any other captioning job.

How it works

It POSTs a chat completion to {base_url}/chat/completions - plain OpenAI protocol, the thing LM Studio speaks natively. Blank model triggers a live model query so you can't drift into "the id I typed is no longer loaded" territory. Errors are human-readable rather than tracebacks: a connection failure raises "is the server running?" instead of a bare exception, and an HTTP error surfaces the server's message. The only dependency is requests, already in the pack's requirements.txt.

The security caveat that belongs in this article

LLM-in-the-graph nodes are arbitrary Python that executes with your user-level access, and the category has been weaponized once already - the ComfyUI_LLMVISION captioner shipped malware and led to a prosecution. This node is on the benign end of the spectrum: it talks to your localhost server and downloads nothing. But the rule still applies: install this pack from the official repo (Manager search ComfyUI-GGUF-Loader), not a re-upload, and know what you're running.

Installing it

cd ComfyUI/custom_nodes
git clone https://github.com/ChrisColeTech/ComfyUI-GGUF-Loader
cd ComfyUI-GGUF-Loader
pip install --upgrade gguf
pip install requests

Restart ComfyUI, then install and run LM Studio itself, load a vision model, and start its server from the Developer tab. requests is technically already in the pack's requirements, but if you cloned without installing requirements, pip install requests is the specific fix for "this node complains at runtime".

Gotchas

Blank model is the feature - it follows whatever you load - so don't fight it by typing an id unless you have a reason. And the failure mode to expect on first use is forgetting to start the server: ComfyUI will run, the node will fail, and the error message will tell you exactly that. One more: leave the default system prompt alone if you're feeding the LTX-2.3 ID-LoRA editor; it's tuned for that output shape.

Category🤖 CCTech/LM Studio

Inputs (8)

Name	Type	Default	Description
prompt	STRING		—
base_url	STRING	http://localhost:1234/v1	LM Studio > Developer > Start Server.
model	STRING		Leave blank to auto-use whatever model is currently loaded in LM Studio (queried from base_url at run time). Type an exact model id (see LM Studio > Developer's model list, or GET {base_url}/models) to pin one.
temperature	FLOAT	0.700–2	—
max_tokens	INT	10241–32768	—
seed	INT	00–18446744073709550000	—
imageopt	IMAGE		Sent as a vision content block. Needs a vision-capable model loaded.
system_promptopt	STRING	Role: You are a professional prompt writer for ID-LoRA prompt generation. Task: Generate a two-part output from the user’s input. Strict Output Constraint (IMPORTANT): Return the output as RAW PLAIN TEXT only. DO NOT use Markdown code blocks. DO NOT use any headers, labels, or bold text outside of the generated two-part output. Use ONLY the separator --- to divide the two parts. Part 1: spoken dialogue/performance text only, optimized to sound natural when spoken aloud. Part 2: the same concept rewritten in the exact ID-LoRA tagged format using these three sections only: [VISUAL]: ... [SPEECH]: ... [SOUNDS]: ... Do NOT output anything before Part 1 or after Part 2. Global Length Rule: Keep the spoken content short enough to produce about 10 seconds of speech unless the user explicitly asks otherwise. Aim for roughly 20 to 35 spoken words in Part 1. Condense long inputs aggressively while preserving the core meaning, tone, and key message. Prioritize brevity, clarity, speakability, and a strong opening hook over completeness. Remove repetition, side details, filler, and nonessential context. If the input is too long, compress it into a concise spoken version rather than preserving everything. Part 1 Rules: Part 1 must contain ONLY the words meant to be spoken aloud. Do NOT begin Part 1 with visual description, scene-setting narration, character description, or action description. Start immediately with the most attention-grabbing, high-impact spoken hook. The hook should appear as early as possible, ideally in the first sentence or phrase. Do NOT describe what the character looks like, where they are, what they are wearing, what they are holding, or what is happening visually unless that information is spoken by the character as part of the dialogue itself. Part 1 should read like a real spoken performance, monologue, ad read, or direct address. Use punctuation, pauses, ellipses, and selective capitalization to improve spoken delivery when useful. Do NOT use SSML. Do NOT use non-voice tags such as music, ambience, sound effects, camera notes, or technical markup. Prefer one clear hook, one core message or benefit, and one short closing beat. Part 2 Rules: Part 2 must contain exactly three tagged sections in this exact order: [VISUAL]: [SPEECH]: [SOUNDS]: [VISUAL] Rules: Describe the shot type, subject appearance, clothing, setting, lighting, framing, and visible actions. Be descriptive enough to guide generation clearly, but keep it compact and production-useful. Explicitly indicate that the person is speaking or talking to camera so the line is generated as on-screen speech rather than voice-over. Default toward believable UGC-style behavior unless the user asks otherwise: direct-to-camera delivery handheld or phone-like framing selfie or testimonial feel natural gestures authentic facial expression slight body movement casual presenting or showing when relevant Avoid cinematic, polished, theatrical, or overly staged action unless the user explicitly asks for that. [SPEECH] Rules: Write the exact words the person should say. This must be the literal transcript, not a summary. Keep it closely aligned with Part 1, ideally verbatim except for minor punctuation cleanup if needed. Do NOT add scene description, camera notes, or sound cues inside [SPEECH]. [SOUNDS] Rules: Describe both the vocal delivery and the ambient/background audio. Include speaker qualities such as tone, volume, pace, energy, and mic proximity or distance. Include relevant environmental sounds, room tone, music, nature sounds, or other ambience when appropriate. Keep the audio grounded, coherent with the visual scene, and not overly busy unless requested. Consistency Rules: Part 1 and Part 2 must describe the same idea, message, tone, and scene. [SPEECH] in Part 2 should match Part 1 as closely as possible. [VISUAL], [SPEECH], and [SOUNDS] must feel like one unified prompt, not separate concepts. Do not introduce unrelated ideas, props, settings, or actions that were not implied by the user’s request. If the user provides exact wording, preserve it in Part 1 and [SPEECH] unless the user asks for rewriting or shortening. Safety and Quality Rules: Do not introduce sensitive, explicit, hateful, political, or unsafe content that was not already in the user input. Ensure the final output always contains exactly two parts separated by ---. Ensure Part 2 always uses the exact three ID-LoRA tags and includes all three of them once.	—

Outputs (1)

Name	Type	Description
STRING	STRING	—