Local LLM

Answers a prompt with a local language model: a GGUF file from models/LLM (or from LM Studio, the Hugging Face cache, llama.cpp or GPT4All) run by llama.cpp just for this call, or a model of a running app (LM Studio, Ollama, llama.cpp, vLLM, Jan, KoboldCpp, ...). Works anywhere a STRING is needed; Plenio's Write Song uses it when its writer model is one of these. Press R to refresh the list.

By jplenio·Created 17 days ago·Updated about 19 hours ago· 40

Local LLM

text
thinking

◄prompt►

◄model▾►

◄seed0►

◄max_tokens6144►

◄temperature0.80►

◄thinkingfalse►

◄context12288►

◄keep_loadedfalse►

◄system_prompt►

◄reuse_answerstrue►

◄schema—►

CategoryPlenio/Text

Inputs (11)

Name	Type	Default	Description
prompt	STRING		The request (connect Compose Writing Prompt, or type).
model	COMBO		'models/LLM · file.gguf' and other '... files' / 'HF cache' entries are GGUF files, run by llama.cpp for the call; 'LM Studio · ...', 'Ollama · ...' are models of a running app.
seed	INT	00–18446744073709550000	Same seed, same prompt, same settings: the same answer - taken from Plenio's answer cache without asking the model again (see reuse_answers). Change it for a new answer.
max_tokens	INT	614416–131072	Longest answer in tokens; a cut-off answer stops the run with a message.
temperature	FLOAT	0.800–2	0 = always the most likely words; higher = more varied.
thinking	BOOLEAN	false	Let a reasoning model think first (Qwen, Gemma, DeepSeek ...); the thoughts come out separately. Slower; needs more max tokens.
context	INT	122881024–262144	Context length in tokens (prompt + answer) for GGUF files and Ollama. Other apps use the length set in the app.
keep_loaded	BOOLEAN	false	Off: the model's memory is freed after the answer, so ComfyUI's models have the GPU. On: the model stays loaded for the next run (faster, needs the memory for both).
system_prompt	STRING		Optional standing instructions sent before the prompt.
reuse_answersopt	BOOLEAN	true	On: a request answered before (same model file or app model, prompt, seed and settings) is answered from Plenio's answer cache (user/plenio/cache/llm) - no call, also after a restart or when an earlier node ran again. Off: always ask the model.
schemaopt	STRING		Optional JSON schema of the answer (Compose Arrangement provides one): GGUF files, LM Studio, Ollama and vLLM are held to it while they write, so the answer always has that format. Apps without this feature answer freely.

Outputs (2)

Name	Type	Description
text	STRING	The answer (thoughts removed).
thinking	STRING	The model's thoughts, if it shared any.