ComfyUI Node: Llama.cpp Server Client

Authored by fidecastro

Created

Updated

9 stars

Run ComfyUI workflows without the setup

No installs, no CUDA version roulette, no GPU sitting idle on your bill. Bring a workflow and run it in the browser.

Category

AI/LlamaCpp

Inputs

server_url STRING
endpoint
  • completion
  • chat_completions
  • embeddings
  • tokenize
  • detokenize
  • apply_template
  • infill
  • reranking
prompt STRING
api_key STRING
timeout INT
n_predict INT
temperature FLOAT
top_k INT
top_p FLOAT
min_p FLOAT
seed INT
dynatemp_range FLOAT
dynatemp_exponent FLOAT
xtc_probability FLOAT
xtc_threshold FLOAT
repeat_penalty FLOAT
repeat_last_n INT
presence_penalty FLOAT
frequency_penalty FLOAT
dry_multiplier FLOAT
dry_base FLOAT
dry_allowed_length INT
dry_penalty_last_n INT
dry_sequence_breakers STRING
mirostat INT
mirostat_tau FLOAT
mirostat_eta FLOAT
typical_p FLOAT
n_keep INT
stop_sequences STRING
ignore_eos BOOLEAN
stream BOOLEAN
n_probs INT
min_keep INT
post_sampling_probs BOOLEAN
return_tokens BOOLEAN
timings_per_token BOOLEAN
grammar STRING
json_schema STRING
logit_bias STRING
cache_prompt BOOLEAN
id_slot INT
samplers STRING
t_max_predict_ms INT
messages STRING
system_message STRING
user_message STRING
assistant_message STRING
max_tokens INT
model STRING
tools STRING
tool_choice STRING
response_format STRING
input_text STRING
encoding_format
  • float
  • base64
embd_normalize INT
content STRING
tokens STRING
add_special BOOLEAN
parse_special BOOLEAN
with_pieces BOOLEAN
input_prefix STRING
input_suffix STRING
input_extra STRING
query STRING
documents STRING
top_n INT
lora STRING
response_fields STRING
image_data STRING

Outputs

STRING

STRING

STRING

INT

Extension: comfyui-llamacpp-client

A comprehensive ComfyUI custom node that provides complete client functionality for llama-server from a/llama.cpp. This node acts as a bridge between ComfyUI workflows and llama-server instances, supporting every single parameter and endpoint that llama-server offers.

Authored by fidecastro

View Nodes

Looking for a different node?

Also provided by 1 other extension

Run ComfyUI workflows without the setup

No installs, no CUDA version roulette, no GPU sitting idle on your bill. Bring a workflow and run it in the browser.

Learn more