Comfy.ICU
ExploreDocsPricing
Support

ComfyUI Node: Load LLM Model Advanced

Authored by daniel-lewis-ab

Created

Updated

42 stars

Category

LLM

Inputs

Model
    n_gpu_layers INT
    split_mode
    • LLAMA_SPLIT_NONE
    • LLAMA_SPLIT_LAYER
    • LLAMA_SPLIT_ROW
    main_gpu INT
    tensor_split FLOAT
    vocab_only BOOLEAN
    use_mmap BOOLEAN
    use_mlock BOOLEAN
    seed INT
    n_ctx INT
    n_batch INT
    n_threads INT
    n_threads_batch INT
    rope_scaling_type
    • LLAMA_ROPE_SCALING_UNSPECIFIED
    • LLAMA_ROPE_SCALING_NONE
    • LLAMA_ROPE_SCALING_LINEAR
    • LLAMA_ROPE_SCALING_YARN
    rope_freq_base FLOAT
    rope_freq_scale FLOAT
    yarn_ext_factor FLOAT
    yarn_attn_factor FLOAT
    yarn_beta_fast FLOAT
    yarn_beta_slow FLOAT
    yarn_orig_ctx INT
    mul_mat_q INT
    logits_all BOOLEAN
    embedding BOOLEAN
    offload_kqv BOOLEAN
    last_n_tokens_size INT
    lora_base STRING
    lora_scale FLOAT
    lora_path STRING
    numa BOOLEAN
    chat_format STRING
    verbose BOOLEAN

    Outputs

    LLM

    Extension: ComfyUI-Llama

    This is a set of nodes to interact with llama-cpp-python

    Authored by daniel-lewis-ab

    View Nodes

    Run ComfyUI workflows in the Cloud!

    No downloads or installs are required. Pay only for active GPU usage, not idle time. No complex setups and dependency issues

    Learn more