ComfyUI Node
Load LLM Model Advanced
A ComfyUI node in LLM with 32 inputs and 1 output.
Load LLM Model Advanced
- LLM
◄Model▾►
◄n_gpu_layers0►
◄split_modeLLAMA_SPLIT_LAYER►
◄main_gpu0►
◄tensor_split0.00►
◄vocab_onlyfalse►
◄use_mmaptrue►
◄use_mlockfalse►
◄seed-1►
◄n_ctx512►
◄n_batch512►
◄n_threads—►
◄n_threads_batch—►
◄rope_scaling_typeLLAMA_ROPE_SCALING_UNSPECIFIED►
◄rope_freq_base0.00►
◄rope_freq_scale0.00►
◄yarn_ext_factor-1.00►
◄yarn_attn_factor1.00►
◄yarn_beta_fast32.00►
◄yarn_beta_slow1.00►
◄yarn_orig_ctx0►
◄mul_mat_q0►
◄logits_allfalse►
◄embeddingfalse►
◄offload_kqvfalse►
◄last_n_tokens_size64►
◄lora_base—►
◄lora_scale0.00►
◄lora_path—►
◄numafalse►
◄chat_formatllama-2►
◄verbosetrue►
CategoryLLM
Inputs (32)
| Name | Type | Default | Description |
|---|---|---|---|
| Model | COMBO | 0 options: | |
| n_gpu_layersopt | INT | 0 | — |
| split_modeopt | COMBO | LLAMA_SPLIT_LAYER | 3 options: LLAMA_SPLIT_NONE, LLAMA_SPLIT_LAYER, LLAMA_SPLIT_ROW |
| main_gpuopt | INT | 0 | — |
| tensor_splitopt | FLOAT | 0.000–1 | — |
| vocab_onlyopt | BOOLEAN | false | — |
| use_mmapopt | BOOLEAN | true | — |
| use_mlockopt | BOOLEAN | false | — |
| seedopt | INT | -1 | — |
| n_ctxopt | INT | 512 | — |
| n_batchopt | INT | 512 | — |
| n_threadsopt | INT | — | |
| n_threads_batchopt | INT | — | |
| rope_scaling_typeopt | COMBO | LLAMA_ROPE_SCALING_UNSPECIFIED | 4 options: LLAMA_ROPE_SCALING_UNSPECIFIED, LLAMA_ROPE_SCALING_NONE, LLAMA_ROPE_SCALING_LINEAR, LLAMA_ROPE_SCALING_YARN |
| rope_freq_baseopt | FLOAT | 0.000–1 | — |
| rope_freq_scaleopt | FLOAT | 0.000–1 | — |
| yarn_ext_factoropt | FLOAT | -1.00 | — |
| yarn_attn_factoropt | FLOAT | 1.000–1 | — |
| yarn_beta_fastopt | FLOAT | 32.00 | — |
| yarn_beta_slowopt | FLOAT | 1.000–1 | — |
| yarn_orig_ctxopt | INT | 0 | — |
| mul_mat_qopt | INT | 0 | — |
| logits_allopt | BOOLEAN | false | — |
| embeddingopt | BOOLEAN | false | — |
| offload_kqvopt | BOOLEAN | false | — |
| last_n_tokens_sizeopt | INT | 64 | — |
| lora_baseopt | STRING | — | |
| lora_scaleopt | FLOAT | 0.000–1 | — |
| lora_pathopt | STRING | — | |
| numaopt | BOOLEAN | false | — |
| chat_formatopt | STRING | llama-2 | — |
| verboseopt | BOOLEAN | true | — |
Outputs (1)
| Name | Type | Description |
|---|---|---|
| LLM | LLM | — |