ComfyUI Extension: comfyui-sg-llama-cpp

Authored by sebagallo

Created

Updated

1 stars

llama-cpp-python wrapper, with support for vision models. It allows the user to generate text responses from prompts using llama.cpp.

Custom Nodes (0)

    README

    comfyui-sg-llama-cpp

    ComfyUI custom node that acts as a llama-cpp-python wrapper, with support for vision models. It allows the user to generate text responses from prompts using llama.cpp.

    Screenshot

    Features

    • Load and use GGUF models (including vision models)
    • Generate text prompts using llama.cpp
    • Support for multi-modal inputs (images)
    • Memory management options
    • Integration with ComfyUI workflows

    Installation

    1. Install the required dependency/wheel from:

      https://github.com/JamePeng/llama-cpp-python/releases

    2. Clone this repository into your ComfyUI custom nodes directory:

      cd ComfyUI/custom_nodes
git clone https://github.com/sebagallo/comfyui-sg-llama-cpp

    3. Restart ComfyUI.

    Usage

    This custom node provides four main components:

    LlamaCPPModelLoader

    • Loads GGUF model files from the text_encoders folder
    • Supports various chat formats
    • Optional multi-modal projector support for vision models

    LlamaCPPOptions

    • Configures model parameters like:
      • GPU layers
      • Context window size
      • Thread count
      • Batch sizes

    LlamaCPPEngine

    • Generates text responses from prompts
    • Supports vision inputs for compatible models
    • Configurable generation parameters (temperature, top_p, etc.)
    • Memory cleanup options

    LlamaCPPMemoryCleanup

    • Manually manages memory usage
    • Various cleanup modes available

    Requirements

    • ComfyUI
    • llama-cpp-python (from https://github.com/JamePeng/llama-cpp-python)

    License

    This project is licensed under the GNU AGPLv3 License - see the LICENSE file for details.

    Repository

    https://github.com/sebagallo/comfyui-sg-llama-cpp