Nodes/ComfyUI-Prompt-MZ/MinusZone - ModelConfigDownloaderSelect(LLamaCPP)
ComfyUI Node

MinusZone - ModelConfigDownloaderSelect(LLamaCPP)

Pick a preset LLM and let the node fetch it for you

By MinusZoneAI·Created 2 years ago·Updated about a year ago· 138
MinusZone - ModelConfigDownloaderSelect(LLamaCPP)
    • llama_cpp_model_config
    model_name
    chat_formatauto

    ModelConfigDownloaderSelect(LLamaCPP) is the "I don't want to source a GGUF file by hand" node. It gives you a dropdown of the pack's curated text-LLM list - Llama 3, Qwen2, Phi-3, omost, a couple of Chinese-tuned Llama 3s - and when you run your workflow it downloads whichever you picked into ComfyUI/models/gguf and hands a config object to CLIPTextEncode(LLamaCPP Universal).

    The list comes from the pack's configs/model_zoo.json, and here's the thing worth knowing: the default download source is ModelScope, with SHA256 checksums pinned per model. That's a China-first choice from the author (the pack's README is bilingual Chinese/English), which means downloads can be slower from outside China but are checksum-verified - a nice touch, since a corrupted GGUF produces baffling prompt output.

    The two inputs

    • model_name - pick from presets like Meta-Llama-3-8B-Instruct.Q4_K_M, qwen2-7b-instruct-q5_k_m, qwen2-0_5b-instruct-q5_k_m, omost-llama-3-8b-Q4_K_M, llama3-zh.Q4_K_M, and more. Default-quality picks: the Q4_K_M Llama 3 is the pack's own default; the 0.5B Qwen is the one to choose if you're on CPU and want speed over eloquence.
    • chat_format - auto plus llama.cpp's chat handler list (llama-2, llama-3, alpaca, qwen, vicuna, and ~25 more). auto detects it from the model; only set it manually if the LLM responds with garbage formatting.

    One output: llama_cpp_model_config, which feeds the llama_cpp_model input on the universal text-encode node.

    How it works

    The node doesn't download anything at node-creation time - it returns a "DownloaderSelect" config, and the actual fetch happens on first run of the encode node, which resolves the model name against model_zoo.json, downloads it if missing, verifies the SHA256, and loads it through llama.cpp. Already have the file? It just uses it. So you can pre-seed ComfyUI/models/gguf/ with your own copy of the same GGUF and the node skips the download entirely.

    The quantization note from the GGUF world applies: Q4_K_M is the accepted sweet spot for 8B models on ~8-12GB cards, and it's what this pack defaults to everywhere. If you want crisper output and have VRAM to spare, grab a Q5/Q6 file manually and use the ManualSelect sibling instead - this node only offers its preset list.

    Installing

    Part of ComfyUI-Prompt-MZ - ComfyUI Manager, search "ComfyUI-Prompt-MZ", or:

    cd ComfyUI/custom_nodes
    git clone https://github.com/MinusZoneAI/ComfyUI-Prompt-MZ
    

    Restart, then require llama-cpp-python + diskcache (the pack's requirements.txt installs them on Manager install).

    Troubleshooting

    The two classic failure points are both in the README FAQ. If the first run dies with a llama_cpp_python wheel build returning nonzero exit status, that's the network failing to pull the binary wheel - retry with a proxy or install llama-cpp-python manually first. If you hit module 'llama_cpp' has no attribute 'LLAMA_SPLIT_MODE_LAYER', your llama-cpp-python is stale; upgrade it. And if the download stalls against ModelScope from a non-China network, be patient or download the same GGUF from HuggingFace into ComfyUI/models/gguf yourself - the node will pick it up.

    CategoryMinusZone - Prompt/others

    Inputs (2)

    NameTypeDefaultDescription
    model_nameCOMBO15 options: Meta-Llama-3-8B-Instruct.Q4_K_M, llama3_if_ai_sdpromptmkr_Q4_K_M, qwen2-7b-instruct-q5_k_m, qwen2-0_5b-instruct-q5_k_m, omost-llama-3-8b-Q4_K_M, omost-phi-3-mini-128k-Q4_K_M, +9
    chat_formatCOMBOauto28 options: auto, llama-2, llama-3, alpaca, qwen, vicuna, +22

    Outputs (1)

    NameTypeDescription
    llama_cpp_model_configLLamaCPPModelConfig