ComfyUI Node

Smart Model Loader

Smart Model Loader [Eclipse]

By r-vage·Created 29 days ago·Updated 3 days ago· 2
Smart Model Loader
    • pipe
    featuresclip,vae,memory_cleanup
    template_actionNone
    template_nameNone
    new_template_name
    model_typeStandard Checkpoint
    ckpt_nameNone
    unet_nameNone
    nunchaku_nameNone
    qwen_nameNone
    zimage_nameNone
    gguf_nameNone
    weight_dtypedefault
    data_typebfloat16
    cache_threshold0.0
    attentionflash-attention2
    i2f_modeenabled
    cpu_offloadauto
    num_blocks_on_gpu30
    use_pin_memoryenable
    gguf_dequant_dtypedefault
    gguf_patch_dtypedefault
    gguf_patch_on_devicefalse
    blocks_to_swap10
    offload_embeddingsfalse
    sampling_methodNone
    shift_video12.00
    shift_audio3.00
    sampling_subtypeeps
    shift3.0
    base_shift0.5
    sampling_width1024
    sampling_height1024
    original_timesteps50
    zsnrfalse
    sigma_max120.00
    sigma_min0.00
    clip_sourceBaked
    clip_count1
    clip_name1None
    clip_name2None
    clip_name3None
    clip_name4None
    clip_typeflux
    enable_clip_layertrue
    stop_at_clip_layer-2
    vae_sourceBaked
    vae_nameNone
    audio_vae_sourceExternal
    audio_vae_nameNone
    resolution1024x1024 (1:1 XL/SD3/Flux/HiDream)
    width1024
    height1024
    lora_count1
    lora_switch_1false
    lora_name_1None
    lora_weight_11.0
    lora_switch_2false
    lora_name_2None
    lora_weight_21.0
    lora_switch_3false
    lora_name_3None
    lora_weight_31.0
    sampler_nameeuler
    schedulernormal
    steps20
    cfg8.0
    denoise1.00
    flux_guidance3.5
    batch_size1
    verify_fileoff
    expected_hashes{}
    air_or_hash
    download_locators[]
    download_target_role
    model_precisiondefault
    seed0

    "Smart" is doing heavy lifting in that name, but for once it's earned. This is the flagship node of the ComfyUI Smart Model Loader pack: one loader that grabs your checkpoint or diffusion model, your CLIP, your VAE, your latent, your sampler settings, your seed, and your LoRAs, and hands the whole lot downstream over a single PIPE wire. If you've ever built a workflow and watched it become forty crossing noodles between the loader, empty latent, and sampler, you know exactly why that exists.

    The trick is the combo-chip bar at the top. features is a comma-separated list of toggles - clip, vae, audio_vae, latent, sampler, lora, model_sampling, block_swap, memory_cleanup, integrity, seed, templates - and each chip you click reveals (or hides) that whole section of the node. Disabled sections don't just look tidy, they're not even put into the pipe. That's the design philosophy: start minimal with clip, vae, memory_cleanup, and grow the loader as the workflow needs it. The selected chip state is saved with the workflow, so a shared graph loads looking exactly like its author left it.

    Pick a model_type and the loader adapts. Standard Checkpoint reads a full .safetensors with baked CLIP and VAE. UNet Model is diffusion-only and expects external CLIP and VAE. Nunchaku and GGUF are the quantized paths - FP4 for NVIDIA via SVDQuant, or llama.cpp-style GGUF - and both need their own ComfyUI extensions installed before the loader will even list those options. Each type surfaces only the widgets that matter to it: weight_dtype for UNets, data_type/attention/i2f_mode for Nunchaku, gguf_dequant_dtype for GGUF. GGUF is worth it if VRAM is the wall you keep hitting; Q8 is basically fp16 at half the size, and if it fits, just use it.

    Two features genuinely set this loader apart. Templates (the templates chip) save your whole configuration - model file, chips, sampler, LoRA slots - to ComfyUI/custom_nodes/ComfyUI_SmartModelLoader/templates/ and restore it with one load. If a template references a model you don't have, the loader keeps the filename selected with a (missing) marker, restores the saved CivitAI AIR identity, and reveals a Download from CivitAI button - it never downloads on its own. Integrity (integrity chip) is where you paste a CivitAI AIR URN or SHA-256 into air_or_hash; the loader hashes the file, warns on mismatch, and can fetch a missing model into the folder you pick with download_target_role.

    Output is one pipe that carries model, clip, vae, latent, dimensions, sampler settings, seed, model_name, and more. Feed it to the pack's IO Checkpoint Loader to unpack individual sockets, to Eclipse KSampler (Pipe) to sample and decode, or to optional ComfyUI Eclipse nodes (Context Image, Generation Data, Concat Pipe Multi, Smart Sampler Settings) that consume the same pipe contract.

    Installing

    ComfyUI Manager - search ComfyUI Smart Model Loader - or manually:

    cd ComfyUI/custom_nodes
    git clone https://github.com/r-vage/ComfyUI_SmartModelLoader.git
    cd ComfyUI_SmartModelLoader
    python -m pip install -r requirements.txt
    

    Restart ComfyUI. The [Eclipse] suffix is a compatibility identifier - ComfyUI Eclipse itself is not required, and existing Eclipse workflows load without node replacement. Nunchaku and GGUF support need ComfyUI-Nunchaku and ComfyUI-GGUF cloned into custom_nodes; without them everything else still works.

    Where people get burned

    • Legacy formats are denied by default. .ckpt, .pt, .pth, .bin (pickle-capable) are rejected until you enable Smart Model Loader → General → Allow Legacy Model Formats. Safe, but surprising if you live on old checkpoints.
    • CLIP errors usually mean you picked the wrong source. UNet and quantized models need clip_source = External; baked only works for standard checkpoints.
    • OOM with a big model? Enable block_swap (start around 10 blocks) or model_sampling → Flux for correct scheduler shift - the default settings won't be right for every architecture.
    Category🌒 Smart Model Loader/ Loader

    Inputs (76)

    NameTypeDefaultDescription
    featuresSTRINGclip,vae,memory_cleanupComma-separated list of enabled loader features. Selected via the Mode Bar chip widget on the frontend.
    template_actionCOMBONoneManage saved presets/templates for this node: • None: Do nothing • Load: Load configurations from template_name • Save: Save current configurations into a new template.
    template_nameCOMBONoneSelect a saved configuration template to load or delete.
    new_template_nameSTRINGEnter a name for the new configuration template when saving current settings.
    model_typeCOMBOStandard CheckpointType of model architecture to load: • Standard Checkpoint: Full model (diffusion, CLIP, VAE) in one file • UNet Model: Diffusion-only model • Nunchaku: FP4 quantized GPU inference models • GGUF Model: llama.cpp/GGUF quantized models
    ckpt_nameCOMBONoneSelect a standard Stable Diffusion or Flux checkpoint file containing diffusion, CLIP, and VAE weights.
    unet_nameCOMBONoneSelect a standalone UNet/Diffusion model checkpoint (e.g. Flux, SD3, AuraFlow) from the diffusion_models directory.
    nunchaku_nameCOMBONoneSelect a Nunchaku FP4-quantized model for Flux to load.
    qwen_nameCOMBONoneSelect a Nunchaku FP4-quantized model for Qwen2-VL to load.
    zimage_nameCOMBONoneSelect a Nunchaku FP4-quantized model for ZImage to load.
    gguf_nameCOMBONoneSelect a GGUF format diffusion model from the diffusion_models_gguf directory.
    weight_dtypeCOMBOdefaultPreferred weight precision for loading checkpoint weights (e.g. default, fp8_e4m3fn, fp8_e5m2, bfloat16). FP8 saves VRAM.
    data_typeCOMBObfloat16Data type precision for Nunchaku FP4 model layers. Select bfloat16 or float16.
    cache_thresholdFLOAT0.00–1GPU memory caching threshold for Nunchaku FP4 layers. Higher values reserve more GPU cache for layers.
    attentionCOMBOflash-attention2Attention implementation variant: • flash-attention2: Highly optimized for modern Ampere/Ada GPUs • nunchaku-fp16: Optimized FP16 attention kernel
    i2f_modeCOMBOenabledGEMM matrix multiplication implementation mode for Nunchaku GPU execution.
    cpu_offloadCOMBOautoToggle offloading of inactive parts of the model from VRAM to CPU RAM to save VRAM.
    num_blocks_on_gpuINT301–60Number of transformer blocks to keep on the GPU for Nunchaku Qwen. Remaining blocks are offloaded to CPU.
    use_pin_memoryCOMBOenableEnable pinned memory (host-allocated memory) for faster tensor transfers between system RAM and GPU VRAM.
    gguf_dequant_dtypeCOMBOdefaultDequantization data type precision for GGUF weights when patching them (default, float16, bfloat16, float32).
    gguf_patch_dtypeCOMBOdefaultData type precision to use when applying LoRA patches to GGUF weights.
    gguf_patch_on_deviceBOOLEANfalseApply patches directly on the GPU rather than host CPU memory (faster but requires more VRAM during load).
    blocks_to_swapINT100–100Number of transformer blocks to swap/offload from GPU to CPU memory. Higher values save significant VRAM but slow down generation. Recommended values (max blocks): • Flux: ~10 (max 57) • SD3: ~8 (max 24) • Wan2.1: ~10 (max 40) • HunyuanVideo: ~10 (max 60) • LTX-Video: ~6 (max 28) Set to 0 to disable.
    offload_embeddingsBOOLEANfalseOffloads the embedding and projection layers (text_embedding, img_emb, time_in) to CPU RAM. Saves ~100-300MB VRAM at a slight speed cost.
    sampling_methodCOMBONoneModel-level sampling correction: • SD3/AuraFlow/Flux/Stable Cascade: Sets standard scheduler shifts • LCM: Configures latent consistency model scheduling • ContinuousEDM/ContinuousV: EDM scheduler • LTXV: LTX-Video scheduler • MiniMax H3: Separate video and audio flow shifts
    shift_videoFLOAT12.000.01–100MiniMax H3 video flow shift. ComfyUI default: 12.0.
    shift_audioFLOAT3.000.01–100MiniMax H3 audio flow shift. ComfyUI default: 3.0.
    sampling_subtypeCOMBOepsSubtype scheduling curve for ContinuousEDM sampling (e.g. eps, v_prediction, edm, cosmos_rflow).
    shiftFLOAT3.00–10Universal scheduling shift multiplier. SD3 default: 3.0, AuraFlow: 1.73, Stable Cascade: 2.0.
    base_shiftFLOAT0.50–10Base scheduling shift for Flux (default: 0.5) and LTX-Video (default: 2.05).
    sampling_widthINT102416–2000Target resolution width for Flux/LTXV sampling shift calculations. Used to scale scheduler step sizes.
    sampling_heightINT102416–2000Target resolution height for Flux/LTXV sampling shift calculations. Used to scale scheduler step sizes.
    original_timestepsINT501–1000Original training timesteps of the LCM model (used to scale distilled step sizes).
    zsnrBOOLEANfalseZero-Terminal Signal-to-Noise Ratio (zsnr) adjustment to allow generating true darks/blacks.
    sigma_maxFLOAT120.000–1000Maximum noise sigma boundary value for ContinuousEDM/ContinuousV scheduling.
    sigma_minFLOAT0.000–1000Minimum noise sigma boundary value for ContinuousEDM/ContinuousV scheduling.
    clip_sourceCOMBOBakedSource of the text encoder (CLIP): • Baked: Uses CLIP embedded in the checkpoint • External: Uses separate CLIP files • External + Model File: Extends external loaders with the UNet file to auto-resolve baked projections (e.g. LTXV Gemma).
    clip_countCOMBO1Number of separate CLIP model/Text Encoder files to load concurrently (e.g., 2 for Flux, 3 for SD3).
    clip_name1COMBONoneSelect the primary CLIP or Text Encoder checkpoint file.
    clip_name2COMBONoneSelect the secondary CLIP or Text Encoder checkpoint file.
    clip_name3COMBONoneSelect the third CLIP or Text Encoder checkpoint file.
    clip_name4COMBONoneSelect the fourth CLIP or Text Encoder checkpoint file.
    clip_typeCOMBOfluxCLIP loader wrapping/architecture mapping to match the target model type (e.g. stable_diffusion, flux, sd3, wan, mochi, ltxv).
    enable_clip_layerBOOLEANtrueEnables stopping CLIP text evaluation at a specific layer (CLIP skip) instead of evaluating all the way to the end.
    stop_at_clip_layerINT-2-24–-1Which layer to stop CLIP text evaluation at (e.g. -2 for SD1.5/SDXL, -3 for SD3). Negative values count back from final layer.
    vae_sourceCOMBOBakedSource of the variational autoencoder (VAE): • Baked: Extract VAE from the checkpoint file • External: Load a standalone VAE file
    vae_nameCOMBONoneSelect a standalone VAE file to load from the vae directory.
    audio_vae_sourceCOMBOExternalSource of the audio decoder/VAE: • External: Uses ComfyUI's general VAE loader for supported audio VAEs, including MiniMax H3 and LTX • Baked: Extracts LTX audio VAE weights directly from an all-in-one model file
    audio_vae_nameCOMBONoneSelect a standalone ComfyUI-supported audio VAE, such as MiniMax H3 or LTX, from the vae directory.
    resolutionCOMBO1024x1024 (1:1 XL/SD3/Flux/HiDream)Select a pre-calculated latent size preset (Select 'Custom' to manually specify width and height). Note: Latent properties (channels, downscale ratio) are automatically detected from the active VAE (baked or external).
    widthINT102416–2000Custom width for empty latent generation. Must be a multiple of 8.
    heightINT102416–2000Custom height for empty latent generation. Must be a multiple of 8.
    lora_countCOMBO1Number of active LoRA slots to configure.
    lora_switch_1BOOLEANfalseToggle to quickly enable or disable this LoRA slot without clearing the filename.
    lora_name_1COMBONoneSelect a LoRA model file from the loras directory.
    lora_weight_1FLOAT1.0-10–10Weight scale to apply to this LoRA's weights. 1.0 is standard strength; negative values invert the effect.
    lora_switch_2BOOLEANfalseToggle to quickly enable or disable this LoRA slot without clearing the filename.
    lora_name_2COMBONoneSelect a LoRA model file from the loras directory.
    lora_weight_2FLOAT1.0-10–10Weight scale to apply to this LoRA's weights. 1.0 is standard strength; negative values invert the effect.
    lora_switch_3BOOLEANfalseToggle to quickly enable or disable this LoRA slot without clearing the filename.
    lora_name_3COMBONoneSelect a LoRA model file from the loras directory.
    lora_weight_3FLOAT1.0-10–10Weight scale to apply to this LoRA's weights. 1.0 is standard strength; negative values invert the effect.
    sampler_nameCOMBOeulerSelect the ComfyUI sampling algorithm (e.g. euler, heun, dpmpp_2m).
    schedulerCOMBOnormalSelect the noise scheduling curve (e.g. normal, karras, exponential, sgm_uniform).
    stepsINT201–150Number of denoising steps. Higher values take longer but refine the image; 20-30 steps is standard for most models.
    cfgFLOAT8.01–30Classifier-Free Guidance (CFG) scale. Controls prompt adherence. Higher values enforce the prompt strictly but can burn colors; 1.0 disables it.
    denoiseFLOAT1.000–1Denoising strength. 1.0 fully denoises the input latent; lower values preserve more of the starting image or latent.
    flux_guidanceFLOAT3.50–10Guidance scale specific to Flux models. Controls prompt adherence/contrast without standard CFG burn.
    batch_sizeINT11–4096Number of latent images to generate in parallel in a single execution batch.
    verify_fileCOMBOoffPrimary model integrity mode: • off: No model hashing. • sidecar: Computes and saves a .sha256 baseline next to the selected primary model. • verify: Compares the primary model when a trusted SHA256 is available and stops on mismatch; without one, it records a local baseline and continues loading. External CLIP, VAE, audio VAE, and LoRA files always retain path and safe-format validation. For downloads, paste a CivitAI URN:AIR or SHA256 into air_or_hash.
    expected_hashesSTRING{}Internal JSON database mapping filenames to expected SHA256 and CivitAI AIR metadata. Managed automatically.
    air_or_hashSTRINGPaste CivitAI AIR (urn:air:...) or a SHA256 hash here. Add +<fileId> to an AIR to select an exact file when several artifacts share a precision. Used to verify integrity on load or trigger auto-downloads if files are missing.
    download_locatorsSTRING[]Internal JSON store containing locator downloads. Managed automatically.
    download_target_roleCOMBOSelect which ComfyUI input folder to save downloaded models into (e.g. checkpoints, diffusion_models, vae).
    model_precisionCOMBOdefaultPreferred weight precision (e.g. fp16, bf16, fp8) when downloading models from CivitAI using URN:AIR locators. 'default' grabs the primary file; an explicit precision selects the unique largest matching artifact. Use the AIR +<fileId> suffix when matching sizes are missing or tied.
    seedINT0-3–18446744073709550000Controls generation reproducibility. Use specific values for deterministic output: • -1: Randomize the seed on every execution • -2: Increment the seed by 1 after each run • -3: Decrement the seed by 1 after each run

    Outputs (1)

    NameTypeDescription
    pipePIPE