Sampler Scheduler Metrics Tester

So you've hit the ritual: someone posts "DPM++ 2M Karras is the best," you wire it in, and it's fine - but you never actually know if it's better than Euler for your checkpoint, your prompt, your seed. The Sampler Scheduler Metrics Tester is a single node that settles it by brute force. You give it a model, a VAE, conditioning, and one latent, and it runs the entire grid of sampler × scheduler combinations, decodes every one, and hands you a labeled contact sheet plus a text report. No re-wiring, no re-queueing, no manually renaming fifty PNGs.

It comes from the Icelandic Center for Artificial Intelligence (ICAI), and the README is honest that it was built for their internal use and published in case anyone else finds it useful. That's exactly the vibe - it's a pragmatic test harness, not a shiny product.

How it works

The mechanism is what makes the results trustworthy. First, the node grabs every sampler and scheduler ComfyUI knows about (comfy.samplers.KSampler.SAMPLERS / SCHEDULERS), then generates one base noise tensor from your seed and reuses it for every combination. That's the important part: same seed, same starting noise, same steps, same cfg, same denoise across the whole grid. The only variable that changes between runs is the sampler and scheduler, so the differences you see on the contact sheet are actually attributable to them. It then times each generation, VAE-decodes it, and computes three image metrics via OpenCV:

• LapVar - Laplacian variance, a standard sharpness/blur measure.
• GradMean - average gradient (edge) strength, a rough detail proxy.
• FFTScore - how much high-frequency content the image has.

Before you get excited: these are sharpness metrics, not quality scores. A high LapVar can just mean noisy, crunchy output. ICAI says they're for internal triage, and that's the right way to read them - useful for spotting the combo that's visibly broken, not for crowning a "best" sampler. One combo throws an error? The node isolates it: the run keeps going, that slot gets a gray placeholder marked ERROR with the message burned onto the image.

The inputs that matter

Everything's a standard KSampler parameter - model, vae, positive, negative, latent_image, seed, steps, cfg, denoise - plus two override strings that are the whole point:

• sampler_list_override - comma/newline list like euler, dpmpp_2m. Empty means all samplers.
• scheduler_list_override - same idea for normal, karras, simple etc. Empty means all.

The three cosmetic inputs (font_size_ratio, text_color, text_bg_color) just control the overlay text. Leave them alone unless the annotation is unreadable.

Outputs

Three: Latents (the full batch of generated latents), Overlaid Images (the annotated RGB batch - feed this to a Preview Image or save node), and Info, a single string with one line per combo: status, sampler + scheduler, seed, time, and the three metrics. Info is plain text, so wire it to a Show Text node to read it.

Installing it

Via ComfyUI Manager, search "Sampler Scheduler Metrics Tester" - or manually:

cd ComfyUI/custom_nodes/
git clone https://github.com/IcelandicCenterArtificialIntelligence/ComfyUI-SamplerSchedulerMetricsTester.git
pip install opencv-python

then restart ComfyUI. The only dependency is OpenCV (opencv-python), which is a chunky install - but chances are a node you already run pulled it in. No model downloads, no API keys, nothing exotic.

Where people get burned

The biggest trap is scale: all samplers × all schedulers is roughly 150–200 full generations, each with a VAE decode. That's a long queue at 20 steps. Keep the grid small - use the override strings, or drop steps for the sweep and re-run the finalists properly. Also remember the sweep is fixed at one steps/cfg/denoise for everything; this node compares samplers and schedulers, not your full hyperparameter space. And one caution that applies to any comparison like this: the "best" combo really does depend on the model. Per the KB, flow-matching checkpoints (Flux, Z-Image, etc.) often hate Karras, while SDXL-era fine-tunes love it - which is exactly why generating your own evidence instead of trusting a random post is the right instinct.