Build AutoTuner Python Evaluator

Be upfront with yourself about who this node is for: it's a developer hook, not a beginner control. If you don't already have Python code you want to run against candidate merges, you don't need this node - LoRA AutoTuner's built-in scoring works fine without it. This exists for the case where you do have something specific to check - a face-similarity score against a reference image, a CLIP-based aesthetic score, a comparison against a known-good output - and you want the AutoTuner's ranking to take that into account instead of relying purely on its internal merge-quality heuristics.

What it is and why you'd use it

LoRA AutoTuner ranks candidate merge configurations using built-in metrics (sparsity, energy, that kind of thing). This node lets you supply an external Python function that gets called for each candidate and returns a score, which the AutoTuner then blends with - or fully replaces - its own ranking. It's the extension point for anything the built-in scoring can't see: "does this merge still look like the reference character," "does this pass my aesthetic filter," anything you can express as code that takes a candidate and returns a number.

How it works

You point it at a Python module and a callable name inside that module. The AutoTuner calls that function once per candidate merge during its sweep, passing in the merged model, clip, the LoRA data, the candidate's config, any context you supplied, and a summary of the analysis. Your function does whatever comparison logic you want and returns either a plain float score or a dict with a score key plus extra details for the report.

The expected signature, per the tooltip: fn(model=..., clip=..., lora_data=..., config=..., context=..., analysis_summary=...) -> float | {"score": float, "details": ...}. That's the contract - match it and the AutoTuner can call your code.

The inputs and outputs that matter

• module_path - a file path or importable module name containing your evaluator function.
• callable_name (default evaluate_candidate) - the function to import from that module.
• combine_mode (default blend) - how your score combines with the AutoTuner's built-in score. blend mixes them by weight, external_only uses just your function's score, multiply combines them multiplicatively.
• weight (default 0.5) - only matters in blend mode; how much your evaluator's score counts relative to the built-in one.
• context_json (default "{}") - a JSON object passed through to your evaluator's context argument, for anything you want available inside the function without hardcoding it.

Output: evaluator (AUTOTUNER_EVALUATOR) - connect it to the evaluator input on LoRA AutoTuner or LoRA AutoTuner Settings.

How to install it

Same as the rest of the pack - ComfyUI Manager, search "LoRA Optimizer", install, restart. Manually:

cd ComfyUI/custom_nodes/
git clone https://github.com/ethanfel/ComfyUI-LoRA-Optimizer.git

Restart - nodes appear under loaders. This particular node has no model downloads of its own, but it does need your own Python evaluator code to already exist somewhere ComfyUI's Python environment can import it.

Common issues & troubleshooting

Module or function not found. module_path needs to be either a file path ComfyUI's process can actually reach, or a module name that's genuinely importable in that Python environment - not just a path that looks right from your file browser. Test the import in a plain Python shell inside the same environment ComfyUI runs in before assuming the node is broken.

Your evaluator changes results in confusing ways. Start with combine_mode=blend and a low weight so your custom score nudges the ranking rather than dominating it, then increase weight once you trust your function's output. Jumping straight to external_only means any bug in your evaluator directly controls which candidate wins, with no built-in sanity check underneath it.

Sweep gets much slower after adding this. That's expected - your function now runs once per candidate on top of the AutoTuner's own scoring pass. If your evaluator does something heavy (running actual inference, hitting an external model), consider lowering top_n on the AutoTuner so fewer candidates reach the evaluation stage.