Eric Duplicate Image Finder v04

If you generate images for a living (or even as a hobby with no self-control), your output folder is a disaster of near-identical renders. Eric Duplicate Image Finder v04 is the cleanup tool: it scans folders, groups images by visual similarity, and lets you tell exact duplicates apart from close variants - then optionally moves or copies the dupes somewhere organized. It's from the AAA Metadata System pack, and it's one of the few nodes in there that solves a real, universal annoyance rather than a niche one.

The pitch is perceptual hashing: two PNGs that are bit-for-bit different but look the same still get grouped, because the comparison is about what the image looks like, not its bytes. That's the difference between this and a checksum tool - and it's what makes it useful for AI output, where re-renders are never byte-identical but frequently visually identical.

How it works

It computes a perceptual hash for each image using one of the imagehash algorithms - phash (perceptual hash, best general-purpose), dhash (difference hash, structural), ahash (average hash, fast and simple), or whash (wavelet hash). Hashes are compared with a Hamming distance, and the three thresholds sort results into tiers: exact_duplicate_threshold, similar_image_threshold, and variant_threshold - exact matches, near-matches, and related-but-different. You can layer a secondary_hash algorithm on top to reduce false positives, and even weigh in filename patterns and metadata via analyze_filename and analyze_metadata.

The action side is where the work gets done: duplicate_action can move or copy duplicates to an output_folder, keep_largest preserves the biggest file in each group, and save_results writes a JSON or CSV report so you have a record before you delete anything. save_hashes stores computed hashes for faster future scans.

Inputs and outputs that matter

• folder_path - where to scan (required).
• primary_hash - phash (default), dhash, ahash, or whash.
• exact_duplicate_threshold / similar_image_threshold / variant_threshold - the similarity tiers. Lower thresholds = stricter.
• recursive / additional_folders - scan scope.
• duplicate_action, output_folder, keep_largest - what to actually do with the dupes.
• save_results / results_format - write a JSON/CSV report.

Outputs: duplicate_groups (STRING, the groupings), total_duplicates (INT), stats_json (STRING, structured stats), and results_path (STRING, where the report went, if saved).

Installing it

Part of AAA Metadata System by Eric Hiss (GitHub: EricRollei). Via ComfyUI Manager (search "AAA Metadata System") or:

cd ComfyUI/custom_nodes
git clone https://github.com/EricRollei/AAA_Metadata_System.git
cd AAA_Metadata_System
pip install -r requirements.txt

imagehash is in the core requirements, so it installs automatically. If the node errors about imagehash, that dependency went missing - reinstall it.

Common issues

Big folders are slow on first scan - hashing thousands of images takes real time, which is exactly why save_hashes exists for subsequent runs. Threshold tuning is the skill here: default thresholds that are too strict split visually-similar sets apart, too loose group unrelated images together, so test on a small subfolder first. If you enable keep_largest with an output_folder, double-check the output location before running - the node is careful about not destroying originals, but you should be careful too. And a word of caution: this finds similar images, not identical intent - two different crops of the same render will group together, which may be exactly what you want or a surprise, depending on your organization scheme.