Nodes/WAS Node Suite v3/Dataset Health
ComfyUI Node Runs on cloud

Dataset Health

Check a set of training pictures and their captions, from a folder or a dataset, before training: caption coverage and consistency, picture quality flags, near-duplicates, aspect balance and, on request, faces. Draws a score radar, the measurements and every flagged picture on the node, and hands on a report, the scores and the flagged and duplicate names.

By WASasquatch·Created 4 years ago·Updated 3 days ago· 1,900
Dataset Health
  • dataset
  • report
  • scores
  • flagged
  • duplicates
◄folder▾►
◄pattern*►
◄caption_extension.txt►
◄subject[trigger]►
◄target_side768►
◄check_facesfalse►
◄min_words5►
CategoryWAS Suite/Dataset

Inputs (8)

NameTypeDefaultDescription
folderCOMBOWhich folder of pictures to check. A bare 'input' is that folder itself; 'lora/cats [input]' is that folder below it. Folders under paths.allow_read in config.yaml are listed under their own names. Ignored while dataset is wired. A wired path, such as `E:\images`, works too.
patternSTRING*Which files to check. '*' = every picture in the folder; 'shot_*.png' = only those; '**/*' = subfolders too. Files that are not pictures are skipped whatever the pattern says. Ignored while dataset is wired.
caption_extensionSTRING.txtThe caption file beside each picture, under the same name: '.txt' = cat.png reads cat.txt; '.caption' = cat.caption. A picture with no such file counts as uncaptioned. Ignored while dataset is wired.
subjectSTRING[trigger]What every caption should name: `[trigger]`, `ohwx woman`. Case ignored, whole words. Captions holding it are counted; phrases holding it leave the phrase list. Empty = no subject check.
target_sideINT7680–163840 = no size check; 768 = a picture whose short side is under 768 pixels is flagged small. Set it to the resolution the set trains at.
check_facesBOOLEANfalse`off` = no face check; `on` = count the faces in every picture, flag no face, several faces or a face under 5% of the picture's height, and add a faces score: the share with exactly one face. For a character set.
min_wordsINT50–10000 = off; 5 = a caption under 5 words counts as short and lowers consistency.
datasetoptWAS_DATASETUnwired = the folder is read; wired = a dataset, such as Load Dataset's, measured as it stands, with captions and pictures edited upstream and not yet saved. folder, pattern and caption_extension are ignored while it is wired.

Outputs (4)

NameTypeDescription
reportSTRINGThe figures as plain text: the score table, the caption and picture measurements, the flag counts, every flagged picture with its reasons and every near-duplicate pair.
scoresDICTEach score from 0 to 1, 1 = healthy: captions, consistency, quality, uniqueness, balance, and faces while check_faces is on.
flaggedARRAYNames of the pictures carrying any flag, most flags first: below the folder, as 'set/cat_03.png', or the dataset's own names, as 'set/cat_03'. Empty when nothing is flagged.
duplicatesARRAYOne 'a.png | b.png' entry per near-duplicate pair, closest first: a resized, re-saved or mirrored copy. Empty when there is none.