Nodes/Model Utility Toolkit/Analyze Embedding Similarity (2 Models)
ComfyUI Node

Analyze Embedding Similarity (2 Models)

Compare two embeddings without guessing which tokens match

By silveroxides·Created about a year ago·Updated about 24 hours ago· 14
Analyze Embedding Similarity (2 Models)
    • comparison_report
    • cwb_report
    • documentation
    execution_mode
    model_a
    model_b
    cwb_similarity_alignmentfalse
    top_weight_differences20
    process_device
    force_clear_cachetrue
    exclude_patterns
    glob_patternsfalse

    Embeddings (textual-inversion vectors) are the smallest thing you can merge, and the least like the other model types. A checkpoint or a UNet has fixed layer coordinates - row 3 of tensor A is the same row in tensor B. An embedding is a bag of token vectors, and nothing guarantees that token #5 in one file means the same thing as token #5 in the other. That's why this node has a knob the checkpoint and diffusion-model versions don't: optional one-to-one alignment.

    How it works

    Like the rest of the pack's analysis nodes, it streams both files from models/embeddings in low-memory mode and produces two text reports. The standard one has the usual metrics - MAE, MSE, RMSE, relative L2, cosine similarity, Pearson correlation, and the exact-equality/sign-agreement checks - while missing keys and shape mismatches are reported separately rather than padded.

    The second report is the CWB diagnostic, and here's where embeddings get special treatment. By default the node assumes tokens line up by index, which is the fast and often-wrong assumption. Flip cwb_similarity_alignment on and it runs a greedy one-to-one matching pass that pairs tokens by cosine similarity instead - reporting coverage, matched-score statistics, and how much better similarity alignment is than index alignment. The tooltip is honest about the cost: it's quadratic, and "can be slow for large embeddings." For a typical SDXL embedding with a few thousand vectors, that's a few seconds; for something huge it's a real wait. Enable it when you genuinely suspect two files were trained with tokens in different orders.

    Inputs that matter

    • model_a / model_b - the two files from your embeddings folder.
    • cwb_similarity_alignment (default off) - the alignment pass described above. The one setting unique to this node.
    • top_weight_differences (default 20) - largest individual differences to detail; zero disables the list.
    • exclude_patterns / glob_patterns - drop matching tensors from every metric. Regex unless you toggle globs.
    • process_device, force_clear_cache, execution_mode - same as the other analysis nodes: CUDA with CPU fallback on OOM, cache flushing on by default, and a DOCUMENTATION ONLY mode that loads nothing.

    Outputs

    comparison_report, cwb_report, documentation - plain text, wire to Show Text. Nothing is written or modified; this node exists purely to inform a decision, usually "should I merge these two embeddings with CWBEmbeddingTwoMerger?"

    Why you'd reach for it

    Two embeddings trained on the same concept by different people, and you want to see whether they learned the same thing or drifted apart. Or you want to check whether a new version of your own embedding is a conservative update or a rewrite. The alignment toggle is the reason to prefer this over comparing the raw files yourself - doing cosine-pair matching by hand is miserable.

    Install

    Part of Model Utility Toolkit (silveroxides/ComfyUI-ModelUtils). ComfyUI Manager → search "Model Utility Toolkit", or:

    cd ComfyUI/custom_nodes
    git clone https://github.com/silveroxides/ComfyUI-ModelUtils
    

    Restart. The meaningful dependency is unifiedefficientloader; the other requirements.txt entries (requests, Pillow, mutagen, av) back the pack's downloader nodes. Keep ComfyUI current - this pack uses the newer extension API and won't appear on a stale core.

    CategoryModelUtils/Analysis

    Inputs (9)

    NameTypeDefaultDescription
    execution_modeCOMBOANALYZE streams both files and produces standard and CWB diagnostic reports. DOCUMENTATION ONLY performs no model loading.
    model_aCOMBOFirst file in the comparison. Reports identify values and keys unique to this input separately from Model B.
    model_bCOMBOSecond file in the comparison. Inputs are analyzed only; neither file is modified or merged.
    cwb_similarity_alignmentBOOLEANfalseRun threshold-independent CWB-style greedy one-to-one alignment. This is quadratic and can be slow for large embeddings.
    top_weight_differencesINT200–1000Number of largest individual absolute parameter differences retained for the detailed report. Zero disables this list.
    process_deviceCOMBODevice for per-work-unit floating-point analysis. A CUDA OOM retries only the affected unit on CPU.
    force_clear_cacheBOOLEANtrueRun garbage collection and clear the CUDA allocator cache after every analyzed work unit. Saves retained memory but slows analysis.
    exclude_patternsSTRINGOne pattern per line. Matching tensors are excluded from all comparison metrics and topology counts. Uses regex unless Glob Patterns is enabled.
    glob_patternsBOOLEANfalseInterpret exclusion entries as shell-style glob patterns instead of regular expressions.

    Outputs (3)

    NameTypeDescription
    comparison_reportSTRING
    cwb_reportSTRING
    documentationSTRING