FABRIC Patch Model

FABRIC stands for "Feedback via Attention-Based Reference Image Conditioning," and the one-sentence version is: it's RLHF without any training. You hand a diffusion model a few reference images you like (and optionally some you hate), and during denoising it steers toward the liked ones and away from the disliked ones. No LoRA, no fine-tune, no extra weights - just an attention trick from a 2023 paper (arXiv 2307.10159). This node is how you bolt that onto ComfyUI without giving up your usual sampler.

The idea caught on in the A1111 world first, where the sd-webui-fabric extension turned it into a "personal RLHF" tool: upvote an output you like, rerun, and the next batch drifts toward your taste. ssitu ported the whole thing to ComfyUI as ComfyUI_fabric, and this node is the entry point that keeps your existing sampler intact.

What it does. Feed it your model, a CLIP encoder, and a set of reference latents, and it returns a patched MODEL that you wire straight into your normal KSampler. That's the whole trick - you keep your regular sampler, scheduler, and settings, and the patch silently changes how self-attention behaves during denoising. It's the lowest-friction way into FABRIC: one node in front of whatever workflow you already have.

How the mechanism works. Under the hood it clones your model and wraps its attention. The reference latents (pos_latents, neg_latents) get noised to each timestep, pushed through the U-Net with an empty text embedding, and the self-attention hidden states of each transformer block are captured. During the real denoising pass, those stored states are concatenated onto the keys and values of your live query, so the image effectively "attends to" your liked and disliked references as extra context. The pos_weight / neg_weight inputs scale how strongly those reference tokens pull. That's it - no gradients, no training loop, just an injection at inference time.

The inputs that matter. Three of them:

• model - your checkpoint's model output.
• clip - the convenience input. This node runs an empty CLIPTextEncode internally to build the "null" conditioning the technique needs, so you don't have to construct it yourself.
• pos_weight / neg_weight - both default to 1.0, and the README is blunt: that's probably too high in most cases. Treat them as your main dial and expect to pull them down.

pos_latents and neg_latents are optional LATENT inputs - your liked/disliked images, encoded with a VAE and fed in as batches. Skip them and the patch silently does nothing (it prints a note and hands you back the unpatched model).

Install. It's a plain clone with no dependencies and no model downloads:

cd ComfyUI/custom_nodes
git clone https://github.com/ssitu/ComfyUI_fabric

Then restart ComfyUI. Or use ComfyUI Manager and search for "ComfyUI fabric" (it's listed as the pack title). Nodes appear under the FABRIC/ menu.

Where people get burned. Three things. First, keep the input latent, pos_latents, and neg_latents all the same pixel size - mismatches get force-resized with bilinear interpolation, which the author calls out as a bad way to resize latents; resize in pixel space instead. Second, attention patching like this is fragile across ComfyUI versions - this pack's last commit is from May 2024, and it monkeypatches the attention backends, so expect it to need babysitting after big ComfyUI updates. Third, memory: reference latents mean extra forward passes, and OOM is the classic complaint. Switch your cross-attention method (xformers ↔ pytorch) or use fewer reference images.

One honest limitation of the patch approach: it applies FABRIC feedback across all denoising steps, and the researchers recommend feedback only in the first half of sampling. If you want a windowed version, use the KSampler FABRIC nodes from this same pack instead.