Nodes/comfyui-timm-backbone/Timm Backbone Image Encode
ComfyUI Node

Timm Backbone Image Encode

Turning a picture into a vector you can actually use

By p1atdev·Created about a year ago·Updated about a year ago· 1
Timm Backbone Image Encode
  • model
  • image
  • TENSOR
feature_typepooler_output
hidden_state_index-1

This is the node that makes the whole pack worth having. Timm Backbone Image Encode takes the model from Timm Backbone Loader, runs your image through it, and hands you the features - a tensor that describes what's in the picture in a way a computer can compare, cluster, and search. Feed a batch of images through it and you've built the front half of an image retrieval or similarity pipeline inside ComfyUI.

Here's the thing to understand before touching it: it does not preprocess your image. The node's contract is "give me an already-normalized image in the shape the model expects." The pack ships Image Normalize and RGB to BGR nodes for exactly this reason - the README says it straight: preprocessing is on you. If you feed raw pixels, your "embeddings" will be garbage and you'll blame the wrong node.

What comes out, and the choice that decides it

One output, TENSOR - a raw torch.Tensor, in fp16, on your GPU. There's a feature_type enum that picks which kind of features you get:

  • pooler_output (default) - one vector per image, shaped [batch, hidden_width]. This is the global descriptor: the class token pulled out of the model's final layer. This is what you want for image similarity, retrieval, or clustering. Same family of descriptor that IP-Adapter-style conditioning draws from.
  • hidden_state - a sequence of patch tokens, shaped [batch, num_tokens, hidden_width]. For a 224px ViT with patch 14, that's 256 spatial tokens - the image broken into a grid of local features. This is the one to reach for when position matters: dense matching, attention maps, or feeding features into a custom node that wants spatial structure.

When feature_type is hidden_state, the hidden_state_index input kicks in: which layer to pull from, defaulting to -1 (the last one). Negative indexing counts from the end, so -2 is the penultimate layer. Exactly like "CLIP skip" but for the vision backbone - earlier layers give you more low-level texture, later layers more semantics, and for most jobs -1 is the sane default.

How it actually runs

The encode call permutes your [batch, h, w, c] ComfyUI image to [batch, c, h, w], casts to fp16, moves it to the torch device, and runs it:

  • pooler_outputmodel.forward_features() then model.forward_head() - the head was stripped by the loader, so you get the pooled feature, not logits.
  • hidden_statemodel.forward_intermediates(), grab intermediates[hidden_state_index], and reshape the feature map from [batch, channels, h, w] to the flat token grid [batch, tokens, channels].

That reshape is why the output is always 3D regardless of which mode you pick.

The pipeline you actually want

Load Image → (resize to the model's input size, e.g. 224)
          → Image Normalize (right mean/std for the model - see that article)
          → [RGB to BGR if the model wants it]
          → Timm Backbone Image Encode
          → Timm Embeds Print (to check you're sane) → your custom node

There's no resize node in this pack, so you'll need one from elsewhere in your graph. And remember the gotcha from the Loader: this MODEL is a timm backbone, not a diffusion model - the only node in this pack that can consume it is this one.

Common issues

  • Embeddings look like noise. Almost always normalization or channel order - check the mean/std against what your chosen model was trained with, not the default.
  • Nothing downstream accepts TENSOR. Correct - that's the point. The output feeds Timm Embeds Print and any custom node you write that declares a TENSOR input. If you need to see the numbers, print them.
  • Wrong image size. The model will run on whatever resolution you feed, but off-spec sizes give degraded features. Match the architecture's training size (the 224 in the model name is a hint).

Install is shared with the rest of the pack: ComfyUI Manager search timm backbone, or clone https://github.com/p1atdev/comfyui-timm-backbone into custom_nodes and pip install -r requirements.txt. One dependency (timm), no bundled models.

Categoryimage

Inputs (4)

NameTypeDefaultDescription
modelMODEL
imageIMAGE
feature_typeCOMBOpooler_output2 options: pooler_output, hidden_state
hidden_state_indexINT-1-128–128Index of the hidden state to extract. Only used when feature_type is 'hidden_state'. -1 means the last, -2 means the penultimate.

Outputs (1)

NameTypeDescription
TENSORTENSOR