Nodes/Trent Nodes/Eneas Segment & Track (Trent)
ComfyUI Node

Eneas Segment & Track (Trent)

Type the object's name, get a mask on every frame

By TrentHunter82·Created 9 months ago·Updated 3 days ago· 39
Eneas Segment & Track (Trent)
  • images
  • mask
mode
textthe person
points[]
annotation_frame0
offload_frames_to_cputrue
accept_threshold0.90
reject_threshold0.10

You have a clip. You need the mask of one thing in it - a person, a jacket, a mug - on every single frame. Hand-rotoscoping is out, and SAM alone doesn't help you because SAM segments the frame it's shown, not the clip. That's the job this node does: describe the object once, name one frame, and get a full-length binary mask back.

What it actually is

Eneas (Embedding-guided Neural Ensemble for Adaptive Segmentation, Apache 2.0 from Sperid Labs) is an open-vocabulary segmentation and tracking model. Trent's node wraps it for ComfyUI as an isolated worker, so you get the whole pipeline behind one node with one output.

Inside, it's an ensemble rather than one model. Florence-2 locates what you described, and SeC-4B tracks that instance forwards and backwards from the frame you annotated. Category mode swaps the tracker out entirely: Florence-2, SigLIP2 NaFlex and SAM2.1 Hiera-large each propose detections independently per frame, and a Qwen3-VL validator (via a private loopback Ollama server the node starts and stops itself) votes down false positives.

If you've read anything about promptable masking, this is the same idea as Grounded SAM - detector finds the thing, segmenter masks it - except one node, no GroundingDINO wiring, and the tracker remembers the instance across frames.

The inputs that matter

images is the batch - a Load Image, a video loader, whatever produces IMAGE frames. mode picks the behaviour, and it changes everything else:

  • track_text - tracking one instance. Fill in text with a description, "the person in the red jacket" rather than "person". Set annotation_frame to the zero-based frame where the object is easiest to see; tracking propagates both directions from there.
  • track_points - same tracking, but you point instead of describe. points is hand-typed JSON: [[180,120,1],[20,20,0]] is pixel x, pixel y, and a label where 1 includes that area and 0 excludes it. Those coordinates are in the input frame's own resolution, and there's no click-to-place canvas - you read them off the image. text is ignored in this mode.
  • category - every instance of a category, per frame, with no identity tracking. Here text is a bare noun, "person", and annotation_frame does nothing. accept_threshold (0.9) and reject_threshold (0.1) only apply in this mode; loosening them finds more and invents more. The node refuses to run if reject is higher than accept.

offload_frames_to_cpu is the one you'll touch for VRAM: it defaults on, which keeps the staged frames in system RAM instead of eating VRAM, at some speed cost.

One output: mask, a float MASK of shape [frames, height, width] at the original resolution, in original frame order. White is foreground, black is background. Wire it into MaskToImage → Preview, an InpaintModelConditioning, or a compositor.

Installing it - this one is not a Manager click

Eneas runs in its own virtualenv because its Transformers 4.53 conflicts with the 5.x that modern ComfyUI uses, so the pack never touches your ComfyUI environment. That also means pip install -r requirements.txt gets you the node but not the model. Get the pack first, either through ComfyUI Manager (search "Trent Nodes") or:

cd ComfyUI/custom_nodes
git clone https://github.com/TrentHunter82/TrentNodes.git
pip install -r TrentNodes/requirements.txt

Then set up the Eneas runtime from a Linux/WSL shell:

cd /path/to/ComfyUI
bash custom_nodes/TrentNodes/tools/install_eneas.sh /mnt/d/models/eneas /mnt/d/models/florence2/large

That wants uv, Git, curl, an x86-64 CUDA box, and downloads Ollama 0.33.3 privately into .eneas/ollama. Omit the second argument and Florence-2 downloads itself. Weights (SeC-4B, Florence-2-large, SigLIP2, SAM2.1 Hiera-large, the Qwen3-VL Ollama tag) fetch on first use - several GB, and interrupted Hugging Face downloads do resume.

Where people get burned

  • Native Windows. The installer is a bash script for WSL/Linux and the node hard-requires CUDA - RuntimeError: Eneas tracking requires an NVIDIA CUDA GPU. No CPU mode, no Mac.
  • ComfyUI Manager being weird about this pack. The author's own note: the repo was renamed early and the registry ended up with two entries, so the Manager installs still throw warnings for some people. Git clone is the boring, reliable path.
  • Models reload every execution. ComfyUI unloads its own models before handing off, and the worker re-reads weights each run. On a big mounted drive that's real startup time per queue, not a bug.
  • Treating the mask as a matte. These are hard binary masks, not alpha. Hair, motion blur and glass will look cut out rather than blended - that's a matting problem, and you want BiRefNet's matting weights or the pack's MatAnyone node instead.
  • Category mode collapsing instances. Every detected instance gets OR'd into one mask per frame, so two people become one blob. One mask per frame is the shape the node promises; don't ask it to separate them.
CategoryTrent/Segmentation

Inputs (8)

NameTypeDefaultDescription
imagesIMAGE
modeCOMBO3 options: track_text, track_points, category
textoptSTRINGthe person
pointsoptSTRING[]JSON pixel coordinates: [[x,y,1],[x,y,0]]. 1 includes, 0 excludes. Used only in track_points mode.
annotation_frameoptINT00–1000000Zero-based frame to annotate. Tracking propagates both forwards and backwards.
offload_frames_to_cpuoptBOOLEANtrue
accept_thresholdoptFLOAT0.900–1
reject_thresholdoptFLOAT0.100–1

Outputs (1)

NameTypeDescription
maskMASK