ComfyUI_FlowRVS

FlowRVS锛孌eforming Videos to Masks: Flow Matching for Referring Video Segmentation锛寉ou can use it in comfyUI when use this node.

By smthemex·Created 11 months ago·Updated 7 months ago· 8

smthemex/ComfyUI_FlowRVS

Nodes6

On cloudLocal install

CategoryFlowRVS_SM

Stars8

Updated7 months ago

Nodes (6)

FlowRVS_SM_Apply_Mask

Cut the segmented subject out of your video — or erase it onto the background

FlowRVS_SM

FlowRVS_SM_Cond

Your video and prompt become a latent here — that's the whole trick

FlowRVS_SM

FlowRVS_SM_Decoder

Blobby mask latents in, clean per-frame masks out — meet the cleanup crew

FlowRVS_SM

FlowRVS_SM_KSampler

A 'sampler' that runs one step by default — and usually should stay that way

FlowRVS_SM

FlowRVS_SM_Model

A Wan 2.1 1.3B that was fine-tuned to find objects

FlowRVS_SM

FlowRVS_SM_VAE

Two VAEs in a trench coat — and why this pack needs both of them

FlowRVS_SM

Readme

ComfyUI_FlowRVS

FlowRVS，Deforming Videos to Masks: Flow Matching for Referring Video Segmentation，you can use it in comfyUI when use this node.

Update

support new checkpoints,and fix some bugs
支持新的模型，调整emb长度，调整解码和加载模型的代码，修复归一代码，方法在fps表现更佳

1. Installation

In the ./ComfyUI/custom_nodes directory, run the following:

git clone https://github.com/smthemex/ComfyUI_FlowRVS.git

2. Requirements

pip install -r requirements.txt

3.Model

3.1 FlowRVS dit and vae from here / 官方底模和vae
3.2 ‘diffuser vae’ and transformer download here / 暂不兼容comfyorg的vae
3.3 T5 download here / comfy官方T5 FP8

--  ComfyUI/models
    |-- vae
        |-- diffusion_pytorch_model.safetensors  # rename diffsuers vae   最好重命名
    |-- clip
        |-- umt5_xxl_fp8_e4m3fn_scaled.safetensors  #  or umt5_xxl_fp16.safetensors
    |-- diffusion_models
        |--  diffusion_pytorch_model.safetensors  #   rename diffsuers transformers   最好重命名
    |-- FlowRVS
        |-- FlowRVS_dit_mevis.pth #2.64G # new checkpoints
        |--tuned_vae.pth

4.Example

5.Citation

@article{wang2025flowrvs,
  title={Deforming Videos to Masks: Flow Matching for Referring Video Segmentation},
  author={Wang, Zanyi and Jiang, Dengyang and Li, Liuzhuozheng and Dang, Sizhe and Li, Chengzu and Yang, Harry and Dai, Guang and Wang, Mengmeng and Wang, Jingdong},
  journal={arXiv preprint arXiv:2510.06139}, 
  year={2025}
}