Nodes/ComfyUI-DragAnything/DragAnythingLoader
ComfyUI Node

DragAnythingLoader

Load the DragAnything + SVD pipeline once instead of paying for it every run

By chaojie·Created 2 years ago·Updated 2 years ago· 72
DragAnythingLoader
    • pipeline
    svd_pathstable-video-diffusion-img2vid
    draganything_pathDragAnything

    If you've heard "DragAnything" at all, it's the research release from ShowLab that let you pick an object in an image and drag it through a video - motion control by drawing a path. The core trick: it glues a motion-control module onto Stable Video Diffusion's image-to-video base, then tells SVD which pixels are the thing to move and where they should go each frame. DragAnythingLoader is the node that builds that whole contraption in memory, once, so you're not rebuilding it on every single queue.

    It's the load-once node in the pack's two-node split. It takes just two inputs - both enum dropdowns populated by what's sitting in the pack's pretrained_models/ folder - and hands you a DragAnythingPipeline on its single output. That pipeline is what you feed into DragAnythingPipelineRun or DragAnythingPipelineRunRandom.

    The two inputs that matter

    Both inputs are the same shape: pick a folder name from the dropdown.

    • svd_path (default stable-video-diffusion-img2vid) - the base image-to-video model.
    • draganything_path (default DragAnything) - the motion-control weights.

    Notice something important: if those folders aren't in pretrained_models/, the dropdowns are empty. The inputs are literally os.listdir() of that directory. Nothing downloaded = nothing to select. This trips up more people than any parameter.

    Under the hood it loads the DragAnything control module, loads SVD's UNet, and wires them into a custom StableVideoDiffusionPipeline, then calls enable_model_cpu_offload(). That last call is why it fits in reasonable VRAM (SVD is heavy on its own; the control module piles on more) at the cost of slower first steps while weights shuffle between RAM and VRAM.

    Why bother with the loader at all?

    The pack gives you a shortcut: DragAnythingRun does everything in one node, loading the pipeline internally each execution. If you're running one clip, that's fine. The moment you're iterating - tweaking a trajectory, bumping controlnet_cond_scale, hunting for the right motion_bucket_id - DragAnythingRun re-loads and re-offloads the whole pipeline every time, and you watch minutes of disk-spinner instead of generating video. The loader exists precisely so you load once and queue many.

    Install

    cd ComfyUI/custom_nodes
    git clone https://github.com/chaojie/ComfyUI-DragAnything.git
    cd ComfyUI-DragAnything
    pip install -r requirements.txt
    

    or install "ComfyUI-DragAnything" via ComfyUI Manager and restart. Then the real work: three git-lfs model downloads into pretrained_models/:

    cd ComfyUI/custom_nodes/ComfyUI-DragAnything/pretrained_models
    git-lfs clone https://huggingface.co/windwhinny/chilloutmix.git
    git-lfs clone https://huggingface.co/weijiawu/DragAnything
    git-lfs clone https://huggingface.co/stabilityai/stable-video-diffusion-img2vid
    

    The README is blunt that you must pin diffusers==0.19.3 (the requirements.txt does it; newer diffusers breaks this pipeline). If the dropdowns are empty after all that, restart ComfyUI and check the folder names - the defaults assume those exact directory names.

    CategoryDragAnything

    Inputs (2)

    NameTypeDefaultDescription
    svd_pathCOMBOstable-video-diffusion-img2vid1 options: pretrained_models.txt
    draganything_pathCOMBODragAnything1 options: pretrained_models.txt

    Outputs (1)

    NameTypeDescription
    pipelineDragAnythingPipeline