Nodes/ComfyUI-SD-CN-Animation/Load FloweR Model
ComfyUI Node

Load FloweR Model

The 8MB brain that predicts what comes next in SD-CN Txt2Vid

By pxl-pshr·Created 5 months ago·Updated 5 months ago· 13
Load FloweR Model
    • flower_model
    model_name

    Load FloweR Model is the feeder node for the SD-CN Animation Txt2Vid pipeline. "FloweR" is the optical flow reconstruction model - the thing that watches the previous four frames of your animation and predicts optical flow, an occlusion mask, and an estimate of the next frame. Without it, SDCNTxt2Vid has no idea where the scene is heading and you're back to per-frame img2img flicker. It's a small, single-purpose loader: pick the weights, get a flower_model wire, plug it into Txt2Vid.

    The pack's README calls the model "~8MB" and that's basically the whole story on size - this is a tiny U-Net, not a diffusion model, which is why the whole Txt2Vid approach is feasible on a consumer card. It's a port of the model from volotat's original 2023 SD-CN-Animation project (MIT licensed), so it's been around for a while and is well settled.

    Inputs and outputs

    There's exactly one input, a model_name dropdown. Don't expect a list of options to browse - at the moment it offers a single choice: FloweR_0.1.2.pth. You pick it because it's the only one, and you don't have to hunt for it.

    The single output is the flower_model wire, a dict containing the loaded state dict. Its only real consumer is SDCNTxt2Vid (the flower_model input), though FloweRPredict - the pack's standalone debug node - takes the same wire if you want to inspect what the model is seeing.

    Install and the auto-download

    The model lives in ComfyUI/models/FloweR/, and here's the part that surprises people in a good way: you don't download it. On first launch, the pack checks that folder and, if the weights are missing, pulls FloweR_0.1.2.pth (~8MB) automatically from the Hugging Face repo pxlpshr/ComfyUI-SD-CN-Animation.

    cd ComfyUI/custom_nodes
    git clone https://github.com/pxl-pshr/ComfyUI-SD-CN-Animation
    

    (or install via ComfyUI Manager, search "ComfyUI-SD-CN-Animation"), then restart. If the auto-download fails - no network, Hugging Face unreachable, that sort of thing - the pack logs an error telling you to grab the file manually and drop it into ComfyUI/models/FloweR/.

    Gotchas

    If the dropdown ever comes up empty, the file isn't in models/FloweR/ - check there first before re-downloading anything. And remember the resolution constraint: FloweR operates on dimensions that are multiples of 128, so Txt2Vid internally rounds your width/height down to the nearest multiple. 512x512 and 1024x1024 are happy; weird sizes like 690x690 just get quietly floored. Nothing to fix, but it explains why you shouldn't bother trying to squeeze the flow prediction out of oddball resolutions.

    CategorySD-CN-Animation

    Inputs (1)

    NameTypeDefaultDescription
    model_nameCOMBO1 options: FloweR_0.1.2.pth

    Outputs (1)

    NameTypeDescription
    flower_modelFLOWER_MODEL