ComfyUI-AnyTop

ComfyUI-AnyTop

Standalone ComfyUI custom nodes for AnyTop - Universal Motion Generation for Any Skeleton Topology.

Package Structure

This is a self-contained package with no symlinks. All AnyTop inference code is included:

ComfyUI-AnyTop/
├── __init__.py                    # ComfyUI node registration
├── anytop_nodes.py               # Node implementations
├── requirements.txt              # Dependencies
├── README.md
└── anytop/                       # Standalone inference package
    ├── __init__.py
    ├── model/                    # Model architecture
    │   ├── __init__.py
    │   ├── anytop.py
    │   ├── motion_transformer.py
    │   └── conditioners.py
    ├── diffusion/                # Diffusion process
    │   ├── __init__.py
    │   ├── gaussian_diffusion.py
    │   ├── respace.py
    │   ├── nn.py
    │   └── losses.py
    ├── utils/                    # Utility functions
    │   ├── __init__.py
    │   ├── model_util.py
    │   ├── parser_util.py
    │   ├── fixseed.py
    │   └── rotation_conversions.py
    └── data_loaders/             # Data processing
        ├── __init__.py
        ├── tensors.py
        └── truebones/
            ├── __init__.py
            ├── data/
            │   ├── __init__.py
            │   └── dataset.py
            └── truebones_utils/
                ├── __init__.py
                ├── motion_process.py
                ├── get_opt.py
                └── param_utils.py

Installation

1. Install dependencies:

pip install -r requirements.txt

2. Install Motion library (for BVH export):

pip install git+https://github.com/inbar-2344/Motion.git

3. Download spacy model (for text processing):

python -m spacy download en_core_web_sm

🎁 Example Assets Included

The assets/ directory contains ready-to-use examples:

Preprocessed Motions (.npy):

• 🐵 Monkey - Attack animation (1.1 MB)
• 🐕 Hound - Attack animation (331 KB)
• 🦅 Ostrich - Attack animation (372 KB)
• 🦂 Scorpion - Slow forward motion (506 KB)

Raw BVH Files:

• 🐔 Chicken - 4 BVH files for preprocessing tests

See assets/README.md for detailed information.

Quick Start

1. Add "(down)Load AnyTop Model" node

   • Select model subset (e.g., "bipeds", "all")
   • Select device (cuda/cpu)
   • Set save directory (default: ./anytop_models)
   • Run once - downloads if needed, then loads!

2. Connect pipeline to "AnyTop Generate Motion" Connect cond_path to "AnyTop Condition Loader"

3. Generate animations!

Nodes

(down)Load AnyTop Model 🆕

All-in-one: Downloads models if needed, then loads them into memory.

Inputs:

• model_subset: Choose from ["all", "bipeds", "quadrupeds", "flying", "millipeds_snakes"]
• device: cuda or cpu
• save_directory: Where to save/find downloads (default: "./anytop_models")
• t5_model_name: T5 model variant (default: t5-base) [optional]
• use_fp16: Use half precision (default: False) [optional]

Outputs:

• pipeline: Loaded model pipeline (ready for generation!)
• cond_path: Path to skeleton conditions file (for Condition Loader)

Behavior:

1. Checks if model exists in save_directory
2. If not found → downloads from Hugging Face automatically
3. Loads model + T5 conditioner into memory
4. Returns ready-to-use pipeline!

Features:

• ✅ Auto-downloads from Hugging Face (inbar2344/AnyTop)
• ✅ Skips download if model already exists
• ✅ Downloads cond.npy, model checkpoint, args.json
• ✅ Loads everything into memory automatically
• ✅ Single node = download + load!

AnyTop Condition Loader

Load skeleton conditioning data from cond.npy file.

Inputs:

• cond_path: Path to cond.npy file
• object_types: Skeleton type names (one per line, e.g., "Flamingo")

Outputs:

• condition: Condition object

AnyTop Generate Motion

Generate motion using the diffusion model.

Inputs:

• pipeline: From Model Loader
• condition: From Condition Loader
• motion_length: Duration in seconds (default: 6.0)
• seed: Random seed
• num_samples: Number of samples to generate (default: 1)
• guidance_scale: Classifier-free guidance scale (default: 1.0)

Outputs:

• motion: Generated motion data

AnyTop Motion Preview

Preview a single frame of the generated motion as a 3D stick figure.

Inputs:

• motion: From Generate Motion
• sample_index: Which sample to preview (default: 0)
• frame_index: Which frame to display (default: 0)

Outputs:

• preview: Image tensor for ComfyUI viewer

AnyTop Export BVH

Export generated motion to BVH file format using inverse kinematics.

Inputs:

• motion: From Generate Motion
• output_path: Path for BVH file (e.g., "output.bvh")
• sample_index: Which sample to export (default: 0)

Outputs:

• bvh_path: Path to exported file

Usage Example

Complete Workflow

1. (down)Load AnyTop Model
   ├─ model_subset: "bipeds"
   ├─ device: "cuda"
   ├─ save_directory: "./anytop_models"
   └─ Outputs: pipeline, cond_path

2. AnyTop Condition Loader
   ├─ cond_path: (from (down)Load node)
   ├─ object_types: "Ostrich" (or "Monkey", "Hound", etc.)
   └─ Output: condition

3. AnyTop Generate Motion
   ├─ pipeline: (from (down)Load node)
   ├─ condition: (from Condition Loader)
   ├─ motion_length: 6.0
   ├─ seed: 42
   ├─ num_samples: 1
   └─ Output: motion

4. AnyTop Motion Preview (to visualize)
   ├─ motion: (from Generate Motion)
   ├─ sample_index: 0
   ├─ frame_index: 0
   └─ Output: preview image

5. AnyTop Export BVH (to save)
   ├─ motion: (from Generate Motion)
   ├─ output_path: "ostrich_motion.bvh"
   ├─ sample_index: 0
   └─ Output: bvh_path

Available Skeletons for Testing

After downloading cond.npy, you can use these skeleton types:

Bipeds: Ostrich, Flamingo, Penguin, Monkey, various humanoids Quadrupeds: Hound, Wolf, Lion, Tiger, Bear, Horse Flying: Parrot, Bat, Eagle, Dragon, Pegasus Millipeds/Snakes: Scorpion, Spider, Centipede, Snake

See included assets/ folder for example motions!

Import Structure

All imports use relative paths for portability:

from .anytop.model import AnyTop, T5Conditioner
from .anytop.diffusion import gaussian_diffusion
from .anytop.utils import fixseed, load_model
from .anytop.data_loaders import truebones_batch_collate

Requirements

• PyTorch >= 2.0.0
• Transformers >= 4.40.0 (for T5)
• NumPy >= 1.24.0
• SciPy >= 1.10.0
• Spacy >= 3.7.0
• Matplotlib >= 3.3.0
• num2words
• Motion library (BVH/IK from github.com/inbar-2344/Motion)

Notes

• Standalone package: No external dependencies on AnyTop repository
• Modular design: Each folder is a proper Python module with __init__.py
• Relative imports: All internal imports use relative paths
• First load may take time due to T5 model download (~1-2GB)
• GPU with at least 8GB VRAM recommended
• Motion length typically limited to 30 seconds based on training data
• T5 model is cached after first load for efficiency

Technical Details

• Uses GRPE (Graph Relative Position Encoding) for skeleton topology
• Supports variable joint counts (up to 143 joints)
• Per-skeleton normalization statistics from cond.npy
• T5-based joint name embeddings for semantic understanding
• Windowed temporal attention for long sequences
• Inverse kinematics for BVH export via Motion library