MuseVRunVid2Vid

MuseVRunVid2Vid is the sibling of MuseVRun in the same pack, and the "Vid2Vid" name is a bit of a lie. It's not video-to-video in the img2img sense of restyling a clip. It's pose transfer: you give it a portrait of your character and a video of someone moving, and it re-performs that movement with your character's face and body. Think "make my OC do this dancer's routine," or drive an avatar from any reference motion you have lying around. That's the killer use case for the whole MuseV family.

How it works

The video_path you feed it is read as a pose video, not a raw source clip. In the shipped workflow (wf_vid2vid.json) the plumbing makes this explicit:

VHS_LoadVideo → DWPreprocessor → VHS_VideoCombine → VHS_FILENAMES_STRING_MuseV → video_path (STRING) → MuseVRunVid2Vid

Your source video gets run through DWPose to strip it down to a pose skeleton, the skeleton video is saved, and that saved path is what the node receives. Then MuseV does its two-phase thing: redraw your ref_image into a clean first frame, and run parallel denoising with the reference net + IP-Adapter locking in your character's appearance while the pose sequence drives the motion. Because which2video defaults to "video", the source video also guides the initial noise - a light img2img-style anchor on top of the pose.

The frames are read via Decord in video_len-sized chunks (drop_last=True), so your source should be long enough to fill whole chunks - a 12-frame default chunk from a clip shorter than that gets silently dropped.

Inputs that matter

Everything from MuseVRun applies - ref_image, prompt, video_len, seed, width/height, the two step/guidance pairs, sd_model_name, uselcm - plus one new one:

• video_path (STRING) - the path to the pose video that drives the motion. Wire it from VHS_FILENAMES_STRING_MuseV like the shipped workflow does rather than typing a path, so it survives moving your ComfyUI install.

Watch the default value: it ships as .../data/source_video/video1_girl_poseseq.mp4 with a trailing space, and the data/source_video/ folder is an empty placeholder in the repo. If you run it untouched it will fail to find a file. Set the path yourself or build it from the VHS node.

Output is the same IMAGE frame batch as MuseVRun - feed it into VHS_VideoCombine to get your mp4.

Install

Same pack, same story as MuseVRun: install "ComfyUI-MuseV" via ComfyUI Manager, or git clone --recursive and pip install -r requirements.txt inside ComfyUI/custom_nodes. You additionally need VideoHelperSuite (for VHS_LoadVideo / VHS_VideoCombine / VHS_FILENAMES_STRING_MuseV) and the DWPreprocessor node pack for pose extraction. The OpenMMLab stack (openmim, mmengine, mmcv==2.0.1, mmdet==3.1.0, mmpose==1.1.0) is effectively mandatory - the node import depends on it. And the multi-gigabyte TMElyralab/MuseV download into ComfyUI/models/diffusers/TMElyralab/MuseV:

huggingface-cli download --resume-download TMElyralab/MuseV --local-dir ComfyUI/models/diffusers/TMElyralab/MuseV

Common issues

• File not found on first run - the trailing-space default path. Replace it.
• Pose video isn't generated / DWPreprocessor errors - the MM packages aren't installed, or the model download is incomplete.
• Nothing happens with a short source clip - chunks shorter than video_len are dropped (drop_last=True). Use a longer source or lower video_len.
• Subject drifts from the ref image - this is the hard truth of pose transfer: appearance comes from your portrait, but complex poses expose where the SD 1.5 base falls short. Stick to human motions, keep video_guidance_scale near 3.5, and expect the best results from subtle, camera-stable movements.

It's a 2024 tool, and the polish shows it. But nothing else lets you swap a character into an arbitrary human motion with this little setup cost, and the DWPose-driven pipeline is a clean template to steal even if you move on to newer models.