Tora Encode OpticalFlow

This is the other way to drive Tora. Where ToraEncodeTrajectory has you hand-draw a path, this node takes a dense optical-flow field - the actual per-pixel motion pulled from a real video - and encodes it into trajectory features CogVideoX can follow. Instead of one line saying "move here," you're handing the model the full motion of a reference clip. Point A goes there, everything else moves the way the source moved. It's motion transfer, not motion sketching.

How it works

Optical flow is the map of how every pixel shifts from one frame to the next. Extract it from a source video and you've captured its motion, stripped of appearance. This node's job is to turn that flow map into Tora's conditioning: it runs the flow field through the VAE and the Tora model, producing trajectory features the sampler uses to bias the generation toward that same movement. So you can take the camera push and subject drift from a clip you like and re-cast it onto a completely different prompt or start image. That's the appeal - dense, realistic motion you didn't have to draw by hand.

The trade against the trajectory node is control versus fidelity. A hand-drawn path is precise but coarse; an optical-flow field is rich and organic but only as good as the source and the flow extraction. Pick this node when you have real footage whose motion you want to borrow.

The inputs that matter

• optical_flow (an IMAGE) - the flow field itself, as an image sequence. This is the whole input, and it's the part that trips people up: it's not your source video. It's the flow derived from that video by an optical-flow estimator upstream, encoded as RGB. You generate it with a flow node, then feed the result here.
• vae and tora_model - the standard pair. The VAE encodes the flow into latent space; the tora_model comes straight from DownloadAndLoadToraModel. Both required, nothing to tune.
• strength (default 1) - how strongly the model is pushed to follow the extracted motion. 1 is the honest starting point. Dial it down if the transferred motion is fighting the scene and producing artifacts.
• start_percent (0) / end_percent (1) - the portion of the denoising schedule the flow guidance is active over. Defaults apply it throughout. Shortening the window lets the flow set up motion early and then loosen its grip, which can soften an over-literal transfer.

The single output is tora_trajectory (type TORAFEATURES), which goes into the tora_trajectory input on the CogVideo Sampler - exactly the same slot ToraEncodeTrajectory feeds. The two are interchangeable at the sampler; they just differ in how you author the motion.

How to install it

It ships with the CogVideoX wrapper - no separate install:

• ComfyUI Manager - search ComfyUI-CogVideoXWrapper, install, restart.
• Manual - cd ComfyUI/custom_nodes && git clone https://github.com/kijai/ComfyUI-CogVideoXWrapper, then pip install -r ComfyUI-CogVideoXWrapper/requirements.txt, restart.

You'll also need the Tora weights loaded via DownloadAndLoadToraModel, and an optical-flow source to feed the optical_flow input.

Common issues & troubleshooting

"It expects an image and I gave it my video." The most common mix-up. The optical_flow input wants a flow field, not raw footage. Run your source through an optical-flow estimator first, then connect that output here. Plain frames won't encode into meaningful motion.

The transferred motion is too aggressive or warps the scene. Turn strength down from 1, or narrow the start_percent/end_percent window so the flow guides less of the render. Like all Tora conditioning, this is a strong suggestion, and pushing it too hard makes motion look mechanical or breaks the image.

Results are inconsistent. Tora is genuinely hit-or-miss - early testers were blunt that some combinations look great and others come out strange or barely move. A clean, unambiguous source flow helps a lot; noisy or low-quality flow extraction gives the model bad instructions to follow.

Out of memory. CogVideoX-5b with Tora is heavy before you add flow encoding. Keep resolution and frame count modest, and lean on fp8 plus offloading at the model loader rather than expecting savings here.