DiffSynth: LingBot Video Loader

LingBot-Video is the odd one out in the local video stack. Robbyant - Ant Group's embodied-AI company - built it as a physics-first Mixture-of-Experts video model for roboticists, not as a cinematic generator, and the community's verdict on it was: interesting, genuinely open, second to Wan on looks. Its one non-negotiable quirk is that it expects a structured JSON caption rather than the loose prose you'd write for Wan. Most people will not write that by hand.

That's exactly why this loader has a processor_config socket on it.

Required inputs

• model_configs - the merged list. The pack's template wires three files: the transformer and VAE from Robbyant/lingbot-video-dense-1.3b, plus a Qwen/Qwen3-VL-4B-Instruct ModelConfig for the prompt rewriter.
• torch_dtype - bfloat16 by default, also float16 and float32.
• device - cuda, or cpu if you enjoy waiting.

Optional inputs that decide whether it works well

processor_config is the one that matters. In the template it points at Qwen/Qwen3-VL-4B-Instruct - Alibaba's vision-language model, which in this stack does double duty as the community's captioner of choice and as a text/prompt encoder for newer models. Here it's the thing that turns your plain-English prompt into the structured caption LingBot actually wants. Skip it and you're on your own with the JSON.

vram_limit takes a VRAM Limit node's output so you can reserve VRAM for the rest of your desktop.

Which LingBot this runs

Worth being precise, because the announcements blurred it: LingBot-Video proper is a 30B-A3B MoE, and there's a separate 1.3B dense variant plus a Qwen-based prompt rewriter. The template here uses lingbot-video-dense-1.3b - the small one. The MoE's economics don't help on a single consumer card anyway, since the full expert pool stays resident regardless of how few experts each token touches.

Also worth knowing before you spend an evening on it: LingBot's VAE is the Wan 2.1 VAE, and that's the only Wan component in it. It's scaled from scratch, not a Wan finetune, despite what a lot of the framing assumes.

Install

ComfyUI Manager, search DiffSynth-ComfyUI, install, restart. Manual:

pip install git+https://github.com/modelscope/DiffSynth-Studio.git
cd ComfyUI/custom_nodes
git clone https://github.com/modelscope/DiffSynth-ComfyUI.git

Nothing here lands in ComfyUI/models/. The weights arrive through the ModelConfig nodes into DiffSynth's own cache, on first execution.

Where people get burned

Skipping the processor and getting mush. LingBot's prompt expectations are specific. Wire the rewriter, then iterate on the inference node's prompt.

Judging it against Wan and being disappointed. Physics behavior - fluids, sand, reflections - is what it was built for. If you want a cinematic five-second hero shot, Wan is the better tool and everybody including LingBot's own testers says so.

Forgetting the extra ModelConfig. Two files come from the model repo; a third comes from the Qwen repo. Miss either and the load fails after the downloads are already done.