DiffSynth: SenseNova U1 Image Loader

The Loader is where this pack hides its complexity. There are no sampler settings here, no prompt box - just three sockets that decide whether the next few minutes of your life go smoothly, and one output you plug into the SenseNova inference node.

Inputs

Required:

• torch_dtype - bfloat16 (default), float16, float32. On an 8B unified model, bfloat16 is the right answer unless you're on an old card that only speaks fp16.
• device - cuda or cpu.
• model_configs - a merged list from Merge ModelConfigs. Required, and this is where your weights come from.

Optional:

• tokenizer_config - one ModelConfig for the tokenizer. In practice you want this: with no tokenizer the pipeline can't turn your prompt into the token sequence the multimodal DiT expects.
• vram_limit - a number in GB from a VRAM Limit node, capping how much the pipeline is allowed to hold.

Output is pipe, single socket, into DiffSynth: SenseNova U1 Image Inference.

The two ModelConfigs

Both point at the same repo, SenseNova/SenseNova-U1.5-8B-MoT on ModelScope:

1. model*.safetensors - the transformer weights themselves.
2. ./ - the entire directory.

That second one looks like nothing and it isn't. In DiffSynth-Studio, a ./ pattern resolves to the whole model folder rather than a glob match, so it's the workaround for models whose support files (configs, tokenizer assets, whatever the pipeline reads at encode time) aren't a single named file. Most ModelConfig nodes in this pack are one-file-one-node; this pipeline deliberately isn't. The example workflow splits it exactly this way, and the second node feeds tokenizer_config.

Where the files go: models/<model_id>/ under the directory you launched ComfyUI from, unless you set DIFFSYNTH_MODEL_BASE_PATH first. Downloads trigger on the ModelConfig nodes rather than here, so the first queue is the slow one - and it needs the disk space up front, because an 8B checkpoint plus companion files is not small.

Install

Manager → search DiffSynth-ComfyUI → install → restart. Manual:

pip install git+https://github.com/modelscope/DiffSynth-Studio.git
cd ComfyUI/custom_nodes && git clone https://github.com/modelscope/DiffSynth-ComfyUI.git

The plugin declares exactly one dependency, diffsynth>=2.1.9. If you want quantization options inside ModelConfig, add the extras:

pip install "diffsynth[quant] @ git+https://github.com/modelscope/DiffSynth-Studio.git"

Memory strategy

Here's the thing worth internalising about this whole pack: the Loader does not choose how VRAM is used. Each ModelConfig carries its own memory plan, set by whichever VRAM Config node you wired into it, and the vram_limit you hand the Loader is the ceiling. Defaults are conservative - weights parked in host RAM and moved to the GPU only when needed - which is why DiffSynth pipelines so often feel slower than a native ComfyUI equivalent but much harder to OOM. If your card has room, push computation_device/computation_dtype to something more aggressive in the VRAM Config and leave the limit generous.

One practical note: 2048² with 50 steps holds a lot of activations. A VRAM Limit with buffer_size around 1 GB keeps ComfyUI's cache from being evicted mid-run by the pipeline.

Common issues

• Prompt encoding errors on the first run. The ./ ModelConfig isn't connected, or its files aren't all present yet.
• Missing sockets after an upgrade. Loader inputs are generated from your installed diffsynth at startup. Restart ComfyUI after pip install -U.
• "It loaded but generation is instant garbage." You probably left device on cpu while expecting GPU semantics, or the dtype no longer matches the checkpoint. Put it back to cuda and bfloat16.
• Host RAM, not VRAM, is the wall. Offload strategies trade GPU memory for system memory. 32GB of RAM is a reasonable floor for this pipeline.