OmniSVG_Loader

OmniSVG is a research model that treats SVG generation as a language problem: instead of a diffusion model painting pixels, it's a vision-language model built on Qwen2.5-VL that's been fine-tuned to emit SVG code token by token. The Loader is the node that drags all of that into VRAM. It doesn't draw anything itself - that's the Sampler's job - but it's where people get stuck, because it's the one node that wants three different model files living in three different ComfyUI folders.

It works by wrapping Qwen2.5_VLForConditionalGeneration in a so-called SketchDecoder and resizing the embedding table to ~197,000 tokens. Those extra tokens are the trick: the model's vocabulary gains special entries for SVG coordinates, colors, and mask/end-of-mask markers, so the VLM can output an SVG as a token sequence instead of natural language. The OmniSVG fine-tuned weights (a .bin file you download separately) get loaded on top of that backbone.

You get two ways to supply the Qwen2.5-VL backbone. Either give qwen_repo a full HuggingFace repo ID or a local folder with all the Qwen2.5-VL files, or - the "I don't want to download a whole repo" path - point qwen_dit at the monolithic qwen_2.5_vl_7b.safetensors in your models/clip folder. Then the transfromer combo selects the OmniSVG weights from models/diffusion_models.

Here's the trap, and the author warns about it in the README: the node decides whether you're running the 4B or 8B config by reading the filename of the file you pick in transfromer. It's literally "4B" if "4b" in transfromer.lower() else "8B". If your weights file doesn't carry a "4b" or "8B" label, you get 8B config whether you meant to or not - and the model size drives a whole different token layout in config.yaml. Keep the label in the filename.

The inputs that actually matter:

• transfromer (yes, that's the real spelling - a typo that shipped): the OmniSVG .bin in diffusion_models. Filename must keep the 4b/8B marker.
• qwen_repo: HF ID or local path to Qwen2.5-VL-3B-Instruct (pair it with 4B weights) or Qwen2.5-VL-7B-Instruct (8B). Defaults to the 3B repo.
• qwen_dit: the optional monolithic visual-encoder .safetensors from the clip folder. Leave it at "none" if you're using the full repo.
• use_accelerate: leave it off. The author's own update notes say the accelerate path on 12GB cards hits meta-data mismatches and performs worse than ComfyUI's dynamic memory handling.
• attn: sdpa is the sane default; flash_attention_2 if you've got the build for it.

Output is a single model wire that plugs straight into the OmniSVG_Sampler's model input.

Install. ComfyUI Manager (search "ComfyUI_OmniSVG") or:

cd ComfyUI/custom_nodes
git clone https://github.com/smthemex/ComfyUI_OmniSVG
pip install -r requirements.txt

then restart ComfyUI. Dependencies include transformers>=4.51.3, qwen-vl-utils, CairoSVG, pycairo, and moviepy - not tiny, and on Windows Cairo itself is the classic pain point (the README points at a gtk3-runtime installer, with a full vcpkg walkthrough in Installation_Guide.md if that fails). Then three model downloads: the OmniSVG .bin into models/diffusion_models, the Qwen2.5-VL files (local folder or let HF pull them via the repo ID), and optionally qwen_2.5_vl_7b.safetensors into models/clip.

Where people get burned. VRAM is the big one: the 4B model is comfortable on ~12GB cards for inference, and the author flatly says the 7B/8B combo is too slow at 12GB - the HF demo reported something like 17GB. Also: if you hit a numpy 2.x error on Windows, the community fix is pinning numpy==1.26.4. And if the load fails on a newer transformers install, note the loader was updated to handle versions up to 4.57.0, remapping state-dict keys for the newer releases - so don't pin an ancient transformers and assume that's safer.