Qwen 2 Causal LM Loader

Qwen 2 Causal LM Loader is the node that makes the whole comfyui-lance-aio pack possible. Lance's language model - a Qwen2-based MoE with 3B active parameters of a roughly 9B total - is the biggest single weight block in the model, and on paper it shouldn't fit on a mid-range card. This loader is where the pack's low-VRAM magic actually happens.

The story behind it is worth knowing. When ByteDance released Lance-3B in May 2026, the release thread filled up fast with people staring at the FP32 checkpoint sizes and declaring it needed 40GB of VRAM. The correction came just as fast: with offloading and weight streaming, you don't need the whole thing resident. This port is that correction in node form. The author explicitly built it for limited-VRAM machines - tested on 12GB VRAM plus 32GB of system RAM, with image tasks claimed to work down around 8GB.

What goes in

• ckpt_dir - path to the Lance checkpoint folder (wire it from Lance Args' LANCE_CKPT_DIR). The loader reads model.safetensors and, importantly, llm_config.json from that folder - that config file sizes the language model, so don't delete it.
• model_args / inference_args - config from Lance Args. They set things like QK-norm, tied vs. untied embeddings, and the position-embedding mode.
• low_memory (default false) - the one knob that matters. Off, the model initializes in bfloat16 directly. On, it initializes on the "meta" device and loads weights with assignment, which is the streaming/low-memory path. If you're under ~16GB VRAM and hitting OOM, flip this.

What comes out

A single QWEN_2_CAUSAL_LM output, feeding the Lance Loader. It's not a raw MODEL - it's a bundle holding the patched language model plus its own ComfyUI model patcher, which is what ComfyUI uses to load and unload it on demand.

How it works

Two things make it special. First, every linear and embedding layer gets swapped to comfy.ops.manual_cast, ComfyUI's operator set that casts on the fly and lets weights live off-device. Second, the model is wrapped in a CoreModelPatcher set to bfloat16 compute with an offload device - so ComfyUI's own model management decides what's on the GPU at any moment instead of keeping a 20GB LLM resident. The MoE experts also get initialized from the shared checkpoint (the init_moe step), which is part of how a 9B-total model behaves like a 3B-active one. The official codebase's Qwen2-NaViT implementation is vendored in the pack rather than pulled from transformers, which is why the transformers version pin matters.

Installing the pack

cd ComfyUI/custom_nodes
git clone https://github.com/SteveImmanuel/comfyui-lance-aio
cd comfyui-lance-aio
pip install -r requirements.txt

or find comfyui-lance-aio in ComfyUI Manager. The transformers>=4.50.3,<5 pin is critical - the vendored code is a fork of a transformers-4-era implementation. Optional but recommended: install flash-attn for speed, or the loader falls back to PyTorch's SDPA with equivalent results at slower speeds. Then drop the Lance checkpoints under ComfyUI/models/lance/ and restart.

Where people get burned

The big one is OOM on the first run and assuming the pack is broken. If you're on a 12GB-class card, turn low_memory on and give it time - the first load streams a large file. The other classic is a transformers 5 upgrade somewhere in the environment (another pack's installer pulling a newer version, say), which produces obscure errors in this loader. Re-pin transformers<5 and restart. And if you see a state-dict mismatch warning scroll by here, it usually means the checkpoint folder doesn't match the model config - check llm_config.json is the one that shipped with the weights.