Load Step3-VL Model

This is the loader you pick when captions need to be right, not fast. Load Step3-VL Model brings up stepfun-ai/Step3-VL-10B - StepFun's 10-billion-parameter vision-language model, the frontier-grade member of this pack - plus a huihui-ai abliterated variant. It's the best describer of the three families in mamorett/orama, and it's also the one with the honest hardware tax: bf16 weights, ~20 GB VRAM minimum, 24 GB recommended. If your card is 12–16 GB, this node will hurt. There's no way around it via this loader, because unlike the other two families it has no quantization dropdown - bf16 is the only dtype, full stop.

How it works, including the messy parts

The download-and-cache plumbing is shared with the rest of the pack (weights land in ComfyUI/models/vision_models/stepfun-ai/..., real copies, reused across runs). Then things get interesting. Step3-VL is a custom architecture, so the loader:

• calls AutoProcessor and AutoModelForCausalLM with trust_remote_code=True - mandatory here, because the model ships its own code, and you should know that means arbitrary model code executes when it loads;
• applies a key_mapping dict at load time to rename the checkpoint keys to match the registered architecture;
• forces torch_dtype=bfloat16 (the only officially supported dtype, and the reason there's no dtype input on this node);
• and then monkey-patches the model's own internals: it overrides prepare_inputs_for_generation to bypass a buggy override in the original code, and wraps forward to return past_key_values properly - fixing a KV-cache bug that caused infinite-generation loops.

That last bit is why the README points at HF Discussion #9 for the infinite-generation problem: this pack is shipping a workaround for a bug in the model itself. It tells you how new this integration is, and how much the author cares about making it actually work.

The inputs

• model_id - two choices: stepfun-ai/Step3-VL-10B or huihui-ai/Huihui-Step3-VL-10B-abliterated.
• device_map - auto, cuda, or cpu. Note: no mps here (unlike the smaller loaders), which is sensible - bf16 on Apple Silicon is a bad time.
• use_flash_attention_2 - speeds up inference if your GPU supports it, but you must install the flash-attn package yourself; this pack won't.

Output is a single STEP3_VL_MODEL socket for the Step3-VL (StepFun) inference node.

A real limitation to plan around

StepFun does publish an FP8 quantized checkpoint (Step3-VL-10B-FP8) for consumer cards, but it is not in this loader's dropdown. So if you were hoping to squeeze this onto 16 GB via FP8, you can't do it here - not through this node. If you're below ~20 GB of free VRAM, either wait for the pack to add quantization support or stick with the 1.6B LFM / 1B MiniCPM loaders.

Installing it

cd ComfyUI/custom_nodes
git clone https://github.com/mamorett/orama
pip install -r ComfyUI/custom_nodes/orama/requirements.txt

Restart ComfyUI, or search orama in ComfyUI Manager. The pack's requirements pin transformers>=5.7.0, which satisfies Step3-VL's needs too - update transformers before installing if your venv is stale. First load downloads a 10B model's worth of weights, so give the queue time and watch the console rather than assuming it hung.