VACE Annotator - Scribble

The most annoying part of a VACE control workflow is the annotator. You want scribble maps of your video frames, and the "proper" way means downloading yet another checkpoint from HuggingFace and hoping it lands in the right folder. This node from ComfyUI-SwissArmyKnife runs the real VACE scribble models when they're present - and, the genuinely useful bit, it doesn't break if you haven't bothered. auto mode silently falls back to a Sobel edge pass and you get a workable control map out of nothing but the frame itself.

Why scribble maps at all? VACE is how Wan does per-frame control: you extract a control map from each frame and feed it to the model alongside the prompt. Scribble is the loose, hand-drawn-looking flavor of that - in ControlNet terms it's the most forgiving preprocessor, the one you reach for when you want structure without a hard outline, and it plays nicely with anime-style output. So the picture is: frames → this node → scribble_maps → a VACE/Van conditioner → your video model.

How it works

The node is a preprocessor with three backends, chosen automatically. With a checkpoint present it runs the vendored VACE ContourInference generator (netG_A_latest.pth for anime, scribble.pth for general, sketch.pth for sketch - all from ali-vilab/VACE-Annotators). Legacy ResNet-style checkpoints get the older quantile-threshold path. If no checkpoint exists, it falls back to a grayscale Sobel filter: normalized per frame, thresholded, inverted to white-on-black scribble. No model, no download, still a usable map.

Models are found by scanning models/vace_annotators/scribble/... under your ComfyUI base directory, then relative paths, then ~/ComfyUI/models/.... They're cached after first load, and model_path lets you point at a file explicitly.

Inputs and output

images (required, IMAGE) takes your frames as a batch - plug in the frames coming out of your loader or a VHS-style video decode. The rest are settings, and you'll touch three:

• style (anime / general / sketch) - which checkpoint flavor to load. Defaults to anime.
• inference_mode (auto / model / fallback) - auto prefers the model and falls back to Sobel; model requires the checkpoint and errors if it's missing; fallback always uses Sobel. Leave it on auto.
• resolution (64–2048, default 512) - the square working resolution. Lower is faster and lighter on VRAM; the node resizes back to your original frame size afterward, so output shape stays stable.
• model_path (optional) - skip default discovery and point at a checkpoint directly.

Output is a single scribble_maps IMAGE, same height and width as your input batch, ready to wire into a VACE conditioner.

Installing it

It ships inside ComfyUI-SwissArmyKnife, so install the pack, not a separate node:

cd ComfyUI/custom_nodes
git clone https://github.com/sammykumar/ComfyUI-SwissArmyKnife.git
pip install -r ComfyUI-SwissArmyKnife/requirements.txt

Or just search "ComfyUI-SwissArmyKnife" in ComfyUI Manager. The pack pulls in opencv-python, pillow, and friends - that's the heaviest part of the install, and FFmpeg is listed as a system requirement for the video side generally. Checkpoints are optional unless you set inference_mode to model; if you do want the real VACE models, drop them under models/vace_annotators/scribble/<style>/ and you're done.

Where people get burned

Mostly at the resolution dial. It's a working resolution, not the output size - crank it to 2048 on a long video batch and you'll watch your VRAM disappear for no visible gain; 512 is the sane starting point. And don't set model mode until you've actually downloaded the checkpoints, or you'll get a hard error pointing at the HuggingFace repo. If you're on auto and your maps look too clean or too noisy, that's the Sobel fallback doing its thing - the fix is the model, not the slider.