Nodes/ComfyUI-Lotus/Lotus Sampler
ComfyUI Node

Lotus Sampler

Run the depth/normal model, no step count to argue with

By kijai·Created 2 years ago·Updated 9 months ago· 168
Lotus Sampler
  • lotus_unet
  • samples
  • samples
seed123
per_batch4
keep_model_loadedfalse

This is where the actual depth or normal map gets produced. Load Lotus Model gets you a checkpoint in VRAM; Lotus Sampler is what you run it through to get a usable map out. If you've used a KSampler before, the shape will feel familiar, but there's a deliberate gap where you'd expect a steps input - and that gap is basically the whole point of Lotus.

Why there's no step count

Regular diffusion sampling - the kind Marigold and most SD-based depth estimators use - iteratively denoises over many steps, sometimes ensembling several runs to smooth out variance, because that's how a diffusion model was trained to generate. Lotus is trained differently: it reformulates the same task as a single deterministic prediction, so one forward pass through the UNet is enough. That's the entire reason people reach for Lotus over Marigold in the first place - it gets diffusion-model quality without the wait, landing well under a minute per image in community benchmarking versus 50+ seconds for the heavier multi-step estimators. There's nothing to tune here because there's no iterative process to tune.

The inputs that matter

  • lotus_unet (LOTUSUNET) - plug in the output from Load Lotus Model. Not interchangeable with a regular MODEL socket.
  • samples (LATENT) - your input image, encoded to latent space (VAE Encode it first, same as any other latent-space node). This is what gets turned into a depth or normal map, depending on which checkpoint you loaded upstream.
  • seed (default 123) - standard reproducibility control. Because the underlying prediction is deterministic, don't expect the wild seed-to-seed variety you'd see re-rolling a text-to-image generation; it's there mostly so re-runs are exactly repeatable.
  • per_batch (default 4, max 4096) - how many latents get pushed through in one forward call. If you're running this over a batch of frames (video depth, say) and you OOM, this is the first dial to turn down. If you've got VRAM to spare and you're processing a big batch, raising it trades memory for fewer passes.
  • keep_model_loaded (default false) - the standard toggle kijai puts on most of his wrapper nodes: leave it off if this is a one-shot run and you want VRAM back afterward, flip it on if you're running Lotus repeatedly in the same session (batch processing, an interactive workflow) and don't want to pay the load cost every time.

The output is a single samples (LATENT) - you'll need a VAE Decode after this to turn it back into a viewable image. From there it goes wherever a depth or normal map goes in your workflow: into a ControlNet's image input for conditioning a new generation, or straight out as a texture/lighting asset if that's what you're after.

Installing it

Same pack as Load Lotus Model, so if you've got that node you've got this one:

  • ComfyUI Manager - search "ComfyUI-Lotus," install, restart.
  • Manual - cd ComfyUI/custom_nodes && git clone https://github.com/kijai/ComfyUI-Lotus, restart.

You'll also need the actual checkpoint files from Kijai/lotus-comfyui sitting in ComfyUI/models/diffusion_models - the README's only real instruction, and this node doesn't run without a valid lotus_unet feeding it.

Common issues

If your output looks like noise or doesn't resemble a depth/normal map at all, the most likely cause isn't this node - it's the checkpoint loaded upstream. Loading a normal-prediction checkpoint but expecting depth output (or the reverse) will run without error and just hand you the wrong kind of map, since this node doesn't know or care what task the weights were trained for.

Watch VRAM if you're batching: per_batch is your main lever here, and it's worth remembering this node processes everything in one pass rather than iterating, so a batch that's too big for your card will fail fast rather than slowly - there's no partial progress to lose, so just drop per_batch and retry.

One thing not to expect: variation between runs. If you're used to depth estimators like Marigold where ensembling and multiple steps introduce some run-to-run smoothing, Lotus's single deterministic pass means the same input and checkpoint will give you close to the same output every time, seed or no. That's a feature, not a bug - it's what makes this fast in the first place - but if you're chasing a "better roll" the way you might with a generation sampler, that's not really available here. Change the checkpoint or the input image instead.

CategoryComfyUI-Lotus

Inputs (5)

NameTypeDefaultDescription
lotus_unetLOTUSUNET
samplesLATENT
seedINT1230–4294967296
per_batchINT41–4096
keep_model_loadedBOOLEANfalse

Outputs (1)

NameTypeDescription
samplesLATENT