ComfyUI-SplatKit-World

ComfyUI-SplatKit-World

One image (or one sentence) → a 3D world you can walk through. A step-by-step app inside ComfyUI, built on ComfyUI-SplatKit. Local, open weights only: no partner nodes, no cloud APIs.

photo / prompt ─▶ 360° pano ─▶ MoGe depth ─▶ YOUR drone paths ─▶ WAN fly-throughs ─▶ SphereSfM ─▶ splat ─▶ browser

Requirements

• ComfyUI (tested on v0.37.0, frontend 1.52.7) on Windows or Linux; macOS untested.
• NVIDIA GPU: tested on 96 GB VRAM / 62 GB RAM. Plan for ≥ 24 GB VRAM, ≥ 64 GB RAM and a large page file.
• Disk: ~46 GB of models, plus ~13 GB per world in ComfyUI's output folder.
• Five custom node packs: ComfyUI-SplatKit, ComfyUI-Mickmumpitz-Nodes, ComfyUI-Krea2-Ostris-Edit, ComfyUI_essentials, ComfyUI_UltimateSDUpscale.
• Models: Krea 2 Turbo + Qwen3-VL 4B + one of two 360 LoRAs (panorama), Wan 2.1 I2V 14B + LoRAs (fly-throughs), Real-ESRGAN x2, MoGe (auto-downloaded by SplatKit).
• Spirula Studio, the free splat trainer (one manual download).

Install

→ Step-by-step guide with all links and download commands: INSTALL.md.

In short:

1. Install the five node packs and this pack (ComfyUI-Manager, comfy-cli, or git clone into custom_nodes/).
2. Download the models into ComfyUI/models/ (table and curl commands in INSTALL.md).
3. Unzip Spirula Studio into custom_nodes/ComfyUI-SplatKit-World/bin/.
4. Restart ComfyUI. If anything is missing, the app lists it in a red box.

Use

Click SplatKit World in ComfyUI's top bar (also under the SplatKit menu), or open <your ComfyUI address>/splatkit/world/ (e.g. http://127.0.0.1:8188/splatkit/world/). The left side walks you through it:

| Step | What happens | |---|---| | 01 Input | Drop a photo, describe a place, or bring a 2:1 panorama. | | 02 Panorama | Krea 2 + a 360 LoRA paint the full 360°, and MoGe reads its depth. View 360° shows it any time. | | 03 Camera paths | Your turn. Nothing expensive runs until you approve. Shape each camera's path on the floor and side views and watch it fly through the scene (synced to its preview). Preview camera renders what WAN will see (black = to be painted). Then approve. | | 04 Fly-throughs | SplatKit's dataset workflow: WAN 2.1 paints what the picture never saw, then HiRes Composite and dual-res SphereSfM → COLMAP. | | 05 Splat training | Spirula Studio, random background, up to 3M splats (Preview 7k steps / Full 30k). | | 06 Your world | Walk through it in the browser (WASD, R/F up/down, drag to look, Shift fast). Download the .ply. |

Jobs live in ComfyUI/output/splatkit_world/ and survive restarts. Resume continues from the first unfinished step. Switch between them in the Worlds menu. The same steps as plain ComfyUI workflows are in example_workflows/ (pano, then SplatKit's dataset graph).

Troubleshooting: see the table at the end of INSTALL.md.

Security

No downloads, no package installs, no shell, no eval/exec, no environment changes. The only program it starts is the Spirula Studio you installed, and its location can only be set from the ComfyUI machine itself. If you run ComfyUI with --listen, the app is reachable from your network just like ComfyUI. Details and how to report a problem: SECURITY.md.

Files

app.py              ComfyUI routes, job queue
pipeline.py         the stages: ComfyUI graphs, Spirula, COLMAP → viewer frame, dependency check
nodes.py            one helper node: SplatKit's scene cloud at an explicit MoGe level
comfy_client.py     ComfyUI API + websocket client
trainer.py          finds Spirula Studio (no downloads)
workflows/          API-format graphs the app runs
example_workflows/  the same steps as UI workflows
static/             the app: index.html, paths.js (camera-path editor), brand/, vendor/ (three.js, Spark)
web/                ComfyUI launcher button
tools/              add an already-trained splat to the gallery
bin/                put Spirula here

Credits

The fly-through method comes from Matrix-3D by Skywork AI (MIT): render a moving camera through the depth mesh, paint the holes black and let a Wan 2.1 LoRA fill them. SplatKit rebuilds it as ComfyUI nodes and adds the path editor, the full-resolution reprojection onto the WAN frames and the splat training. The Wan 2.1 Pano360 LoRA is their LoRA converted to ComfyUI's format. Depth by MoGe (Microsoft), video by Wan 2.1, splat training by Spirula Studio, rendering by Spark.

Thanks to Ostris for the Krea 2 edit training approach (ai-toolkit, ComfyUI-Krea2-Ostris-Edit) that the photo → 360 LoRA is trained and run with, and to nomadoor, whose FLUX.2 klein 360 ERP outpaint LoRA showed the photo → panorama outpainting approach.

License

The code in this repository is MIT. Bundled three.js and Spark are MIT.

Everything it runs keeps its own license; THIRD_PARTY_NOTICES.md has the full list. In particular:

• Krea 2 and the two 360 LoRAs are under the Krea 2 Community License: commercial use only below USD 1M annual revenue, and content filtering is required if you run it for others.
• Spirula Studio is GPL-3.0 and runs as a separate program you install.
• ComfyUI and ComfyUI_UltimateSDUpscale are GPL-3.0.
• All other models used are Apache-2.0, MIT or BSD-3-Clause.