ComfyUI-Environment-Visualizer

This ComfyUI node pack allows the user to take a panoramic image and a corresponding depth map, and turn them into a 3D environment, which they can view in an immersive WebXR environment. It also includes utility nodes to assist with creating equirectangular images.

Operational overview

When this node pack is installed, ComfyUI will launch a web server on the same machine when ComfyUI is initialized. By default, the server will be hosted on port 4443, but this can be changed by modifying the value of SERVER_PORT at the top of server.py. As WebXR requires an HTTPS environment to run, a self-signed certificate is generated on first launch, which will display a warning when loaded on most browsers. The web page includes a dropdown to pick from previously saved environments, as well as several controls to adjust the presentation of the environment. The user can turn left or right in the environment with the joysticks on a VR controller, or with A/D and Left/Right on a keyboard. As ComfyUI will always launch this web server on start whenever the node pack is active, you may consider disabling the pack when it is not in use.

Included ComfyUI nodes

• EnvironmentVisualizer: This output node accepts two image inputs, a 2:1 image for the panoramic skybox, and a 2:1 image for the depth map. The node includes a text input for the environment name, and a toggle to automatically launch the generated visualization in a new tab when the workflow has completed. Running this node will automatically save the texture and depth map images into a new subdirectory in the /environments directory with the provided name. The depth input is optional, and the environment will be displayed as a flat skybox if it is omitted.
• InterpolateEdges: This node takes an image, and linearly interpolates the edge pixel values along the vertical edges such that they continuously blend to the opposite edge. This is intended for smoothly blending the seams of the generated depth maps to remove discontinuities. The smoothing will be applied over the given number of pixels on each side. While this approach will work on the panoramic texture or any other image, it is more noticeable in color, and is primarily intended for use with the depth map.
• MapEquirectangular: This node accepts an image, alongside camera parameters such as its FOV and orientation, and projects the image onto an equirectangular image of a given width. Pixels in the resulting equirectangular image that are not mapped from the input image will be black and have zero opacity. This allows the image to be split into both the RGB texture and mask to simplify inpainting. Note that processing the projection with a yaw of 180 degrees will split the image across the horizontal edges, which may help with creating seamless panoramas.

Generation tips

Popular image diffusion models such as Stable Diffusion and FLUX are capable of generating panoramic style images, though they may benefit greatly from a checkpoint or LoRA trained specifically on 360 degree panoramas. Additionally, including phrases in the prompt such as 'HDRI', 'panoramic image of...', or '360HDR' may improve your results. All images should be generated with a 2:1 aspect ratio, such as 2048x1024. There are many existing tools available in ComfyUI to generate depth maps from images, such as Marigold Depth Estimation and Depth Anything.

Standalone use

This WebXR server included in this repository can be used as a standalone utility by running server.py manually. The dropdown on the page will display the names of all subdirectories in /environments, and attempt to load the skybox.png and depth.png images in the selected subdirectory. Although the skybox texture is required, the depth map is optional. While running the Environment Visualizer node in ComfyUI will automatically save these files to the correct path, any correctly named images placed in the environments directory can be loaded and viewed.