Nodes/KJNodes for ComfyUI/End Recording CUDAMemory History
ComfyUI Node Runs on cloud

End Recording CUDAMemory History

Find out what's actually eating your VRAM

By kijai·Created 3 years ago·Updated about 17 hours ago· 2,930
End Recording CUDAMemory History
  • input
  • input
  • output_path
output_pathcomfy_cuda_memory_history

"CUDA out of memory" tells you that you ran out, not why. If you're chasing a VRAM spike that only shows up mid-workflow - a particular node, a particular resolution, a leak that builds up over a batch - guessing at settings and re-running is slow and often wrong. EndRecordCUDAMemoryHistory is Kijai's answer: it's the bookend of a start/stop pair that wraps a section of your graph and dumps PyTorch's own memory-allocation history to a file you can actually inspect, instead of squinting at nvidia-smi during a crash.

This is a profiling tool, not a fix. It won't lower your VRAM usage by itself - it tells you exactly where to look before you start changing things.

How it works

You place a matching start-recording node before the section of your graph you're worried about, and this node after it. Between the two, PyTorch's CUDA memory allocator logs every allocation and free. When execution hits this node, it stops the recording and writes the log to a .pickle file. You can then load that file at pytorch.org/memory_viz - an interactive timeline of every allocation, its size, and its call stack - or feed it into the pack's own VisualizeCUDAMemoryHistory node for an in-graph look.

The inputs and outputs that matter

  • input - accepts anything (type *). This is a pass-through: you route whatever you're already carrying through your graph (model, image, latent, doesn't matter) into this node so it lands at the right point in execution order, then it comes back out the other side unchanged. It's a timing anchor, not a real dependency.
  • output_path (default comfy_cuda_memory_history) - the filename the memory snapshot gets written to. Change it if you're running multiple profiling passes and don't want them overwriting each other.

Outputs mirror the inputs - input passed through untouched, plus output_path as a string, useful if you want to wire the filename into a downstream node (like VisualizeCUDAMemoryHistory) rather than re-typing it.

How to install it

Via ComfyUI Manager, search "KJNodes for ComfyUI" and install. Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/kijai/ComfyUI-KJNodes
pip install -r ComfyUI-KJNodes/requirements.txt

then restart. No extra dependency beyond what your ComfyUI install already needs for CUDA - this reads PyTorch's built-in memory API, it doesn't ship its own.

Common issues & troubleshooting

The file never shows up, or looks empty. This only records CUDA activity, so it needs an actual GPU run in between the start and end node - if the section you're wrapping got skipped, cached, or ran on CPU, there's nothing to log. Double check execution actually flowed through both nodes for that queue.

You don't have a matching start node in the graph. This node is meaningless on its own - it closes a recording that has to have been opened. If you only dropped this one in, go find (or add) the StartRecordCUDAMemoryHistory counterpart first.

memory_viz is intimidating the first time. It's a flame-graph-style timeline, not a simple bar chart - give it a minute. Look for the tallest, widest blocks first; that's usually your actual culprit, not the many small short-lived allocations around it. If you'd rather stay inside ComfyUI, the pack's VisualizeCUDAMemoryHistory node renders a simpler view without leaving the canvas.

This isn't a tool most workflows need day to day - it's for the specific moment you're debugging a real OOM and guessing has stopped working.

CategoryKJNodes/memory

Inputs (2)

NameTypeDefaultDescription
input*
output_pathSTRINGcomfy_cuda_memory_history

Outputs (2)

NameTypeDescription
input*
output_pathSTRING