Memories Video Captioning

If you've ever hand-captioned a pile of generated clips for a dataset or an index, you know the pain. This node automates it: feed it a video and a "memories" vision model on Runware's cloud watches the footage and writes a natural-language description of what's happening. Text in, text out, nothing to train, no local GPU involved.

How it works

It's a caption task in the Runware pack, sharing the same base machinery as the image-caption nodes: your request goes to Runware's inference API, the model analyzes the video, and the result comes back as a text output (STRING). Wire that into any text display or consumer node and you've got a caption.

The only input is video, and it's a string - a UUID or URL, not a VIDEO tensor. This matches the pack's rule for media inputs that don't come from in-graph generation (README: audio, video, and document inputs take a URL, path, or UUID). A local file needs to be hosted or uploaded to Runware before this node can see it.

One thing to know going in: the output is a single text string describing the whole video, not per-frame captions or a structured transcript. Great for search indexing, alt-text, or bulk tagging; not the tool if you need timestamps and per-shot descriptions - you'd run it per-clip instead.

Install

It ships in ComfyUI-Runware, so one install gets you this and every other node in the pack:

cd ComfyUI/custom_nodes
git clone https://github.com/Runware/ComfyUI-Runware
pip install -r ComfyUI-Runware/requirements.txt

Restart ComfyUI, or use ComfyUI Manager → search "Runware" → install. The dependencies are light (runware-sdk, pillow, soundfile) and the vision model runs in the cloud, so hardware requirements are basically nil. You still need a Runware API key from the dashboard - Settings → Runware API key, the RUNWARE_API_KEY env var, or runware auth login.

Gotchas

Pay-per-run and network-bound; the title bar shows the cost after each call. The classic trap is the video input: it expects a URL or UUID string, so don't try to plug a VIDEO socket into it. And manage expectations on caption quality - these models summarize whole clips, so a busy multi-scene video gets a high-level description, not a shot list. If you need fine-grained output, split the video and caption each segment.