LeapTalk Runtime

Two dropdowns, and one of them matters a lot

LeapTalk Runtime produces no image and no video. It outputs one thing - a runtime handle - and it's the required input on LeapTalk Generate. So why does it deserve its own node instead of being a widget on Generate? Because it's a trust boundary.

Custom nodes execute arbitrary Python, and a workflow JSON is something people share. This pack splits the difference: a workflow can only pick a pre-registered runtime, never name a program path, so the workflow someone posted can't point ComfyUI at an interpreter of its choosing. That's why a fresh install shows (no runtime configured) until an administrator writes the config file.

The other dropdown is the one you'll care about day to day: backend. runtime default means whatever the runtime was configured with, which is one-shot unless an admin changed it. Set persistent and the pack stops being slow.

one-shot vs persistent, in numbers

One-shot is v0.1 behaviour: every queued job starts a fresh runtime process, loads ~7 GB of weights, generates, exits. On the author's RTX 5090 about 23 s of each job is that startup, so a 9.4 s speech clip took 33–36 s end to end and a 34 s clip about 45 s.

Persistent is the opt-in v0.2 path. The first job starts a worker that keeps the model loaded, and later queue items reuse it: 6.2 s for that same 9.4 s clip (median of five), 17.7 s for the 34 s clip. That's not a marginal gain, it's a different experience. The cost: the worker holds memory while doing nothing - around 3.5 GiB CUDA (about 4 GiB on the GPU) and 8.4–8.8 GiB of Windows commit on the tested machine, plus a job's peak.

So: persistent when you're iterating - trying a portrait, re-running a re-recorded line - and one-shot when you're done for the night, or on a card where that idle memory would starve everything else. Want the memory back without restarting ComfyUI? That's the LeapTalk Worker node.

What it does mechanically

select() resolves the runtime id against leaptalk.runtimes.json, validates the backend choice, and returns a dict - runtime id plus backend override (or None for "use the runtime's default"). Generate reads that dict, re-resolves the runtime, and passes the effective backend into the job client. Jobs are still per-job folders on disk; persistent just hands that folder to a worker over a local pipe.

The runtime identity is fingerprinted - settings, decoder, code, protocol, upstream commit, model file stamps. Change the decoder or edit runtime files and the worker is replaced rather than reused, with the reason in the report. After any error the worker is thrown away too, so the next job pays full startup. Expected, not a bug.

Inputs and outputs

Required: runtime_id, an enum listing whatever is in the config file, sorted.

Optional: backend, one of runtime default, persistent, one-shot.

One output: runtime (LEAPTALK_RUNTIME), which goes into LeapTalk Generate and nothing else.

Setting up the runtime it points at

The node itself installs like any other pack, and adds nothing to ComfyUI's Python environment:

cd ComfyUI/custom_nodes
git clone https://github.com/hiroki-abe-58/ComfyUI-LeapTalk

The real work is the config, at ComfyUI/user/leaptalk.runtimes.json (or wherever COMFYUI_LEAPTALK_CONFIG points). Start from examples/leaptalk.runtimes.example.json and fill in absolute paths: python (the venv interpreter), upstream_dir (the pinned LeapTalk checkout), the three models directories, ffmpeg, plus optional env, timeout_minutes, max_audio_seconds (up to 1800, rejects longer clips rather than trimming them), backend, worker_idle_seconds (120 default, 10–86400), and a memory_guard block. A v0.1 config file keeps working unchanged.

Restart ComfyUI after editing the file, then run LeapTalk Doctor before you queue anything.

Traps

• The dropdown is empty / reads (no runtime configured) - no valid config file, or a ConfigError in it. The node swallows config errors when listing ids so the UI still loads; Doctor is what surfaces the actual message.
• runtime 'x' is not configured at queue time - you renamed or deleted the entry after the workflow was built. Re-pick it.
• You changed the decoder and got a long job anyway - new worker, new load. Same for editing runtime files.
• Windows-only in practice. Windows 11 with a native Windows runtime is the tested path. WSL2 runtimes aren't supported in this release and macOS/Apple Silicon isn't either; the Linux path exists and is covered by CPU tests only, with no real-weights verification.
• max_audio_seconds is not a trim. Exceed it and the job is refused. Raise the value or split the audio.
• The memory guard can refuse a load you know would fit. Its defaults are conservative settings for a 64 GB Windows workstation, not requirements - but the author explicitly doesn't recommend loosening them as a first move.