TS RT Prompt Enhancer

An LLM as a node, not a panel

The pack ships two ways to run Gemma 4 on Google's LiteRT-LM runtime: TS Super Prompt RT, which is all buttons, panels and a microphone, and this one. TS RT Prompt Enhancer is the boring version, and boring is what you want inside a pipeline: text, pictures and sound in, text out, executed by the queue like any other node. No hand on a button, no API key, nothing leaving your machine.

That's the difference between using an LLM and building with one. Caption a folder of images in one run. Enhance a prompt that arrived from another node. Write a music prompt from a reference track. Turn a voice note into a prompt. Translate before the text encoder sees it. All of that needs a node the queue can call, which is exactly what this is - the same relationship TS Qwen 3 has to TS Super Prompt, for people who've used that pair.

What the model actually is

Two abliterated Gemma 4 builds - E2B (2.4 GB) and E4B (3.4 GB, the default) - pulled on first use from a public Hugging Face repo into models/LLM/litert. No token, nothing to fetch by hand. Abliterated meaning the refusal behaviour is trained out of the weights: same weights, same speed, and it matters a lot for a node whose job is writing prompts. E4B writes better; E2B is roughly twice as fast and lighter.

The runtime is the interesting bit. LiteRT-LM runs on WebGPU, not CUDA, and ComfyUI's memory manager cannot see a single byte of it - torch.cuda.mem_get_info reports the same free memory whether Gemma is resident or not, while nvidia-smi moves by 3.5 GB. So a resident model is memory ComfyUI believes it still has, and the sampler that follows plans accordingly. That's why the model is unloaded after every run by default, and why keep_loaded exists with a tooltip explaining the price rather than a cheerful "faster repeats!". Unloading costs about a second, a reload about five.

The context window is 8192 tokens, every preset in the pack fits, and a prompt that wouldn't fit is refused with a readable message before three gigabytes come off disk. Speculative decoding is off so seeds mean something - same seed, same answer. And one thing that isn't a footnote: LiteRT holds a single engine per process, so requests queue. Loading a second model mid-generation took the whole server down with an access violation once, measured, which is why a waiting request says it's waiting instead of looking frozen.

The inputs worth thinking about

• model - E4B (default) or E2B. Download happens on first use.
• system_preset - what the model is asked to do. Same preset file as TS Super Prompt RT and TS Qwen 3: enhance, caption, describe an image, write out speech, translate, the music and SFX presets. "Your instruction" hands control to custom_system_prompt.
• prompt - your text, multiline. It can be empty when pictures or a recording are connected, which is how image captioning and audio description work.
• max_new_tokens - 0 keeps the preset's own ceiling. The tooltip is honest about what it is: a guard against a runaway answer, and it will stop the model mid-sentence.
• enable - off, the prompt passes through unchanged and nothing is loaded. Handy for A/B'ing a graph without rewiring.
• audio_mode - listen or transcribe. More below.
• Optional: images, audio, custom_system_prompt.

Output is a single text STRING - into a CLIPTextEncode, another LLM, a text-saving node, whatever.

Pictures: one, two, or a whole clip's frames. At most four reach the model; a longer batch is sampled evenly from first to last rather than cut after the fourth, so a 60-frame clip gives you frames 1, 20, 40, 60 - each shrunk to 1024 px, because every pixel costs context. There is deliberately no VIDEO input: ComfyUI would decode the whole clip into frames (a minute of 1080p is tens of gigabytes) for four of them. Feed frames to images and the soundtrack to audio.

Sound, in one of two modes. listen lets the model hear the recording itself - music, sounds, a short phrase - and only the first 30 seconds, the longest clip Gemma is trained on. Feeding a long recording as several clips doesn't work; measured, two different tracks in one message came back as the same song described twice. transcribe writes speech out in 30-second pieces, up to five minutes, and appends it to the prompt. The transcription is tuned for Russian speech with English technical terms, so ComfyUI, workflow and LoRA come back in Latin script the way people actually write them.

Installing it

The node needs the pack plus a runtime that is deliberately not in requirements.txt and not in the pack's [all] extra - it's an exact pin on a native runtime with no Intel-macOS wheel:

pip install litert-lm-api==0.16.1

Or, from inside the clone: pip install -e .[litert]. Use the Python ComfyUI runs from (Windows portable: python_embeded\python.exe -m pip install ...). Wheels exist for Windows, macOS on Apple Silicon and Linux x86_64/aarch64 - an earlier version of the pack's README claimed Linux had none, which was wrong. On Linux, if the GPU backend won't come up, the node falls back to the CPU by itself.

Then, for the pack itself: ComfyUI Manager → search Timesaver → install → restart, or clone and pip install -r requirements.txt.

Where it goes wrong

The node isn't in the menu. Check the startup log - Timesaver prints a load report with Optional missing imports and Module load issues. Nine times out of ten it's litert-lm-api not installed on the right interpreter. Install it, restart fully (not just a browser refresh).

You set keep_loaded and then the sampler blew up. That's the invisible memory doing exactly what the tooltip warned about. Turn it off and pay the reload.

Two runs seem to wait for each other. They do. One engine per process, by design, and the node says so in the progress panel.

A red node with a sentence about the prompt. That's the intended behaviour: when there's nothing to work from, or the prompt is too long for the window, the run stops with a readable message instead of sending the word "ERROR" downstream as your prompt.