FFMPEG Agent

FFMPEG Agent is the flagship node of the ComfyUI-FFMPEGA pack, and it does the thing that sounds fake until you try it: you type "add a cinematic letterbox, fade in, then compress for web at 720p" and it builds and runs the actual ffmpeg command chain. It's a natural-language front end over one of the most obtuse CLIs ever written - 200+ skills mapped to ffmpeg filters, from brightness and vignette up to Whisper subtitles and SAM3 masking.

Here's the part people miss: the name is a lie, or at least an aspiration. The agent doesn't call any API by default and needs no key. llm_model defaults to none, and no_llm_mode defaults to manual - meaning the node can execute an Effects Builder pipeline directly with zero AI involved. The LLM is a mode, not a requirement. That's the right design, honestly: an LLM guessing at ffmpeg filter syntax is a genuinely dicey proposition, and the manual path is how the pack stays deterministic.

The inputs that matter

• video_path - absolute path to the source video. Used as the ffmpeg input unless you connect frames via images_a.
• prompt - the natural-language instruction ("Speed up 2x, keep the audio pitch"). Not needed in manual mode.
• llm_model - none (no-LLM), local Ollama models (qwen3:8b, llama3.3, etc.), CLI tools (gemini-cli, claude-cli - no API key, but they need Node.js 18+), or cloud APIs (gpt-, claude-, gemini-* - those want api_key). Or custom to type any model name.
• no_llm_mode - what to do when llm_model is none: manual (Effects Builder pipeline), or the direct AI modes - sam3_masking, transcribe, karaoke_subtitles, generate_audio, lip_sync, animate_portrait, video_depth, flux_klein, minimax_remover, ai_upscale, rembg.
• quality_preset - draft → lossless; crf and encoding_preset override it if you know what you're doing.
• pipeline_json - where the Effects Builder connects. In manual mode it's executed directly; with an LLM it serves as skill hints.

You'll also want preview_mode (quick 480p/10s check before committing to a full render), save_output, use_vision (embeds frames as images for vision models - costs more tokens), and verify_output, which makes the agent re-inspect its render and auto-correct if it didn't match intent. Outputs: images, audio, the rendered video_path, plus command_log (the actual ffmpeg command - great for learning) and analysis.

Installing it

ComfyUI Manager → search ComfyUI-FFMPEGA → Install, or:

cd ComfyUI/custom_nodes
git clone https://github.com/AEmotionStudio/ComfyUI-FFMPEGA.git
cd ComfyUI-FFMPEGA
pip install -r requirements.txt

Restart ComfyUI. Requirements: ffmpeg on PATH (ffmpeg -version), Python 3.10+. Ollama is optional (for local models - ollama pull qwen2.5:8b), Node.js 18+ only for the CLI model connectors. The AI modes auto-download their models on first use - SAM3 ~300 MB, Whisper up to ~3 GB, MuseTalk, LivePortrait, and the big one, FLUX Klein 4B at ~15 GB. allow_model_downloads on the node turns that off.

Common issues

If nothing runs, it's almost always ffmpeg not on PATH. Ollama models failing usually means the server isn't running (ollama serve) or the model isn't pulled. Empty LLM responses are often a still-loading model or a prompt too long for the context window - retry or swap models. The README's best practical advice is worth taking seriously: one task per run. Chain ten edits into one prompt and the agent's attention splits; do the trim, save, then load the result and do the color grade. On low VRAM, same rule for model-loading tasks - one per run, because SAM3 + Whisper in one pass will OOM a 12 GB card.

One more thing, with the pack's own framing in mind: this is an LLM-driven node, and LLM-driven nodes are exactly the category the community has learned to be careful with (the ComfyUI_LLMVISION malware incident ran on the same "downloads its own model" trust model). This one is open source, GPLv3, from an author with multiple established packs - but if you don't need the LLM, the manual path is genuinely the safer, cheaper default anyway. On the frontier of "describe it in English," you're paying for convenience, not capability.