VRGDG_SpeechEmotionExtractor

VRGDG_SpeechEmotionExtractor takes an AUDIO input and turns it into a per-scene emotion timeline. Two inputs in - audio and scene_duration_seconds (a float, default 4, range 1–10) - one string out: emotion_timeline. The idea is simple and genuinely useful: a song isn't one emotional note, it's a curve, and if you're generating a music video you want the visuals to follow that curve rather than ignore it.

How it works

The node slices the audio into scenes of scene_duration_seconds each and analyzes each chunk for emotional character. That's why librosa is in the pack's requirements.txt - this is exactly the sort of audio analysis it does: energy, tempo, and spectral character over time windows. Fast, loud, driving segments read as "energetic" or "intense"; slower, quieter ones as "calm" or "sad." The output is a timeline string - one label (or label pair) per scene, in order - that other nodes in the workflow can consume when building scene prompts.

The practical chain in the pack: this timeline feeds the storyboard stage, so scene 1 might get "calm, introspective" visuals while scene 4 gets "energetic, climactic," matching the actual structure of the track instead of a flat mood the whole way through. The 4-second default is a sensible starting point - most pop music's emotional beats land somewhere in the 2–8 second range, and 4 gives you readable chunks without turning a 3-minute track into 45 scenes.

What to set

The only field that matters is scene_duration_seconds. Shorter = more granular, more scenes, more risk of noisy labels. Longer = coarser, steadier, fewer surprises. If your track has a clear verse/chorus structure, try 4 first, then check whether the timeline actually tracks the song's energy changes. If it's a slow build, a longer window will look more sensible. If you're getting flips every chunk, lengthen it.

Install

Same as the pack:

cd ComfyUI/custom_nodes
git clone https://github.com/vrgamegirl19/comfyui-vrgamedevgirl

or ComfyUI Manager → "vrgamedev", restart. And this node is the reason librosa is in the requirements at all - if your ComfyUI install was set up before the pack and you're seeing an import error around this node, make sure the requirements actually installed (python.exe -m pip install -r ..\custom_nodes\comfyui-vrgamedevgirl\requirements.txt from python_embeded on Windows).

One expectation to set: this is a heuristic, not a sentiment-analysis model. It's reading audio energy and tempo, not the semantics of the words. A tender ballad with a loud chorus will read as emotional peaks at the chorus - which, honestly, is exactly what you want for video pacing. If you were hoping for "the model understands this is a breakup song," that's not what this does, and it doesn't claim to.