ComfyUI Extension: ComfyUI_Yvann-Nodes

Authored by yvann-ba

Created

Updated

417 stars

Audio Reactive nodes for AI animations 🔊 Analyze audio, extract drums, bass, vocals. Compatible with IPAdapter, ControlNets, AnimateDiff... Generate reactive masks and weights. Create audio-driven visuals. Produce weight graphs and audio masks. Ideal for music videos and reactive animations. Features audio scheduling and waveform analysis

Custom Nodes (14)

README

🔊 ComfyUI_Yvann-Nodes Check the Tutorial

Made with the great help of Lilien 😎

A pack of custom nodes that enable audio reactivity within ComfyUI, allowing you to generate AI-driven animations that sync with music

What Does This Do?

  • Create Audio Reactive AI videos, enable controls over AI generations styles, content and composition with any audio
  • Simple: Just Drop one of our Workflows in ComfyUI and specify your audio and visuals input
  • Flexible: Works with existing ComfyUI AI tech and nodes (eg: IPAdapter, AnimateDiff, ControlNet, etc.)

Video result (generated by our amazing users 😁)

Audio Reactive Images to Video ⬇️

<table border="0" style="width: 100%; text-align: left; margin-top: 20px;"> <tr> <td> <video src="https://github.com/user-attachments/assets/615394cd-c829-4ee0-94de-1ffd20d35b9d" width="100%" controls autoplay loop></video> </td> <td> <video src="https://github.com/user-attachments/assets/d9a630d5-cd13-4cf4-a1da-282e6078cd49" width="100%" controls autoplay loop></video> </td> <td> <video src="https://github.com/user-attachments/assets/cd6af84d-db51-47cc-897d-c880271f2971" width="100%" controls autoplay loop></video> </td> </tr> </table> <table border="0" style="width: 100%; text-align: left; margin-top: 20px;"> <tr> <td> <video src="https://github.com/user-attachments/assets/f4b64874-5ca9-49ea-8d2c-40b377a5b5bd" width="100%" controls autoplay loop></video> </td> <td> <video src="https://github.com/user-attachments/assets/8e75df33-6426-4d6e-98d1-f8288cc87b74" width="100%" controls autoplay loop></video> </td> <td> <video src="https://github.com/user-attachments/assets/9d179485-011d-4de2-a4fb-d8489f20a2cf" width="100%" controls autoplay loop></video> </td> </tr> </table>

Audio Reactive Video to Video ⬇️

<table border="0" style="width: 100%; text-align: left; margin-top: 20px;"> <tr> <td> <video src="https://github.com/user-attachments/assets/c0450100-a61f-4707-9e14-0d4ca563a2b1" width="100%" controls autoplay loop></video> </td> <td> <video src="https://github.com/user-attachments/assets/c0fa2ca0-6c0f-4687-b1c9-fe531278c58e" width="100%" controls autoplay loop></video> </td> </tr> </table> <table border="0" style="width: 70%; text-align: left; margin-top: 20px;"> <tr> <td> <video src="https://github.com/user-attachments/assets/a25bd896-4030-4576-9129-670c2cb2e544" width="100%" controls autoplay loop></video> </td> </tr> </table>

Quick Setup

Pick a Workflow (Images → Video or Video → Video)

  1. Images → Video

    • Takes a set of images plus an audio track.
    • Watch Tutorial:
      Images2Video

  2. Video → Video

    • Takes a source video plus an audio track.
    • Watch Tutorial:
      Video2Video

Load Your Chosen Workflow in ComfyUI

  1. Download the .json file for the workflow you picked:

  2. Drop the .json file into the ComfyUI window.

  3. Open the "🧩 Manager" → "Install Missing Custom Nodes"

    • Install each pack of nodes that appears.
    • Restart ComfyUI if prompted.

  4. Set Your Inputs & Generate

    • Provide the inputs needed (everything explained here
    • Click Queue button to produce your audio-reactive animation!

That's it! Have fun playing with the differents settings now !! (if you have any questions or problems, check my Youtube Tutorials

Nodes Details

<details> <summary><strong>Click to Expand: Node-by-Node Reference</strong></summary>

Audio Analysis 🔍

Analyzes audio to generate reactive weights for each frame.

<details> <summary><em>Node Parameters</em></summary>
  • audio_sep_model: Model from "Load Audio Separation Model"
  • audio: Input audio file
  • batch_size: Frames to associate with audio weights
  • fps: Frame rate for the analysis

Parameters:

  • analysis_mode: e.g., Drums Only, Vocals, Full Audio
  • threshold: Minimum weight pass-through
  • multiply: Amplification factor

Outputs:

  • graph_audio (image preview),
  • processed_audio, original_audio,
  • audio_weights (list of values).
</details>

Load Audio Separation Model 🎧

Loads or downloads an audio separation model (e.g., HybridDemucs, OpenUnmix).

<details> <summary><em>Node Parameters</em></summary>
  • model: Choose between HybridDemucs / OpenUnmix.
  • Outputs: audio_sep_model (connect to Audio Analysis or Remixer).
</details>

Audio Peaks Detection 📈

Identifies peaks in the audio weights to trigger transitions or events.

<details> <summary><em>Node Parameters</em></summary>
  • peaks_threshold: Sensitivity.
  • min_peaks_distance: Minimum gap in frames between peaks.
  • Outputs: Binary peak list, alternate list, peak indices/count, graph.
</details>

Audio IP Adapter Transitions 🔄

Manages transitions between images based on peaks. Great for stable or style transitions.

<details> <summary><em>Node Parameters</em></summary>
  • images: Batch of images.
  • peaks_weights: From "Audio Peaks Detection".
  • blend_mode, transitions_length, min_IPA_weight, etc.
</details>

Audio Prompt Schedule 📝

Links text prompts to peak indices.

<details> <summary><em>Node Parameters</em></summary>
  • peaks_index: Indices from peaks detection.
  • prompts: multiline string.
  • Outputs: mapped schedule string.
</details>

Audio Remixer 🎛️

Adjusts volume levels (drums, vocals, bass, others) in a track.

<details> <summary><em>Node Parameters</em></summary>
  • drums_volume, vocals_volume, bass_volume, others_volume
  • Outputs: single merged audio track.
</details>

Repeat Image To Count 🔁

Repeats a set of images N times.

<details> <summary><em>Node Parameters</em></summary>
  • mask: Mask input.
  • Outputs: Repeated images.
</details>

Invert Floats 🔄

Flips sign of float values.

<details> <summary><em>Node Parameters</em></summary>
  • floats: list of floats.
  • Outputs: inverted list.
</details>

Floats Visualizer 📈

Plots float values as a graph.

<details> <summary><em>Node Parameters</em></summary>
  • floats (and optional second/third).
  • Outputs: visual graph image.
</details>

Mask To Float 🎭

Converts a mask into a single float value.

<details> <summary><em>Node Parameters</em></summary>
  • mask: input.
  • Outputs: float.
</details>

Floats To Weights Strategy 🏋️

Transforms float lists into an IPAdapter "weight strategy."

<details> <summary><em>Node Parameters</em></summary>
  • floats: list of floats.
  • Outputs: dictionary with strategy info.
</details> </details>
<h3 align="center"> Please give a ⭐ on GitHub it helps us enhance our Tool and it's Free !! (: </h3> <p align="center"> <a href="https://www.star-history.com/#yvann-ba/ComfyUI_Yvann-Nodes&Date"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=yvann-ba/ComfyUI_Yvann-Nodes&type=Date&theme=dark" width="50%" /> <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=yvann-ba/ComfyUI_Yvann-Nodes&type=Date" width="50%" /> <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=yvann-ba/ComfyUI_Yvann-Nodes&type=Date" width="65%" /> </picture> </a> </p> <p align="center"> <a href="https://github.com/yvann-ba/ComfyUI_Yvann-Nodes/stargazers"> <img src="https://reporoster.com/stars/dark/yvann-ba/ComfyUI_Yvann-Nodes" alt="Stargazers" width="50%"> </a> </p>