Extensions/NodeGPT
ComfyUI Extension

NodeGPT

Implementation of AutoGen inside ComfyUI. This repository is under development, and not everything is functioning correctly yet.

By antonym-git·Created 3 years ago·Updated 2 years ago· 355
xXAdonesXx/NodeGPT
Nodes24
On cloudLocal install
CategoryAutoGen/Agents, AutoGen
Stars355
Updated2 years ago

Nodes (24)

AppendAgent

When one agent just isn't enough, jam two together

AutoGen/Agents
Assistant

The Assistant node is your default agent — AutoGen's workhorse, with a name you set

AutoGen/Agents
Chat

The Chat node is where the conversation actually happens — NEW to start, SEND to continue

AutoGen
ChatGPT

The 'ChatGPT' node that isn't a chatbot — it's the brain box behind your AutoGen agents

AutoGen/LLM
Conditioning

The node that hands your LLM's answer to Stable Diffusion — a CLIPTextEncode with extra steps

AutoGen
Critic

Your AI's designated contrarian

AutoGen/Agents
DisplayText

Finally, a way to actually see what your agent said

AutoGen
DisplayTextAsImage

When the graph needs your LLM's words as pixels

AutoGen
Engineer

The Engineer agent writes the code; don't let it run anything

AutoGen/Agents
Executor

The Executor agent runs the code the Engineer wrote — treat it with respect

AutoGen/Agents
GroupChat with Manager

Round up a whole squad of agents and let one referee

AutoGen
llama-cpp

Run a GGUF model right inside the graph — slowly, but locally

AutoGen/LLM
llava

The node that gives your workflow eyes — image in, caption out, fully local

AutoGen
LM Studio

The pack author's own recommendation for running local models

AutoGen/LLM
Load API config

The node that keeps your API key out of your workflow JSON (listen to it)

AutoGen
LoadTXT

Drag a text file into your graph like it's any other asset

AutoGen
Ollama

Local models through a LiteLLM proxy, and the defaults are a trap

AutoGen/LLM
oobaboogaOpenAI

Point NodeGPT at text-generation-webui's OpenAI-compatible API

AutoGen/LLM
Output2String

The type adapter that lets your agent talk to the rest of ComfyUI

AutoGen
Planner

Think before you act, in agent form

AutoGen/Agents
Scientist

The code-writing AutoGen agent hiding inside ComfyUI

AutoGen/Agents
TextGeneration

NodeGPT's bare-bones single-shot LLM call

AutoGen
LogAll

How NodeGPT saves the whole agent conversation

AutoGen
UserProxy

The agent that actually runs the code in NodeGPT

AutoGen
Readme

NodeGPT

Implementation of AutoGen inside ComfyUI.

This repository is under development, and not everything is functioning correctly yet.

Dont share your workflow with any API keys inside!!!

Example of Agents

Screenshot 2023-10-16 171331

Screenshot 2023-10-16 165601 The example above should work mostly after installation. The output is displayed in the terminal, and the workflow can be found in the repository (Task_Solving_with_Code_Generation.json).

Screenshot 2023-10-16 165434

Features

  • autogen: Automated Multi Agent Chat

  • Automated Task Solving with Code Generation, Execution & Debugging

  • Automated Complex Task Solving by Group Chat

  • New: llama-cpp (slow), llava

To Do

  • Functions

  • MemGPT (NotWorkingFolder)

Installing

please let me now if you have any issus installing.

Install: https://github.com/comfyanonymous/ComfyUI

I would recomend using LM Studio: https://lmstudio.ai/

Git clone the repository: git clone https://github.com/xXAdonesXx/NodeGPT into the "custom nodes" folder inside ComfyUI

Upon starting, ComfyUI should install the requirements.

Troubleshooting

Try running the Update.bat inside ComfyUI\custom_nodes\NodeGPT

Usage

Start the LM Studio Server

Start ComfUI and place nodes.

For llava:

https://huggingface.co/mys/ggml_llava-v1.5-7b

Contributing

Pull requests, suggestions, and issue reports are welcome.

Credits

LM Studio

ComfyUI: https://github.com/comfyanonymous/ComfyUI

oobabooga: https://github.com/oobabooga/text-generation-webui

chatGPT: https://chat.openai.com/chat

autogen: https://github.com/microsoft/autogen