ComfyUI_LLMVISION

This repository provides integration of GPT-4 and Claude 3 models into ComfyUI, allowing for both image and text-based interactions within the ComfyUI workflow.

Features

Call GPT-4 and Claude 3 models directly from ComfyUI
Support for image-based interactions using GPT-4 Vision and Claude-3 Image Chat
Text-based chat functionality with OpenAI Chat and Claude-3 Chat
Customizable prompts and model settings
Easy integration into existing ComfyUI workflows

Installation

Clone this repository into the custom_nodes folder of comfyui:

git clone https://github.com/AppleBotzz/ComfyUI_LLMVISION.git

Install the required dependencies:
```
pip install -r requirements.txt
```
Import the workflow.json file into your ComfyUI workspace.

Usage

Open the ComfyUI interface and navigate to your workspace.
Locate the imported nodes in the node library under the AppleBotzz Category:
- GPT-4V Image Chat
- OpenAI Chat
- Claude-3 Image Chat
- Claude-3 Chat
Drag and drop the desired node into your workflow.
Configure the node settings, such as API keys, model selection, and prompts.
Connect the node to other nodes in your workflow as needed.
Run the workflow to execute the LLM-based tasks.

Configuration

openai_api_key: Your OpenAI API key for accessing GPT-4 models.
claude_api_key: Your Anthropic API key for accessing Claude-3 models.
endpoint: The API endpoint URL (default: OpenAI or Anthropic endpoints).
model: Select the specific model to use for each node.
prompt: Customize the prompt for the LLM-based task.
max_token: Set the maximum number of tokens for the generated response.

Make sure to keep your API keys secure and do not share them publicly.

API KEYS WILL GET SAVED IN WORKFLOWS, DONT FROGET. USE THE ENVIRONMENT VARIBLES IF YOU WANT TO ENSURE NO LEAKAGE

Acknowledgements

ComfyUI - A powerful and extensible UI framework for machine learning workflows.
OpenAI - Provider of the GPT-4 models.
Anthropic - Provider of the Claude-3 models.