A connector to use ComfyUI in serverless deployments
ComfyUI is incredibly flexible and fast; it is the perfect tool launch new workflows in serverless deployments. Unfortunately, there isn't a lot on API documentation and the examples that have been offered so far don't deal with some important issues (for example: good ways to pass images to Comfy, generalized handling of API json files, etc).
Using the websocket endpoint that Comfy offers, I created a very simple way to generate images from Comfy via code. With the ComfyConnector class, you may:
- Automatically start Comfy webserver (no need to python3 main.py)
- Easily edit the API json file using the replace_key_value method, regardless of how complex/spaghettized your workflow may be (no need to hardcode your inputs based on each workflow; just tell the function what parameter to look for and it will change it for you)
- Send to Comfy the API json file containing the workflow topology and editable parameters, and receive back the generated images as an image list object
- Send to Comfy images and receive back an image list object for use in img2img/controlnet workflows
- Automatic discovery of the outputs in a JSON file via the find_output_node method
- Kill main.py after generation using the kill_api method
With this repository, generating images with Comfy takes two lines of code:
prompt = json.load(open('workflow_api.json')) # loads the workflow_api.json file as exported by ComfyUI dev mode
images = api_singleton.generate_images(prompt) # sends the json file and receives back the generated images
This repository makes it very simple to connect ComfyUI with handling functions. It is a drop-in addition to the repository and only interacts with the existing codebase as it extends the utility of its already-functioning websocket API service.
(IMPORTANT: Your workflow MUST CONTAIN one, and only one, SaveImage node for the connector to work, since this will be deemed your output node and will be used by the connector to fetch any generated images.)
To make it work, you basically just need to make sure the API_COMMAND_LINE variable is set to the correct path to run your ComfyUI install. Once that's done, you'll be able to launch ComfyUI by just creating an instance of ComfyConnector, say, by adding the following lines to your python code (assuming you cloned this repository and are working on the base folder of it):
from comfy_serverless import ComfyConnector
comfy_connector = ComfyConnector()
Once you do that, ComfyUI will start, and the comfy_connector instance will be a singleton of the ComfyConnector class. That means that wherever you invoke it in your code, you'll get the same ComfyUI session you were working with.
The generally useful methods in the ComfyConnector class should be the following:
generate_images: This is the main reason why I created this repository. This method is used to pass to ComfyUI the workflow_api.json that can be exported from ComfyUI on Developer mode. Its output is a list of the Image objects generated by ComfyUI (note that the images you create should also be present in the ComfyUI output folder as usual). Once again, VERY IMPORTANT: workflow_api.json must contain one, and only one, SaveImage node for the method to work.
upload_image: This method is used to upload an image to the API server for use in img2img, controlnet, etc.
replace_key_value: This method can be used to edit the payload of a prompt. It might be helpful if you are building a system that would require some parameters of the API to be changed programmatically.