✨ ComfyUI SDK ✨

A robust and meticulously crafted TypeScript SDK 🚀 for seamless interaction with the ComfyUI API. This SDK significantly simplifies the complexities of building, executing, and managing ComfyUI workflows, all while providing real-time updates and supporting multiple instances. 🖼️

🧭 Table of Contents

🌟 Key Features 🌟
📦 Installation 📦
🚀 Getting Started 🚀
📚 API Reference 📚
📂 Examples
🤝 Contributing
📜 License

🌟 Key Features 🌟

💎 TypeScript Powered: Enjoy a fully typed codebase, ensuring enhanced development, maintainability, and type safety. 🛡️
🏗️ Workflow Builder: Construct and manipulate intricate ComfyUI workflows effortlessly using a fluent, intuitive builder pattern. 🧩
🤹 Multi-Instance Management: Handle a pool of ComfyUI instances with ease, employing flexible queueing strategies for optimal resource utilization. 🌐
⚡ Real-Time Updates: Subscribe to WebSocket events for live progress tracking, image previews, and error notifications. 🔔
🔑 Authentication Flexibility: Supports Basic Auth, Bearer Token, and Custom Authentication Headers, catering to diverse security requirements. 🔒
🔌 Extension Support: Seamlessly integrate with ComfyUI Manager and leverage system monitoring through the ComfyUI-Crystools extension. 🛠️
🔀 Flexible Node Bypassing: Strategically bypass specific nodes in your workflows during generation, enabling advanced customization. ⏭️
📚 Comprehensive Examples: Includes practical examples for Text-to-Image (T2I), Image-to-Image (I2I), and complex multi-node workflows. 📝
🚨 Robust Error Handling: Provides detailed error messages to facilitate debugging and graceful handling of API failures. 🐛
📝 Automatic Changelog: Automatically generates a changelog with each release, utilizing auto-changelog for transparent version tracking. 🔄

📦 Installation 📦

bun add @saintno/comfyui-sdk

npm i @saintno/comfyui-sdk

🚀 Getting Started 🚀

🎬 Basic Usage

Here's a simplified example to quickly get you started:

import { ComfyApi, CallWrapper, PromptBuilder, TSamplerName, TSchedulerName, seed } from "@saintno/comfyui-sdk";
import ExampleTxt2ImgWorkflow from "./example-txt2img-workflow.json";

const api = new ComfyApi("http://localhost:8189").init();
const workflow = new PromptBuilder(
  ExampleTxt2ImgWorkflow,
  ["positive", "negative", "checkpoint", "seed", "batch", "step", "cfg", "sampler", "sheduler", "width", "height"],
  ["images"]
)
  .setInputNode("checkpoint", "4.inputs.ckpt_name")
  .setInputNode("seed", "3.inputs.seed")
  .setInputNode("batch", "5.inputs.batch_size")
  .setInputNode("negative", "7.inputs.text")
  .setInputNode("positive", "6.inputs.text")
  .setInputNode("cfg", "3.inputs.cfg")
  .setInputNode("sampler", "3.inputs.sampler_name")
  .setInputNode("sheduler", "3.inputs.scheduler")
  .setInputNode("step", "3.inputs.steps")
  .setInputNode("width", "5.inputs.width")
  .setInputNode("height", "5.inputs.height")
  .setOutputNode("images", "9")
  .input("checkpoint", "SDXL/realvisxlV40_v40LightningBakedvae.safetensors", api.osType)
  .input("seed", seed())
  .input("step", 6)
  .input("cfg", 1)
  .input<TSamplerName>("sampler", "dpmpp_2m_sde_gpu")
  .input<TSchedulerName>("sheduler", "sgm_uniform")
  .input("width", 1024)
  .input("height", 1024)
  .input("batch", 1)
  .input("positive", "A picture of cute dog on the street");

new CallWrapper(api, workflow)
  .onFinished((data) => console.log(data.images?.images.map((img: any) => api.getPathImage(img))))
  .run();

🔍 Breakdown

Import essential components from the SDK.
Create and initialize the ComfyApi instance.
Use PromptBuilder to define the workflow structure and set input nodes.
Set specific input values, including the checkpoint path, seed, and prompt.
Execute the workflow and log the generated image URLs using the CallWrapper.

🔄 Managing Multiple Instances with `ComfyPool`

import {
  ComfyApi,
  CallWrapper,
  ComfyPool,
  EQueueMode,
  PromptBuilder,
  seed,
  TSamplerName,
  TSchedulerName
} from "@saintno/comfyui-sdk";
import ExampleTxt2ImgWorkflow from "./example-txt2img-workflow.json";

const ApiPool = new ComfyPool(
  [new ComfyApi("http://localhost:8188"), new ComfyApi("http://localhost:8189")],
  EQueueMode.PICK_ZERO
)
  .on("init", () => console.log("Pool in initializing"))
  .on("add_job", (ev) => console.log("Job added at index", ev.detail.jobIdx, "weight:", ev.detail.weight))
  .on("added", (ev) => console.log("Client added", ev.detail.clientIdx));

const generateFn = async (api: ComfyApi, clientIdx?: number) => {
  const workflow = new PromptBuilder(
    ExampleTxt2ImgWorkflow,
    ["positive", "negative", "checkpoint", "seed", "batch", "step", "cfg", "sampler", "sheduler", "width", "height"],
    ["images"]
  )
    .setInputNode("checkpoint", "4.inputs.ckpt_name")
    .setInputNode("seed", "3.inputs.seed")
    .setInputNode("batch", "5.inputs.batch_size")
    .setInputNode("negative", "7.inputs.text")
    .setInputNode("positive", "6.inputs.text")
    .setInputNode("step", "3.inputs.steps")
    .setInputNode("width", "5.inputs.width")
    .setInputNode("height", "5.inputs.height")
    .setInputNode("cfg", "3.inputs.cfg")
    .setInputNode("sampler", "3.inputs.sampler_name")
    .setInputNode("scheduler", "3.inputs.scheduler")
    .setOutputNode("images", "9")
    .input("checkpoint", "SDXL/realvisxlV40_v40LightningBakedvae.safetensors", api.osType)
    .input("seed", seed())
    .input("step", 6)
    .input("width", 512)
    .input("height", 512)
    .input("batch", 2)
    .input("cfg", 1)
    .input<TSamplerName>("sampler", "dpmpp_2m_sde_gpu")
    .input<TSchedulerName>("scheduler", "sgm_uniform")
    .input("positive", "A close up picture of cute Cat")
    .input("negative", "text, blurry, bad picture, nsfw");

  return new Promise<string[]>((resolve) => {
    new CallWrapper(api, workflow)
      .onFinished((data) => {
        const url = data.images?.images.map((img: any) => api.getPathImage(img));
        resolve(url as string[]);
      })
      .run();
  });
};

const jobA = ApiPool.batch(Array(5).fill(generateFn), 10).then((res) => {
  console.log("Batch A done");
  return res.flat();
});

const jobB = ApiPool.batch(Array(5).fill(generateFn), 0).then((res) => {
  console.log("Batch B done");
  return res.flat();
});

console.log(await Promise.all([jobA, jobB]).then((res) => res.flat()));

🔍 Breakdown

Create a ComfyPool with multiple ComfyApi instances.
Set up event listeners for pool initialization, job additions, and client connections.
Define an async function (generateFn) that creates a workflow, sets its inputs, and executes it with a CallWrapper.
Use ApiPool.batch to run multiple jobs and wait for all batches to complete.

🔑 Authentication

import { ComfyApi, BasicCredentials, BearerTokenCredentials, CustomCredentials } from "@saintno/comfyui-sdk";

// Basic Authentication
const basicAuth = new ComfyApi("http://localhost:8189", "node-id", {
  credentials: { type: "basic", username: "username", password: "password" } as BasicCredentials
}).init();

// Bearer Token Authentication
const bearerAuth = new ComfyApi("http://localhost:8189", "node-id", {
  credentials: { type: "bearer_token", token: "your_bearer_token" } as BearerTokenCredentials
}).init();

// Custom Header Authentication
const customAuth = new ComfyApi("http://localhost:8189", "node-id", {
  credentials: { type: "custom", headers: { "X-Custom-Header": "your_custom_header" } } as CustomCredentials
}).init();

🔍 Breakdown

Import the necessary types from the SDK.
Create ComfyApi instances using the corresponding credential types: BasicCredentials, BearerTokenCredentials, and CustomCredentials..

📚 API Reference 📚

`ComfyApi`

🏗️ Constructor

constructor(host: string, clientId: string, opts?: { forceWs?: boolean, wsTimeout?: number, credentials?: BasicCredentials | BearerTokenCredentials | CustomCredentials; })

host: The base URL of your ComfyUI server.
clientId: A unique ID for WebSocket communication (optional). Defaults to a generated ID.
opts: Optional settings:
- forceWs: Boolean to force WebSocket usage.
- wsTimeout: Timeout for WebSocket connections (milliseconds).
- credentials: Optional authentication credentials.

⚙️ Methods

init(maxTries?: number, delayTime?: number): Initializes the client and establishes connection.
on<K extends keyof TComfyAPIEventMap>(type: K, callback: (event: TComfyAPIEventMap[K]) => void, options?: AddEventListenerOptions | boolean): Attach an event listener.
off<K extends keyof TComfyAPIEventMap>(type: K, callback: (event: TComfyAPIEventMap[K]) => void, options?: EventListenerOptions | boolean): Detach an event listener.
removeAllListeners(): Detach all event listeners.
fetchApi(route: string, options?: FetchOptions): Fetch data from the API endpoint.
pollStatus(timeout?: number): Polls the ComfyUI server status.
queuePrompt(number: number | null, workflow: object): Queues a prompt for processing.
appendPrompt(workflow: object): Adds a prompt to the workflow queue.
getQueue(): Retrieves the current state of the queue.
getHistories(maxItems?: number): Retrieves the prompt execution history.
getHistory(promptId: string): Retrieves a specific history entry by ID.
getSystemStats(): Retrieves system and device statistics.
getExtensions(): Retrieves a list of installed extensions.
getEmbeddings(): Retrieves a list of available embeddings.
getCheckpoints(): Retrieves a list of available checkpoints.
getLoras(): Retrieves a list of available Loras.
getSamplerInfo(): Retrieves sampler and scheduler information.
getNodeDefs(nodeName?: string): Retrieves node object definitions.
getUserConfig(): Get user configuration data.
createUser(username: string): Create new user.
getSettings(): Get all setting values for the current user.
getSetting(id: string): Get a specific setting for the current user.
storeSettings(settings: Record<string, unknown>): Store setting for the current user.
storeSetting(id: string, value: unknown): Store a specific setting for the current user.
uploadImage(file: Buffer | Blob, fileName: string, config?: { override?: boolean; subfolder?: string }): Uploads an image file.
uploadMask(file: Buffer | Blob, originalRef: ImageInfo): Uploads a mask file.
freeMemory(unloadModels: boolean, freeMemory: boolean): Frees memory by unloading models.
getPathImage(imageInfo: ImageInfo): Returns the path to an image.
getImage(imageInfo: ImageInfo): Returns the blob data of image.
getUserData(file: string): Get a user data file.
storeUserData(file: string, data: unknown, options?: RequestInit & { overwrite?: boolean, stringify?: boolean, throwOnError?: boolean }): Store a user data file.
deleteUserData(file: string): Delete a user data file.
moveUserData(source: string, dest: string, options?: RequestInit & { overwrite?: boolean }): Move a user data file.
listUserData(dir: string, recurse?: boolean, split?: boolean): List a user data file.
interrupt(): Interrupts the execution of the running prompt.
reconnectWs(opened?: boolean): Reconnects to the WebSocket server.

`CallWrapper`

🏗️ Constructor

constructor(client: ComfyApi, workflow: PromptBuilder<I, O, T>)

client: An instance of the ComfyApi client.
workflow: An instance of PromptBuilder defining the workflow.

⚙️ Methods

onPreview(fn: (ev: Blob, promptId?: string) => void): Set callback for preview events.
onPending(fn: (promptId?: string) => void): Set callback when job is queued.
onStart(fn: (promptId?: string) => void): Set callback when the job is started.
onOutput(fn: (key: keyof PromptBuilder<I, O, T>["mapOutputKeys"], data: any, promptId?: string) => void): Sets a callback for when an output node is executed.
onFinished(fn: (data: Record<keyof PromptBuilder<I, O, T>["mapOutputKeys"], any>, promptId?: string) => void): Set callback when the job is finished.
onFailed(fn: (err: Error, promptId?: string) => void): Set callback when the job failed.
onProgress(fn: (info: NodeProgress, promptId?: string) => void): Set callback for progress updates.
run(): Executes the workflow.

`PromptBuilder`

🏗️ Constructor

constructor(prompt: T, inputKeys: I[], outputKeys: O[])

prompt: The initial workflow data object.
inputKeys: An array of input node keys.
outputKeys: An array of output node keys.

⚙️ Methods

clone(): Creates a new PromptBuilder instance with the same configuration.
bypass(node: keyof T | (keyof T)[]): PromptBuilder<I, O, T>: Marks node(s) to be bypassed at generation.
reinstate(node: keyof T | (keyof T)[]): PromptBuilder<I, O, T>: Unmarks node(s) from bypass at generation.
setInputNode(input: I, key: DeepKeys<T> | Array<DeepKeys<T>>): Sets input node path for a key.
setRawInputNode(input: I, key: string | string[]): Sets raw input node path for a key.
appendInputNode(input: I, key: DeepKeys<T> | Array<DeepKeys<T>>): Appends a node to the input node path.
appendRawInputNode(input: I, key: string | string[]): Appends a node to the raw input node path.
setOutputNode(output: O, key: DeepKeys<T>): Sets output node path for a key.
setRawOutputNode(output: O, key: string): Sets raw output node path for a key.
input<V = string | number | undefined>(key: I, value: V, encodeOs?: OSType): Sets an input value.
inputRaw<V = string | number | undefined>(key: string, value: V, encodeOs?: OSType): Sets a raw input value with dynamic key.
get workflow: Retrieves the workflow object.
get caller: Retrieves current PromptBuilder object.

`ComfyPool`

🏗️ Constructor

constructor(clients: ComfyApi[], mode: EQueueMode = EQueueMode.PICK_ZERO)

clients: Array of ComfyApi instances.
mode: The queue mode using EQueueMode enum values.

⚙️ Methods

on<K extends keyof TComfyPoolEventMap>(type: K, callback: (event: TComfyPoolEventMap[K]) => void, options?: AddEventListenerOptions | boolean): Attach an event listener.
off<K extends keyof TComfyPoolEventMap>(type: K, callback: (event: TComfyPoolEventMap[K]) => void, options?: EventListenerOptions | boolean): Detach an event listener.
addClient(client: ComfyApi): Adds a new client to the pool.
removeClient(client: ComfyApi): Removes a client from the pool.
removeClientByIndex(index: number): Removes a client by index.
changeMode(mode: EQueueMode): Changes the queue mode.
pick(idx?: number): Picks a client by index.
pickById(id: string): Picks a client by ID.
run<T>(job: (client: ComfyApi, clientIdx?: number) => Promise<T>, weight?: number, clientFilter?: { includeIds?: string[]; excludeIds?: string[] }): Run a job with priority on an available client.
batch<T>(jobs: Array<(client: ComfyApi, clientIdx?: number) => Promise<T>>, weight?: number, clientFilter?: { includeIds?: string[]; excludeIds?: string[] }): Run multiple jobs concurrently.

🗂️ Enums

EQueueMode:
- PICK_ZERO: Selects the client with zero remaining queue.
- PICK_LOWEST: Selects the client with the lowest remaining queue.
- PICK_ROUTINE: Selects clients in a round-robin manner.

🗄️ Types

OSType:
- POSIX: For Unix-like systems.
- NT: For Windows systems.
- JAVA: For Java virtual machine.
TSamplerName: A union type of all available sampler names.
TSchedulerName: A union type of all available scheduler names.

🧩 Features

ManagerFeature: Provides methods to manage ComfyUI Manager Extension.

const api = new ComfyApi("http://localhost:8189").init();
await api.waitForReady();

if (api.ext.manager.isSupported) {
  await api.ext.manager.getExtensionList().then(console.log);
  // Check api.ext.manager for more methods
}

MonitoringFeature: Provides methods to monitor system resources using ComfyUI-Crystools Extension.

const api = new ComfyApi("http://localhost:8189").init();
await api.waitForReady();

if (api.ext.monitor.isSupported) {
  // For subscribing to system monitor events
  api.ext.monitor.on("system_monitor", (ev) => {
    console.log(ev.detail);
  });

  // For getting current monitor data
  console.log(api.ext.monitor.monitorData);
}

Note: Features require respective extensions (ComfyUI-Manager and ComfyUI-Crystools) to be installed.

📂 Examples

The examples directory contains practical demonstrations of SDK usage:

example-i2i.ts: Demonstrates image-to-image generation.
example-pool.ts: Demonstrates how to manage multiple ComfyUI instances using ComfyPool.
example-pool-basic-auth.ts: Demonstrates how to use ComfyPool with HTTP Basic Authentication.
example-t2i.ts: Demonstrates text-to-image generation.
example-t2i-upscaled.ts: Demonstrates text-to-image generation with upscaling.
example-img2img-workflow.json: Example workflow for image-to-image.
example-txt2img-workflow.json: Example workflow for text-to-image.
example-txt2img-upscaled-workflow.json: Example workflow for text-to-image with upscaling.

🤝 Contributing

Contributions are always welcome! Feel free to submit pull requests or create issues for bug reports and feature enhancements. 🙏

📜 License

This project is licensed under the MIT License - see the LICENSE file for more details. 📄

comfy-addons/comfyui-sdk

✨ ComfyUI SDK ✨

🧭 Table of Contents

🌟 Key Features 🌟

📦 Installation 📦

🚀 Getting Started 🚀

🎬 Basic Usage

🔍 Breakdown

🔄 Managing Multiple Instances with ComfyPool

🔍 Breakdown

🔑 Authentication

🔍 Breakdown

📚 API Reference 📚

ComfyApi

🏗️ Constructor

⚙️ Methods

CallWrapper

🏗️ Constructor

⚙️ Methods

PromptBuilder

🏗️ Constructor

⚙️ Methods

ComfyPool

🏗️ Constructor

⚙️ Methods

🗂️ Enums

🗄️ Types

🧩 Features

📂 Examples

🤝 Contributing

📜 License

🔄 Managing Multiple Instances with `ComfyPool`

`ComfyApi`

`CallWrapper`

`PromptBuilder`

`ComfyPool`