/AVeryComfyNerd

ComfyUI related stuff and things

Primary LanguageJupyter NotebookMIT LicenseMIT

Overview

A variety of ComfyUI related stuff. You'll need different models and custom nodes for each different workflow.

Trobleshooting

  • By default, models are saved in subdirectories under ComfyUI/models, though some custom nodes have their own models directory.
  • Use ComfyUI Manager to install missing custom nodes by clicking "Install Missing Custom Nodes"
  • If ComfyUI Manager can't find a node automatically, use the search feature
  • When troubleshooting, do all of these steps in order
  • Be sure to keep ComfyUI updated regularly - including all custom nodes. Old versions may result in errors appearing.
  • These are just workflows - no custom nodes here, so no code!
  • Don't mix SDXL and SD1.5 models unless stated (such as SDXL needing the SD 1.5 vision model) - chances are you'll get an error!
  • Sometimes custom nodes make breaking changes and the same node may need to be re-created. Typically the custom node GitHub page has such information.
  • Sometimes custom nodes just break! Check the github page for the custom node causing any issues for more information and to raise issues
  • Sometimes custom nodes change functionality, such as:
    • Segement anything - mask output inverted
    • Dynamic Thresholding - output different
  • Not sure where the GitHub page is for a custom node? You can click on it via ComfyUI Manager
  • Custom node still red after installing it? Remember to restart ComfyUI!
  • Custom node still giving an error? Check the GitHub page for that custom node
  • The Microsoft Windows portable version of ComfyUI apparently has issues with various custom nodes, whereas normal installs are OK. Unknown error? Try a normal install!
  • Need more help? See this Playlist with loads of ComfyUI guides

Resources

Item Description Link
ComfyUI The main thing you'll need! https://github.com/comfyanonymous/ComfyUI
See https://youtu.be/2r3uM_b3zA8 for an install guide
ComfyUI Manager Install any missing nodes using this https://github.com/ltdrdata/ComfyUI-Manager
Stability AI Models & VAEs https://huggingface.co/stabilityai
Text-to-Image models Text-2-image models https://huggingface.co/models?pipeline_tag=text-to-image&sort=trending
SSD-1B Text2-image model https://huggingface.co/segmind/SSD-1B
ControlNet Models ControlNet Models https://huggingface.co/lllyasviel/sd_control_collection/tree/main
https://huggingface.co/comfyanonymous/ControlNet-v1-1_fp16_safetensors/tree/main
QR Code Monster Control Net ControlNet Model https://huggingface.co/monster-labs/control_v1p_sd15_qrcode_monster
T2IAdapter Github Repo https://github.com/TencentARC/T2I-Adapter
IP Adpater Github Repo https://github.com/tencent-ailab/IP-Adapter
Control LoRA Control Models https://huggingface.co/stabilityai/control-lora
AnimateDiff Original repo, many links and more info https://github.com/guoyww/AnimateDiff
Original IPAdapter Extension Custom Nodes for ComfyUI https://github.com/laksjdjf/IPAdapter-ComfyUI
IPAdapter models Models https://huggingface.co/h94/IP-Adapter
Latent Consistency Models Models https://huggingface.co/latent-consistency
Upscale Wiki Many models & info https://upscale.wiki/wiki/Main_Page
Artist Style Studies SDXL Prompt output examples for inspiration https://sdxl.parrotzone.art/

Updating / Installing Custom Nodes

  • Use ComfyUI Manager to install and update custom nodes with ease!
  • Click "Install Missing Custom Nodes" to install any red nodes
  • Be sure to keep ComfyUI updated regularly - including all custom nodes. Old versions may result in errors appearing.

Workflows available

In ComfyUI the image IS the workflow. Simply drag or load a workflow image into ComfyUI!

Workflow Description Version
Basic SDXL ControlNet workflow.
Introductory SDXL Canny & Depth ControlNet example.
See https://youtu.be/reqamcrPYiM for more information.		 SDXL
Basic QR Code Monster SD 1.5 ControlNet - make spiral art!
See also - https://youtu.be/D4oJz0w36ps		 SD 1.5
QR Code Monster SD 1.5 ControlNet - make animated spiral art!
See also: https://youtu.be/D4oJz0w36ps		 SD 1.5
Updated QR Code Monster SD 1.5 ControlNet with AnimateDiff and FreeU
Works best with the v1 QR Code Monster - https://huggingface.co/monster-labs/control_v1p_sd15_qrcode_monster		 SD 1.5
AnimateDiff with Montion LoRA example. Pan up, down, left right, etc. SD 1.5
Instant LoRA 1
Inspired by AloeVera (almost identical).
Really simple, no training, "LoRA" like functionality.
SD 1.5. IP Adapter models:
1. https://huggingface.co/h94/IP-Adapter/blob/main/models/ip-adapter-plus_sd15.bin -> custom_nodes/IPAdapter-ComfyUI/models.
2. https://huggingface.co/h94/IP-Adapter/blob/main/models/image_encoder/model.safetensors -> models/clip_vision.
Video guide - https://youtu.be/HtmIC6fqsMQ		 SD 1.5
Instant LoRA 2
As above, but with ControlNet to guide the shape		 SD 1.5
Instant LoRA 3
As above, but with QR Code Monster ControlNet too :)		 SD 1.5
Instant LoRA 4
As above, but with basic upscaling		 SD 1.5
Instant LoRA 5
As above, but with more upscaling to 16k+		 SD 1.5
Instant LoRA 6
As above, but different upscaling to 16k+		 SD 1.5
Morphing AI videos of any length using AnimateDiff. SD 1.5. Includes IPAdapter & Upscaling. IP Adapter models:
1. https://huggingface.co/h94/IP-Adapter/blob/main/models/ip-adapter-plus_sd15.bin -> custom_nodes/IPAdapter-ComfyUI/models.
2. https://huggingface.co/h94/IP-Adapter/blob/main/models/image_encoder/model.safetensors -> models/clip_vision.
Video guide - https://youtu.be/6A3a0QNPhIs		 SD 1.5
Morphing AI videos of any length using AnimateDiff. SD 1.5. Includes Upscaling. Like above, but without IPAdapter controls. SD 1.5
SDXL "Instant LoRA" - basic.
Really simple, no training, "LoRA" like functionality.
Uses SDXL IP Adapter - https://huggingface.co/h94/IP-Adapter
Video - https://youtu.be/dGL02W4QatI		 SDXL
SDXL "Instant LoRA" - with CLIP Vision
Uses SDXL IP Adapter - https://huggingface.co/h94/IP-Adapter
Also use "Revisions" CLIP vision - https://huggingface.co/stabilityai/control-lora		 SDXL
SDXL "Instant LoRA" - with CLIP Vision & ControlNet
Uses SDXL IP Adapter - https://huggingface.co/h94/IP-Adapter
Also use "Revisions" CLIP vision - https://huggingface.co/stabilityai/control-lora		 SDXL
AnimateDiff + QRCode (Vid2Vid)
Use any high-contrast input video to create guided animations! Spirals away...		 SD 1.5


SD 1.5 Reposer (3 versions) - single face image to any pose. Get consistent faces!
No "roop" or similar face-swapping nodes required = easy install!
SD 1.5 ControlNet models:
https://huggingface.co/comfyanonymous/ControlNet-v1-1_fp16_safetensors/tree/main
IP Adapter models:
1. https://huggingface.co/h94/IP-Adapter/blob/main/models/ip-adapter-plus-face_sd15.bin -> custom_nodes/IPAdapter-ComfyUI/models.
2. https://huggingface.co/h94/IP-Adapter/blob/main/models/image_encoder/model.safetensors -> models/clip_vision.
IPAdapter Extension - https://github.com/laksjdjf/IPAdapter-ComfyUI
Updated due to node changes - now also with fewer noodles!
Reposer, Reposer Plus and Reposer Plus BG
Reposer Basic Video guide - https://youtu.be/SacK9tMVNUA
Reposer Plus Video guide - https://youtu.be/ZcCfwTkYSz8		 SD 1.5
SD 1.5 Video Styler! Combining IPAdapter with Video-to-video for strange styles and weird animations
Uses https://github.com/cubiq/ComfyUI_IPAdapter_plus
The pre-trained models are available on huggingface, download and place them in the ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/models directory.
For SD1.5 you need:
* ip-adapter_sd15.bin
* ip-adapter_sd15_light.bin
* ip-adapter-plus_sd15.bin
* ip-adapter-plus-face_sd15.bin
Additionally you need the image encoder to be placed in the ComfyUI/models/clip_vision/ directory.
They are the same models used by the other IPAdapter custom nodes ;) - symlinks are your friend!		 SD 1.5
SDXL version of Reposer using the SDXL "IPAdapter Plus Face" model
Pick a face then add a body in any pose - no training!
Works with photorealistic faces, anime faces, cartoon faces, etc		 SDXL
SSD-1B Workflow - SDXL for 8GB VRAM cards!
Model - https://huggingface.co/segmind/SSD-1B
Video - https://youtu.be/F-bKndyQ7L8		 SSD-1B
LCM LoRA vs Normal 1.5, SDXL, SSD-1B

Custom Node List

The current custom node list is shown in the image below. This is a complete list. Not all nodes are used in workflows.
Install custom nodes using ComfyUI manager