/ST-WebUI-Diffusion

Alternative Stable Diffusion Extension for SillyTavern

Primary LanguageJavaScript

ST-WebUI-Diffusion

Modified SillyTavern Stable Diffusion Extension. Utilises SD Forge.

Additions:

  1. ControlNet Reference.
  2. ControlNet FaceID (InstantID).
  3. ADetailer
  4. Character specific settings.
  5. Integrated Forge Interrogator (gather prompts from avatar - no working yet. On todo list)
  6. Uses expressions (if on) in image prompt.
  7. Different prompt to request LLM for keywords as existing prompt didn't work well for me.
  8. Integrated FreeU Integrated (SD Forge only) with recommended settings for model type (noticeable difference in images)
  9. Optional - Custom write your own AlwaysOnScript json for each character (disables all AOScript settings)

Removed:

  1. All remote API's and their options.
  2. ComfyUI
  3. Styles.

Reason for extension

Is a personal experiment to see if it's possible to create consistent characters dynamically within a generated story. SillyTavern is a good base environment to try this.

Image below: ControlNet (FaceID 0.7 weight), avatar interrogated keywords and FreeU. SD Model: MklanAnime with E-ESRGAN 4x+ Anime6B upscale. These 3 images are not cherry picked. They were generated one after the other without regeneration or change of settings with LLM generated keywords from last message. No lora or character specific embeddings are used.

To Install

Read this carefully. It's easy but also easy to fuck up.

  • Go to your SillyTavern Install Folder
  • Then to /public/scripts/extensions
  • replace the stable-diffusion folder with the one from this repo.

Important!

You must also install the server plugin. You need to drag and drop this into the ./Plugins folder and restart SillyTavern https://github.com/vincedundee/ST-Diffusion-Server-Plugin

----------------------------------------------------------

Tested with: SillyTavern 1.12.4 (local) SD Forge 1.10.1 (local)

Improvements that can be made...

  • AnimateDiff or SVD can be added in or another extension can be made to turn images to short videos.
  • Can probably generate expressions...not sure how interesting this is
  • Generate the sway/animate thingy

Support

Unlikely. Look at the wiki on how to set up: https://github.com/vincedundee/ST-WebUI-Diffusion/wiki/1.-How-to-setup-SD-Forge-and-SillyTavern