/unsuperior-ai-waifu

AI waifu that can run on your phone or PC

Primary LanguageJavaScriptMIT LicenseMIT

unsuperior-ai-waifu

Demo

Neuro-sama knockoff that runs completely in your browser. Should run on anything, but the page layout is janky on anything but a mobile phone.

Features

  • Completely free with given free credits.
  • Easy to run.
  • You can interact with her by poking her. I wouldn't recommend it though.
  • The mouths of normal VTubers, neurosama move based on the volume of their speech. USAW's mouth movements are more accurate because they're based off the phonetic sound she's making. Works, but WIP
  • She has various expressions that change depending on her emotion
  • Choose her accent
  • She has emotional voices (she can sound happy, sad, etc)

Goals

  • I want this project to be easy to use by most people. Ideally, all people should have to do is sign up for some websites, get API keys, and fill out the form on my website.
  • It should be easy to host for me. This is a bunch of static HTML/CSS/JS files so I can easily host it on GH pages.

How to use

See article for detailed guide on how to set this up

Open the index.html file in your modern browser OR use this link

You need to pass some parameters to the page through GET variables in the URL. Example:

https://hackdaddy.dev/usaw/?openai=sk-12345&engine=native&model=shizuki&prompt=She%20loves%20me%20even%20though%20I%27m%20not%20funny

Logic

Choose one of the following

GET Param
openai Your OpenAI API key
usaws URL to USAW server endpoint

Speech

GET Param Required?
engine What is used for speech recognition + TTS. Choose "azure" or "native. default "native"
speech_key Your Microsoft Azure speech key if engine="azure"
speech_region Your Microsoft Azure speech region if engine="azure"
voice Voice for TTS. Depends on whether or not you use Azure. See blog. no

Customization

GET Param Required?
model Name of waifu model. See /models.json yes
username Your name (Purely cosmetic) no
name your waifu's name no
prompt The base prompt that will be used when generating your waifu's dialogue. yes

How to add your own model

This software uses Live2D. It's the same program VTubers use for their avatars.

Available models are defined in models.json.

{
    "name": "arbitrary_but_unique_name",
    "description": "Description that shows up on the form on hackdaddy.dev",
    "url": "URL or file path to a .model.json file",
    "kScale": 0.000625, // multiplier to grow or shrink the model to make it fit the screen
    "kXOffset": 1150, // shifts model horizontally. Play around with both these values and see what works.
    "idleMotionGroupName": "Idle",
    "emotionMap": {
        "neutral": 0,
        "anger": 2,
        "disgust": 2,
        "fear": 1,
        "joy": 3,
        "sadness": 1,
        "surprise": 3
    }
},

This is what a .model.json file looks like:

{
  ...,
  "expressions": [
    {"name": "f01", "file": "expressions/f01.exp.json"},
    {"name": "f02", "file": "expressions/f02.exp.json"},
    {"name": "f03", "file": "expressions/f03.exp.json"},
    {"name": "f04", "file": "expressions/f04.exp.json"}
  ],
  ...
  "motions": {
    "idle": [
      {"file": "motions/idle_00.mtn", "fade_in": 2000, "fade_out": 2000},
      {"file": "motions/idle_01.mtn", "fade_in": 2000, "fade_out": 2000},
      {"file": "motions/idle_02.mtn", "fade_in": 2000, "fade_out": 2000}
    ],
    "tap_body": [
      {"file": "motions/tapBody_00.mtn", "sound": "sounds/tapBody_00.mp3"},
      {"file": "motions/tapBody_01.mtn", "sound": "sounds/tapBody_01.mp3"},
      {"file": "motions/tapBody_02.mtn", "sound": "sounds/tapBody_02.mp3"}
    ],
    ...
  }
}

idleMotionGroup will play one of the "idle" motions when she idles. emotionMap will show the respective expression when she feels an emotion. For example, if she feels anger, she will show the expression at index 2: "f03". Expressions are zero-indexed meaning "f01" is at index 0, "f01" is at index 1, etc.

This website gives you a bunch of models that are easy to add. See if there are any you like. Please help me by adding them for me.