/LocalAIVtuber

A tool for hosting AI vtubers that runs fully locally and offline.

Primary LanguagePython

Local AI Vtuber (A tool for hosting AI vtubers that runs fully locally and offline)

Full demo and setup guide: https://youtu.be/Yl-T3YgePmw?si=n-vaZzClw0Q833E5

  • Chatbot, Translation and Text-to-Speech, all completely free and running locally.
  • Support voice output in Japanese, English, German, Spanish, French, Russian and more, powered by RVC, silero and voicevox.
  • Includes custom finetuned model to avoid generic chatbot responses and breaking character.
  • Gradio UI web interface.
  • plugin support for easily adding other providers.

Installation

Manual setup (Tutorial video: Full demo and setup guide)

install python 3.10 https://www.python.org/downloads/release/python-3100/

install CUDA toolkit 12.4 https://developer.nvidia.com/cuda-12-4-0-download-archive

install visual studio and add desktop development with C++ component https://visualstudio.microsoft.com/downloads/

Screenshot 2024-10-03 100032

1. Download the project from releases

2. open command prompt in project folder.

3. Create environment

python -m venv venv
.\venv\Scripts\activate

(If you encounter an error that says “cannot be loaded because the execution of scripts is disabled on this system. Open powershell with admin privilage and run Set-ExecutionPolicy RemoteSigned)

4. Install packages

pip install -r requirements.txt
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
pip install llama-cpp-python

pip install nltk
python -m nltk.downloader -d C:\nltk_data all

5. Start Program

python main.py

When you see this message, go to http://localhost:7860 to see web UI 
```
Running on local URL:  http://127.0.0.1:7860
To create a public link, set `share=True` in `launch()`.
```

Notes:

To start the program again, run:

.\venv\Scripts\activate
 python main.py

If you have a decent GPU, You can install the GPU version of llama-cpp-python:

$env:CMAKE_ARGS ="-DGGML_CUDA=ON"
 pip install llama-cpp-python --force-reinstall --no-cache-dir --verbose --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/124

This can improve latency further.

One click setup (Outdated and may not work)

  1. Download the project from releases
  2. Extract and double click run.bat
  3. When you see this message, go to http://localhost:7860 to see web UI
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.

TODO (This project is still under development and more features are planned)

  • Fetch chat input from streaming platforms (Finished)
  • Improve local LLM (Finetuned model avaliable https://huggingface.co/xiaoheiqaq/Aya-7b-gguf)
  • Write plugins for cloud providers(Azure tts, elevenlabs, chatgpt, whisper...)
  • GPU support (Finished)
  • Vtube studio integration (Finished)
  • Let AI play games and provide commentary. (can currently play chess and keep talking nobody explode)
  • AI singing

FAQ:

  • To fetch chat from Youtube, copy the youtube_video_id from the stream url like this:

image

Then press start fetching chat

image