This is an LLM-powered chat interface integrated with voice synthesis model, web search-based RAG, and python environment.
Everything is running locally in your computer, so it doesn't require any API keys.
For better results, I strongly recommend you to select a model large enough or trained for tool use.
1. Streaming output.
2. Context shifting manipulating KV-Cache significantly reducing evaluation time (called StreamingLLM).
3. Automatic tool use (aka. function calling) involving "Web Search", "Python Interpreter".
4. Integration to Japanese text-to-speech model called style-bert-vits2.
5. Image recognition utilizing Image-to-Text model.
1. Jupyter must be installed.
$ pip install jupyterlab2. Activate ipywidgets by one of following commands:
2-1. jupyter-lab
$ jupyter labextension install @jupyter-widgets/jupyterlab-manager2-2. jupyter notebook / Google Colab
$ jupyter nbextension enable --py widgetsnbextension3. Install dependencies.
$ pip install -r requirements.txt4. (Optional) You might want to utilize hardware acceleration (cuda/mps).
In that case, you need to reinstall llama-cpp-python in the way corresponding to your device.
Installation instruction can be found here llama-cpp-python
5. (Optional) You might want to unlock websocket message size limit to send large files or to play longer sounds (the limit is 10MB by default).
$ jupyter notebook --generate-configAfter running the command, edit following line of the config file:
c.ServerApp.tornado_settings = {"websocket_max_message_size":100*1024*1024}
