/jetsonGPT

Using FastChat-T5 Large Language Model, Vosk API for automatic speech recognition, and Piper for text-to-speech

Primary LanguagePython

How to Run a ChatGPT-Like LLM on NVIDIA Jetson Xavier NX board(Hackster.io)

JetsonGPT is a python based voice assistant that takes two different wake up words running on the Nvidia Jetson Xavier NX. One for the activation of VOSK API Automatic Speech recognition and the other will prompt the FastChat-T5 Large Larguage Model to generated answer based on the user's prompt. For transcribing user's speech implements Vosk API. Text-to-speech is done using Piper TTS.

Main Requirements

  • python >= 3.7
  • numpy
  • fastapi
  • espeak_phonemizer
  • uvicorn
  • onnxruntime-gpu
  • vosk

Usage

  • Download a Piper TTS model voice from here and extract the .onnx and .onnx.json files.

  • Download the Vosk model for ASR from here.

  • Open a terminal and run Piper TTS server program

python3 webserver.py
  • Open another terminal and run a main program
python3 chat.py

Acknowledgements

The implementation of the project relies on:

I thank the original authors for their open-sourcing.