/ModernFastAPI

Repo of the code from the Medium article - Build a powerful LLM API right on your computer

Primary LanguagePython

ModernFastAPI

Repo of the code from the Medium article - Build a powerful LLM API right on your computer

This Project has 3 parts:

  1. Create your first FastAPI and interact with it
  2. Create a Streamlit AI app where you use TinyLlama-1B-OpenOrca as Instruction AI you can reach in your Local Network
  3. Use llama-cpp-python built in API and Streamlit to give your Team a nice Chatbot (coming soon0

Here the articles on Medium

Part 3 file

This is the python file for the textual interface as described in the part 3 article

The result will be as shown below (terminal server llama-cpp-python on the left, chat interface on the right):