This is a simple API that prompts Stanford's Alpaca7B, a fine-tuned version of Meta's LLaMa 7B, and gives back the response. It's designed to run on Colab.
Open the notebook on Colab. Execute each cell. The last cell will take a couple of minutes. Once everything is done, copy the ngrok link that has been generated.
To prompt the language model, send a json POST request to the /ask endpoint, including a json structured like this:
{"input":"<your prompt>"}
A quick command to test it is
curl -H "Content-Type: application/json" -X POST -d "{\"input\":\"What are Large Language Models?\"}" <your_ngrok_link>/ask
I also included a very simple web app to prompt the LLM, just run it using python3 webapp.py. Don't forget to change the ngrok link with the one you generated.