🍌 Banana Serverless
This repo gives a basic framework for serving ML models in production using simple HTTP servers.
Quickstart:
The repo is already set up to run a basic HuggingFace BERT model.
- Run
pip3 install -r requirements.txt
to download dependencies. - Run
python3 server.py
to start the server. - Run
python3 test.py
in a different terminal session to test against it.
Make it your own:
- Edit
app.py
to load and run your model. - Make sure to test with
test.py
!
if deploying using Docker:
- Edit
download.py
(or theDockerfile
itself) with scripts download your custom model weights at build time.
Move to prod:
At this point, you have a functioning http server for your ML model. You can use it as is, or package it up with our provided Dockerfile
and deploy it to your favorite container hosting provider!
If Banana is your favorite GPU hosting provider (and we sure hope it is), read on!
🍌
Deploy to Banana Serverless:
Four steps:
- Create your own copy of this template repo. Either:
- Click "Fork" (creates a public repo)
- Click "Use this Template" (creates a private or public repo)
- Create your own repo and copy the template files into it
-
Install the Banana Github App to your new repo.
-
Get your Banana API Key by logging in here.
-
Email us at
onboarding@banana.dev
with the following message:
Hello, I'd like to be onboarded to serverless.
My github username is: YOUR_GITHUB_USERNAME
My Banana API Key is: YOUR_API_KEY
My preferred billing email is: YOU@EMAIL.COM
Your github username, banana api key, and email are required for us to authorize you into the system.
We will reply and confirm when you're added.
From then onward, any pushes to the main
branch trigger Banana to build and deploy your server, using the Dockerfile.
Throughout the build we'll sprinkle in some secret sauce to make your server extra snappy
It'll then be deployed on our Serverless GPU cluster and callable with any of our serverside SDKs:
Use Banana for scale!