This library provides utility for the training and evaluation of NLP models. It provides
several abstractions to work with namely NLPModel
and NLPDataset
, that would (in theory)
allow for training and evaluation of many models across various datasets.
- Clone the repo locally
$ git clone https://github.com/patrickjohncyh/nlp_ml_api.git
- Navigate to repository
$ cd nlp_ml_api
- Install using pip:
$ pip install .
from nlp_ml_api.datasets import C19TwitterSentimentDataset
from nlp_ml_api.Modeler import Modeler
# load dataset
dataset = C19TwitterSentimentDataset.from_local('datasets/corona_nlp')
# load Modeler with model and dataset
modeler = Modeler('CountModel', dataset)
# train
modeler.train()
# evaluate on test data
modeler.evaluate(split='test')
# get model from Modeler
model = modeler.model
# invoke Model locally
model.predict('Quarantine Day 5: I forgot how the outside world looks like anymore')
You can invoke the model either in python (e.g. notebook) or locally (localhost) or deploy the model as a Heroku app and make a HTTP request at the endpoint.
model.predict('I like cats')
We build a docker container, containing a FastAPI app which loads the model and handles requests.
deploy_model(model, mode='localhost')
Deployment to Heroku is more involved and assumes that you have Heroku CLI
installed, and have logged in (i.e. heroku login
). Once that is done, you can deploy the model
via deploy_model
function. Under the hood, python issues commands via a Makefile
.
deploy_model(model, mode='heroku', endpoint_name='tweet_sentiment_nlp_endpoint')
To kill the endpoint, issue in command-line from project root,
HEROKU_APP_NAME=<ENDPOINT-NAME> make -C endpoint delete_app
At a high level there are two main abstractions:
This abstraction implements a type of classification model. To create a new model,
you should inherit this class and implement predict
and fit
functions. We provide
several models out of the box:
CountModel
: Classification model based on token counts and logistic regressionTFIDFModel
: Classification model based on tf-idf and logistic regressionTransformerModel
: Transformer based model (Trainable on Colab!)
Optionally, each NLPModel
has an NLPModelParams
dataclass which allows for changing
of hyper-parameters.
This abstraction wraps a dataset and provides a unified interface for models to interact with.
We provide C19TwitterSentimentDataset
dataset out of the box.
Modeler
takes in anNLPDataset
and an NLPModel
, and provides functionality
to train
and evaluate
the model on given dataset. It also provides utility to save
and load a model after training.
Model is deployed by wrapping it with a FastAPI app and dockerizing it. We are able to call the model locally by building
the docker image on the local machine and can also serve the docker image as a Heroku app. The main steps of
serving on Heroku can be found here. We use a
Makefile
to compose commands for ease of use.
- Metaflow/Ray: While local machines may not have GPUs locally, Metaflow / Anyscale(Ray) can provide preemptable GPU resources specifically for the training step for large models (e.g. Transformers).
- Deployment: Deployment currently relies on model being
git push-able
, which is impractical for models which large memory footprint. - Enable custom metrics for evaluation: Evaluation is hard-coded, but could potentially allow uers to pass in their own evaluation functions if required.
- Improve abstractions to suit NLP use-case: There are often standard steps in NLP (e.g. pre-processing) which
can be made more explicit as part of the
NLPModel
abstraction. - Improve debugging / reporting