/bert_qa_api_base_image

Primary LanguagePython

Huggineface BERT-QA API base Image

Author: Teng Fu

AIM

It is for Bert-QA development purpose, it consists following feature:

  • fastAPI, gunicorn and uvicorn for API development

  • PyTorch 1.6.0

  • Cuda 10.1 and cuDNN 7

  • Huggingface Transformer library

This allows huggingface BERT models able to:

  • execute with GPU-enabled

  • provide REST API service through fastAPI

Base Images

This image is based on:

pytorch-cuda image is the main base image. In this main base image, the Python version is 3.8.5.

Usage

Make sure the nvidia driver is >=R418

sudo docker run -it --runtime=nvidia huggingface-torch-gpu-fastapi:0.1 sh