OpenVino GPT-2
- Python3
- Decent hardware - for now, we rely only on CPU, but OpenVino can utilze your iGPU if present
This should work on Windows, Linux and macOS
-
Create a venv. Follow instructions here to setup and activate the environment https://docs.python.org/3/library/venv.html
-
Clone the repository
git clone https://github.com/yousseb/ov-gpt-2
cd ov-gpt-2
- Install requirements
pip3 install -r requirements.txt
- Run
python3 main.py
The first time you run the application, it will
- Download the huggingface pytorch GPT2 model
- Convert it to ONNX model
- Convert the ONNX model to OpenVino IR
- Download vocab.json and merges.txt required for GPT2
- Start chatting