This script allows you to convert Hugging Face's T5/GPT-2 models to Core ML models. Core ML is a framework provided by Apple that enables deploying machine learning models on Apple devices.
- Python 3.10
- ane_transformers==0.1.1
- coremltools==7.0b1
- numpy==1.24.2
- tensorflow==2.12.0
- torch==2.0.0
- transformers==4.29.2
- transformers-exporter
git clone https://github.com/huggingface/exporters.git && cd exporters && pip install -e .
- Clone the repository:
git clone https://github.com/kendreaditya/neural-engine-benchmark
-
Install the required dependencies using pip:
pip install -r requirements.txt
To convert a T5/GPT-2 model to a Core ML model, follow these steps:
-
Open a terminal and navigate to the project directory.
-
Run the script with the desired T5/GPT model name as the argument.
Available T5 models are:
t5-small
,t5-base
,t5-large
,t5-3b
,t5-11b
Available GPT-2 models are:gpt2
,gpt2-medium
,gpt2-large
,gpt2-xl
For example, to convert the
t5-base
model, run:python t5-models.py t5-base
-
The script will convert the T5 model to a Core ML model and save it as a
.mlmodel
file. -
The converted Core ML model will be saved in the current directory with the following naming convention:
-
To verify performance, launch Xcode and simply add this model package file as a resource in their projects. From the Performance tab, you can generate a performance report on locally available devices, for example, on the Mac that is running Xcode or another Apple device that is connected to that Mac.
<model_name>-<num_params>.mlmodel
For example, the t5-base
model with 220 million parameters will be saved as t5-base-220m.mlmodel
.
- This script is intended for converting T5 models to Core ML models only.
- Make sure you have enough disk space available as the converted models can be large in size.
This project is licensed under the MIT License. See the LICENSE file for details.
- This script is based on the work of Apple Docs, Apple Blog, FastT5 and Hugging Face.