Kerbitat is a profiling tool for pridicting running time of a DNN model on NVIDIA GPUs. It can predict the performance of the target GPU from the profiling result of the source GPU.
pip install kerbitat- Kerbitat use KernelProfiler from habitat to get mangled kernel names of running DNN model.
- Kerbitat convert the mangled kernel names to feature vector.
- Kerbitat use a trained model to predict the performance of the target GPU from the feature vector.
- Kerbitat use the predicted performance to estimate the execution time of the target GPU. A Big Thanks to habitat team for providing KernelProfiler and all the other great thoughts.
from kerbitat import Kerbitat, GPU_TYPE
... # your own code
kerbitat_instance = Kerbitat(GPU_TYPE.A40)
kerbitat_instance.profiling(lambda: train_one_step(model, optimizer, x, y))
kerbitat_instance.get_target_time(GPU_TYPE.RTX2080Ti)
# 0.03143 sec per mini batch predicted on GPU RTX 2080 Ti
kerbitat_instance.convert_rate(GPU_TYPE.M40)
# The execution time on GPU M40 is 4.722 times longer than on GPU A40This work is based on CentML's Habitat, A very big thanks to them.
DeepView.Predict (aka Habitat) began as a research project in the EcoSystem Group at the University of Toronto. The accompanying research paper appeared in the proceedings of USENIX ATC'21. If you are interested, you can read a preprint of the paper here.
@inproceedings{habitat-yu21,
author = {Yu, Geoffrey X. and Gao, Yubo and Golikov, Pavel and Pekhimenko,
Gennady},
title = {{Habitat: A Runtime-Based Computational Performance Predictor for
Deep Neural Network Training}},
booktitle = {{Proceedings of the 2021 USENIX Annual Technical Conference
(USENIX ATC'21)}},
year = {2021},
}Kerbitat is part of Siyuan Zhang's undergraduate thesis at ShanghaiTech University.