Check this tutorial : https://youtu.be/aC13Qna-3cA
pip install torch==2.2.1 torchvision==0.17.1 torchaudio==2.2.1 --index-url https://download.pytorch.org/whl/cu121
pip install transformers
pip install timm
pip install packaging
pip install wheel
pip install ninja
pip install flash_attn # not on windows
Download wheel file from : https://github.com/bdashore3/flash-attention/releases
pip install flash_attn-2.5.9.post1+cu122torch2.2.2cxx11abiFALSE-cp311-cp311-win_amd64.whl
pip install einops
pip install accelerate
pip install peft
pip install matplotlib
(Fine tuned Trained model is available in model_checkpoints folder)