MDK8888/GPTFast

Update PyTorch to 2.2

Closed this issue · 2 comments

AndreaPi commented

Very interesting work! I see you pinned torch==2.1.2 . PyTorch 2.2 promises a 2x improvement to scaled_dot_product_attention and a few torch.compile improvements:

https://pytorch.org/blog/pytorch2-2/

I don't think using PyTorch 2.2 will result in a big speed increase for GPTFast, but still it may be nice to try. What do you think?

MDK8888 commented

I definitely agree! Apologies for the late response-I will probably upgrade it in the next version, which will be out soon.

  • 👍1
MDK8888 commented

Hey, the new version does indeed have torch==2.2! Closing this now :)

  • 👍1