EleutherAI/oslo

fused_scale_mask_softmax on GPT2 model

loopinf opened this issue · 0 comments

loopinf commented

Describe a TODO feature

  • Current implementation does not use scale part on fused_scale_mask_softmax
  • Change it to use only not reorder_and_upcast part

Assignees