awf/functional-transformer

Issues

• Implementation of Attention is wrong.

  #8 opened 8 months ago by AlexanderMath
  1

• Make value heads nonsquare and add back head concatenation

  #7 opened 2 years ago by awf
  0

• Add non-learned position encodings

  #5 opened 2 years ago by awf
  0

• Initialization of positional encodings?

  #1 opened 2 years ago by btanner
  3

• Bug? Loss curves show a distinct correlation with batch id.

  #2 opened 2 years ago by awf
  0