Rishit-dagli/Perceiver

Create a FeedForward layer

Rishit-dagli opened this issue · 0 comments

Create a simple FeedForward layer as a tf.keras.layers.Layer which should essentially contain a Dense layer with the modified GELU activation (#2 ), optionally I could also include a dropout layer and another Dense layer which should have the number of neurons equal to the dimension