FluxML/Flux.jl

Adding Simple Recurrent Unit as a recurrent layer

Jafagervik opened this issue · 1 comments

Motivation and description

The recurrent.jl file already has implementations for RNN, LSTM and GRU layer, and I'm starting to work on implementing my own SRU to use in Julia as well. Its based on this paper. I thought this could be of general usage as well

Possible Implementation

I don't have any code yet so far for it, so heres the attached pseudocode for a single layer:

1711203661

This repo https://github.com/MartinuzziFrancesco/RecurrentLayers.jl
has a good collection on less common recurrent layers. You could contribute the layer there.