chnsh/DCRNN_PyTorch

Double sigmoid inside RNNCell

razvanc92 opened this issue · 4 comments

razvanc92 commented

I was looking at the implementation of the DCGRUCell, and I've spotted something out of order. If we are using 'fc' for U and R gates we are going to apply the sigmoid twice (line 121 and 96). Is this how it's intended to work, or is there a bug?

yuqirose commented

could you share a screenshot?

razvanc92 commented

@yuqirose Sure. If we look at the following image on line 96 the value is sigmoid(fn(...)).

ss1

Now if we look at the next picture (line 121) we are applying again sigmoid.
ss2

Am I missing something?

yuqirose commented

@razvanc92 line 121 is applied to Wx, line 96 is applied to f(Wx) + b

razvanc92 commented

@yuqirose Thank you for your replay. Following the code we are doing sigmoid(sigmoid(Wx) + b) which is different from the original implementation of GRU which uses sigmoid(Wx +b).

  • 👀1