omarsar/nlp_overview

Can't find the equation labels + typos

Daynity opened this issue · 3 comments

For example, in the 'D. Contextualized Word Embeddings' session, you wrote 'In Eq. 8, s_{j}^{task} ...'. Although we could figure it out from the context. :)

And a typo in LSTM equations. The input of eq. $c_t = ...$ should be lower case vector x, not the upper case.

GRU equations

$\bold{z}_t = \sigma(U_z \cdot \bold{x}t + W_z \cdot \bold{h}{t-1})$

$\bold{r}_t = \sigma(U_r \cdot \bold{x}t + W_r \cdot \bold{h}{t-1})$

$\bold{s}_t = \tanh ( U_s \cdot \bold{x}t + W_s \cdot (\bold{h}{t-1} \bigodot \bold{r}_t))$

$\bold{h}_t = (1- \bold{z}_t) \bigodot \bold{s}_t + \bold{z}t \bigodot \bold{h}{t-1}$

I added footnotes to $\bold{z}$ and $\bold{r}$ in order to make it clear that they are the results for time $t$.

And in the the third equation parameters for $\bold{x}_t$ should be $U_s$ rather than $U_z$.

Hi Daynity! Thanks for the comments. Is it possible you put these changes in a PR?