Failed to reproduce the results and Loss soar----pytorch

[catherinezll95]

Hi, thank you for opening up the code. In the process of running the program implemented in pytorch, I encountered several problems, I hope to get your answer.

1. Failed to reproduce the results.

---

For example:
1)Res. sup. (MA)
python src/translate.py --residual_velocities --learning_rate 0.005

the results I get:
global step:10000 learning rate: 0.0047

milliseconds | 80 | 160 | 320 | 400 | 560 | 1000 |
walking | 0.374 | 0.636 | 0.878 | 0.980 | n/a | n/a |
eating | 0.263 | 0.457 | 0.710 | 0.864 | n/a | n/a |
...

In your paper
milliseconds | 80 | 160 | 320 | 400 |
walking | 0.28 | 0.49 | 0.72 | 0.81 |
eating | 0.23 | 0.39 | 0.62 | 0.76 |
...

2)Sampling-based loss (SA)
python src/translate.py --action walking --seq_length_out 25

the results I get:
global step:10000 learning rate: 0.0047

milliseconds | 80 | 160 | 320 | 400 | 560 | 1000 |
walking | 0.60 | 0.607 | 0.808 | 0.889 | 1.029 | 1.186 |

In your paper
milliseconds | 80 | 160 | 320 | 400 |
walking | 0.92 | 0.98 | 1.02 | 1.20 |

---

2.When the global step>10000, the loss will soar.

Res. sup. (MA)
Global step = 10000

milliseconds | 80 | 160 | 320 | 400 | 560 | 1000 |
walking | 0.374 | 0.636 | 0.878 | 0.980 | n/a | n/a |
eating | 0.263 | 0.457 | 0.710 | 0.864 | n/a | n/a |
...

Global step = 10200

milliseconds | 80 | 160 | 320 | 400 | 560 | 1000 |
walking | 1.559 | 2.374 | 3.420 | 3.835 | n/a | n/a |
eating | 1.271 | 1.732 | 2.439 | 2.789 | n/a | n/a |
smoking | 1.464 | 2.132 | 3.167 | 3.508 | n/a | n/a |
discussion | 1.828 | 2.441 | 3.116 | 3.379 | n/a | n/a |
...

[yuyujunjun]

I tried pytorch version and I encountered the problem 2, too. At line 211 of translate.py, changing the optimiser from Adam to SGD will solve the problem 2. The loss exactly will grow up when iterations are too large but do not soar anymore. In #22, the author suggests the iterations might be 10000 or 20000 so it's not a big problem.
My experiment:
python src/translate.py --residual_velocities --learning_rate 0.005 --action walking