Initial values of mu and sigma

[david-bernstein]

According to the Eslami at al supplemental materials, it seems that line 45 of run_gqn.py should read

mu, sigma = mu_i, sigma_i

instead of

mu, sigma = mu_f, sigma_f

The subscripts appear to indicate initial and final values of the annealed quantities.

[wohlert]

Does it have any functional implications to change it from what it is currently?

[david-bernstein]

Not really. For the first epoch the learning rate is the final value (5e-5). After the first epoch it then jumps up to near the initial value and then begins its slow annealing descent. So for the first pass through the data the learning rate is too low. It is cosmetic as the first epoch doesn't really count for anything convergence-wise.