g_loss function
Opened this issue · 2 comments
According to the paper, the objective of RL is to get a larger reward, if the output of the discriminative model gets larger, means that D see generated as a true one.
However, we found that there are some questions in the following code snippet :
self.gan_loss = -tf.reduce_mean(tf.log(self.i_prob) * self.reward) + self.lamda * ( tf.nn.l2_loss(self.u_embedding) + tf.nn.l2_loss(self.i_embedding) + tf.nn.l2_loss(self.i_bias))
Q1. Ignore the regulation, the tf.log(self.i_prob) should be negative and self.reward may be negative or positive, thus, when reward is positive the above function will minimize value of the reward, which is opposite to the RL design statement and concept of GAN.
Could you explain the reason why self.reward may be negative or positive?
Please correct me if there's anything wrong, thank you!
Have same question !
I think the learning goal of G is not to maximize the reward, since the reward is detached anyway. Rather it will learn to assign high probabilties for samples which give high reward to achieve minimization of log(~1) = ~0 which is consistent with the overall objective.