some confusions

[leileilin]

hey man,
i got confused about the training process of the electra model,
in my point, the generator should be trained first, but in your code, the generator and discriminator is trained together.
can u tell me why?
thank u

[nawnoes]

Hello, leileilin
I understand electra model pretrain generator and discriminator together.
So, loss is sum of generator loss and discriminator loss using each weight:

# return weighted sum of losses
total_loss = self.gen_weight * gen_loss + self.disc_weight * disc_loss

Thank you :)

[leileilin]

Hello, leileilin I understand electra model pretrain generator and discriminator together. So, loss is sum of generator loss and discriminator loss using each weight:

# return weighted sum of losses
total_loss = self.gen_weight * gen_loss + self.disc_weight * disc_loss

Thank you :)

hey man,
Thank you very much. I see.
But if I want to train generator first, then use the trained generator to do sample for the trainging of the discrimator.
are u done expriments for that?
thank u.