What is the weight_decay value used?

Question

What is the weight_decay value used?

look4pritam opened this issue 5 years ago · 9 comments

look4pritam commented 5 years ago

Default value for weight_decay is 0.0.
What is the weight_decay value used for training the model?

Answer 1 · 2020-05-22T13:07:15.000Z

What are the hyper parameters used for training the model?

Answer 2 · 2020-05-22T13:37:31.000Z

@look4pritam It's just zero as you see, i.e., no weight_decay is applied to the model.

Answer 3 · 2020-05-22T13:42:32.000Z

I was trying to reproduce your results using TensorFlow-2 version. Here generator loss is decreasing from 70. to 7. but discriminator loss is oscillating around 10.41. I am able to recreate input image with good quality, modified attributes are not getting applied. What may be the error?

Answer 4 · 2020-05-22T13:47:38.000Z

You can find implementation at https://github.com/look4pritam/TensorFlowExamples/blob/master/GAN/AttGAN/AttGAN-Attributes.ipynb

Answer 5 · 2020-05-22T14:07:18.000Z

@look4pritam I think you should print the losses separately, and find out which loss is abnormal so that you can check the corresponding part.

Answer 6 · 2020-05-22T14:13:49.000Z

Thanks for prompt reply. Are 60 epochs sufficient? Reconstructed image is good. Discriminator output is bad. I think more epochs are needed. But as suggested by you, I will check each losses to fix the problem. Thanks and Regards

…

On Fri, May 22, 2020, 19:37 LynnHo ***@***.***> wrote: @look4pritam <https://github.com/look4pritam> I think you should print the losses separately, and find out which loss is abnormal so that you can check the corresponding part. [image: image] <https://user-images.githubusercontent.com/16457298/82676172-846d8500-9c78-11ea-9992-2aca4a6619c1.png> [image: image] <https://user-images.githubusercontent.com/16457298/82676187-8afbfc80-9c78-11ea-8cc1-13586650f37e.png> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#38 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABJOVYWRR3JDVZXTHOUICK3RS2BKJANCNFSM4NHYOWKQ> .

Answer 7 · 2020-05-22T14:30:07.000Z

@look4pritam 60 epochs should work. Actually, you can see the editing effect in about 10 epochs.

Answer 8 · 2020-05-22T14:36:47.000Z

Thanks a lot, it will save time. I can train for epochs just more than 10 and see for attribute changing. Before that, I wiil find out problematic loss part as per your suggestion. Thanks a lot for your help.

…

On Fri, May 22, 2020, 20:00 LynnHo ***@***.***> wrote: @look4pritam <https://github.com/look4pritam> 60 epochs should work. Actually, you can see the editing effect in about 10 epochs. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#38 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABJOVYQ5TWTSE6M54XUD6NTRS2D75ANCNFSM4NHYOWKQ> .

Answer 9 · 2020-05-30T04:17:33.000Z

Thank you very much for your reply. Now I am able to train the model.