pkuxmq/Invertible-Image-Rescaling

About hyper-parameters lambda

codyshen0000 opened this issue · 5 comments

codyshen0000 commented

Hi
In Equation 10 of the paper, lambda_1 and lambda_3 are 1, but lambda_2 is set to 16. why setting lambda_2 so large?

pkuxmq commented

Hi, because we calculate the sum of losses on all pixels, so we simply set lambda_2 = s^2 (where s is the scale) to balance the weight for HR images and LR images.

  • 👍1
codyshen0000 commented

Thanks for your reply!
Since it is used for balancing HR and LR, I'm curious why not choose to increase the reconstruction weight of lambda_1?

pkuxmq commented

Hi, because HR images have s^2 times the number of pixels of LR images, so its loss is larger (we calculate the sum). If the reconstruction weight is much larger, there may be training instability at first. We simply keep their order to be equal (it is equivalent if we set lambda to be equal when we calculate the loss by mean over pixels). It is possible if there are better hyper-parameters.

codyshen0000 commented

I see. Thanks again!

johnnylu305 commented

@pkuxmq
Hi, can you explain why there may be training instability at first if the reconstruction weight is larger?