About loss function
liujiyuan13 opened this issue · 3 comments
The last term in the loss function is commonly referred to as weight decay and it is considered in another place. Its strength (lambda) is set through the parameters --weight_decay
(and for pretraining --ae_weight_decay
) which are passed to the optimizer.
The last term in the loss function is commonly referred to as weight decay and it is considered in another place. Its strength (lambda) is set through the parameters
--weight_decay
(and for pretraining--ae_weight_decay
) which are passed to the optimizer.
Thanks very much for your reply!
I've found the corresponding code for the weights of neural network.
As @rsaite pointed out, PyTorch makes it simple to add weight decay regularization on the network weights via the optimizer
. Just to add the specific lines for reference:
Deep SVDD trainer
Autoencoder trainer for pretraining
Deep-SVDD-PyTorch/src/optim/ae_trainer.py
Line 30 in 5d7195d