lukasruff/Deep-SVDD-PyTorch

About loss function

liujiyuan13 opened this issue · 3 comments

In paper, "Deep One-Class Classification", the loss function goes as
image
However, the loss is calculated as
image
They are different in these two places. The loss in this code lacks of the weights of neural network. Is it important or considered in another place?

The last term in the loss function is commonly referred to as weight decay and it is considered in another place. Its strength (lambda) is set through the parameters --weight_decay (and for pretraining --ae_weight_decay) which are passed to the optimizer.

The last term in the loss function is commonly referred to as weight decay and it is considered in another place. Its strength (lambda) is set through the parameters --weight_decay (and for pretraining --ae_weight_decay) which are passed to the optimizer.

Thanks very much for your reply!
I've found the corresponding code for the weights of neural network.

As @rsaite pointed out, PyTorch makes it simple to add weight decay regularization on the network weights via the optimizer. Just to add the specific lines for reference:

Deep SVDD trainer

optimizer = optim.Adam(net.parameters(), lr=self.lr, weight_decay=self.weight_decay)

Autoencoder trainer for pretraining

optimizer = optim.Adam(ae_net.parameters(), lr=self.lr, weight_decay=self.weight_decay)