Virtual Adversarial Training (VAT) implementation for Pytorch
- Distributional Smoothing with Virtual Adversarial Training - https://arxiv.org/abs/1507.00677
- Virtual Adversarial Training: a Regularization Method for Supervised and Semi-supervised Learning - https://arxiv.org/abs/1704.03976
for batch_idx, (data, target) in enumerate(train_loader):
data, target = data.to(device), target.to(device)
optimizer.zero_grad()
vat_loss = VATLoss(xi=10.0, eps=1.0, ip=1)
cross_entropy = nn.CrossEntropyLoss()
# LDS should be calculated before the forward for cross entropy
lds = vat_loss(model, data)
output = model(data)
loss = cross_entropy(output, target) + args.alpha * lds
loss.backward()
optimizer.step()