rwightman/gen-efficientnet-pytorch

meaning of `ns` vs `ap`

Borda opened this issue · 1 comments

Borda commented

Hello, may I have an n00b question, what is the meaning of these two abbreviations ns and ap? 🐰

Borda commented

AdvProp is an adversarial training scheme that treats adversarial examples as additional examples, to prevent overfitting. Key to the method is the usage of a separate auxiliary batch norm for adversarial examples, as they have different underlying distributions to normal examples. (from tf-efficientnet-b3-ap)

Noisy Student Training is a semi-supervised learning approach. It extends the idea of self-training and distillation with the use of equal-or-larger student models and noise added to the student during learning. The algorithm is iterated a few times by treating the student as a teacher to relabel the unlabeled data and training a new student. (from tf-efficientnet-b7-ns)