Number of classes changed discards pretrained classifier

[Geeks-Sid]

Hey @rwightman ,

I tried to change the number of classes for my finetuning project with 'tf_efficientnet_b4_ap' and for some reason, when I change the number of classes, i get the following output:
=> Discarding pretrained classifier since num_classes != 1000
but I think it shouldn't be discarded for num_classes != 1000 because it should be solved by something as simple as
model = geffnet.create_model(modelname, pretrained=True) model.classifier(nn.Sequential(nn.Linear(1792, num_classes)))
If this cannot be done, could you please explain why?
I would honestly like to run tf_efficientnet instead of regular ones as they do provide better results.

Also, the adv_prop normalization could adapt to other normalizations, right?

Thanks

[rwightman]

I don't understand why it matters? What you are doing discards the original classifier as well. So why does it matter if it's done twice? You can still do what you propose in either case if you prefer to change the classifier in a different manner. Or you can create the model with num_classes=1000 and then do what you propose.

Note, if it's not clear, when you see that message, there is still a classifier (by classifier we are talking about the final Linear layer), it has the num_classes you specified instead of 1000. Obviously it cannot load the pretrained weights for that last layer since it's a different size, it's randomly initialized. The rest of the network still has the pretrained weights. It's doing the modification to the network for you.

If you fine tune a model with a different input normalization it will adapt, but it takes more time. You'll get the best/fastest results if you match the original input normalization ... setting mean and std to 0.5, 0.5, 0.5 matches the adv prop models.