Protocols for finetuning on CIFAR10 or CIFAR100.

Question

Protocols for finetuning on CIFAR10 or CIFAR100.

alirezaabdollahpour opened this issue a year ago · 6 comments

alirezaabdollahpour commented a year ago

Hi @gabrielilharco ;)
Could you kindly provide the protocols you use for finetuning on CIFAR10 or CIFAR100?
For example, factors such as the learning rate and the number of epochs... Please refer to Figure 16 in the main paper.

Answer 1 · 2023-07-25T17:50:46.000Z

Hi @alirezaabdollahpour, the hyper-parameters are specified in Appendix D.2, please let us know if you have any questions

Answer 2 · 2023-07-26T08:34:50.000Z

Hi @gabrielilharco , thank you for responding. In the section you mentioned earlier, you discussed the learning rates and augmentation. What is the recommended number of epochs for fine-tuning CIFAR10 models?

Answer 3 · 2023-07-26T15:29:02.000Z

We do 10 epochs

Answer 4 · 2023-07-27T07:44:15.000Z

Thanks a lot, @gabrielilharco.

Answer 5 · 2023-08-13T20:27:27.000Z

I have a question about the warmup length in the paper.

The paper states that the warmup length is 500 steps, but for CIFAR100 with a batch size of 512, one epoch only takes 98 steps to train.
This means that the warmup phase occupies almost half of the total training time (total 10 epochs).
Is this correct?

It would be greatly helpful if you could clarify this point.
Thank you.

Answer 6 · 2023-08-13T22:41:06.000Z

I believe so, do you find this drops accuracy compared to e.g. 50 warmup steps?