Performance of GSAM w/ Adaptive Perturbation

[SXKJames]

Hi,

Thanks for creating a PyTorch version of your code!

I saw that in you Cifar example you use the adaptive perturbation rather than the traditional perturbation. However, in figure 5 of you paper (which was a bar plot of different SAM methods), your results suggested that GSAM with the adaptive perturbation performed worse. The results also suggested that ASAM always performed worse than SAM in your results - which to me suggests that rho may not have been tuned properly ( although I might just misinterpretating these results).

Have you noticed better performances with using adaptive GSAM and have you found the adaptive approach to be harder to tune?

Thanks,
SXKJames

[juntang-zhuang]

Please ignore the results in this repo if you want a strong baseline. This repo is a demo and not extensively tested. The original code is the jax version in google research repo, and you can exactly reproduce it.

…

On Thu, Sep 14, 2023 at 9:21 PM SXKJames ***@***.***> wrote: Hi, Thanks for creating a PyTorch version of your code! I saw that in you Cifar example you use the adaptive perturbation rather than the traditional perturbation. However, in figure 5 of you paper (which was a bar plot of different SAM methods), your results suggested that GSAM with the adaptive perturbation performed worse. The results also suggested that ASAM always performed worse than SAM in your results - which to me suggests that rho may not have been tuned properly ( although I might just misinterpretating these results). Have you noticed better performances with using adaptive GSAM and have you found the adaptive approach to be harder to tune? Thanks, SXKJames — Reply to this email directly, view it on GitHub <#8>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKTELHLWB2QVUJE54LBJAY3X2NGM3ANCNFSM6AAAAAA4YSUIQ4> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>