Changing init learning rate

[Kraut-Inferences]

Does modifying the initial learning rate hurt the algorithm in any way? Wanting to use exponential decay but don't know if it would improve the performance.

[juntang-zhuang]

From my experience with a ViT model on ImageNet, AdaBelief improves over Adam when both use a default cosine learning rate. I think it should work with other models.

[Kraut-Inferences]

thank you.