[Feature Request] Differentiable Adan optimizer support
Benjamin-eecs opened this issue · 0 comments
Benjamin-eecs commented
Motivation
Adan optimizer is a big hit recently, good feature to support for meta learning research
Resource
- paper: https://arxiv.org/abs/2208.06677
- source code: https://github.com/sail-sg/Adan
Roadmap
- low-level alias
- high-level
Optimizer - high-level
MetaOptimizer - low-level test
- high-level
Optimizertest - api docs
Checklist
- I have checked that there is no similar issue in the repo (required)