The notebooks seen here are based on the assignements recquired for the Optimization for Data Science course given at Telecom Paristech as part of the Master Data Science (X, ENSAE, Telecom) by Stéphane Gaiffas (Ecole polytechnique) and Alexandre Gramfort (Telecom ParisTech)
This course aims to understand, as from a theoretical point of view and from a practical point of view, the various optimization algorithms. In practice, the experience gained with this course is really relevant to understand the impact of hyperparameter and to develop a consistent search strategies for any Machine Learning models based on optimization.
As most (all?) of the theoretical results in optimization are included in a convexe framework, all of the algorithms seen here are based on convex hypothesis. Still, the results and the observations seen in this context stand as strong guidelines for future application to non-convex model.