Welcome to our brand new project!
Awesome-AutoML-Papers is a curated list of automated machine learning papers, articles, tutorials, slides and projects. Star this project, then you can keep abreastof the latest developments of this booming research field. Thanks to all the people who made contributions to this project. Join us and you are welcome to be a contributor.
Automated Machine Learning provides methods and processes to make Machine Learning available for non-Machine Learning experts, to improve efficiency of Machine Learning and to accelerate research on Machine Learning.
Machine learning (ML) has achieved considerable successes in recent years and an ever-growing number of disciplines rely on it. However, this success crucially relies on human machine learning experts to perform the following tasks:
- Preprocess the data
- Select appropriate features
- Select an appropriate model family
- Optimize model hyperparameters
- Postprocess machine learning models
- Critically analyze the results obtained.
As the complexity of these tasks is often beyond non-ML-experts, the rapid growth of machine learning applications has created a demand for off-the-shelf machine learning methods that can be used easily and without expert knowledge. We call the resulting research area that targets progressive automation of machine learning AutoML. As a new sub-area in machine learning, AutoML has got more attention not only in machine learning but also in computer vision, natural language processing and graph computing.
There are no formal defination of AutoML. From the descriptions of most papers,the basic framework of AutoML can be shown as the following.
In recent years, AutoML has attracted much attention from a bunch of giant companies and start-up companies. An overview comparison of some of them can be summarized to the following table.
Company | Automated Feature Engineering | Hyperparameter Optimization | Architecture Search |
---|---|---|---|
4paradigm | √ | √ | × |
Alibaba | × | √ | × |
Baidu | × | × | √ |
× | √ | √ | |
H2O | √ | √ | × |
Microsoft | × | √ | √ |
RapidMiner | √ | √ | × |
Tencent | × | √ | × |
Transwarp | √ | √ | √ |
- Papers
- Tutorials
- Articles
- Slides
- Books
- Projects
- Prominent Researchers
-
- 2017 | AutoLearn — Automated Feature Generation and Selection | Ambika Kaul, et al. | ICDM |
PDF
- 2017 | One button machine for automating feature engineering in relational databases | Hoang Thanh Lam, et al. | arXiv |
PDF
- 2016 | Automating Feature Engineering | Udayan Khurana, et al. | NIPS |
PDF
- 2016 | ExploreKit: Automatic Feature Generation and Selection | Gilad Katz, et al. | ICDM |
PDF
- 2015 | Deep Feature Synthesis: Towards Automating Data Science Endeavors | James Max Kanter, Kalyan Veeramachaneni | DSAA |
PDF
- 2017 | AutoLearn — Automated Feature Generation and Selection | Ambika Kaul, et al. | ICDM |
-
- 2016 | Cognito: Automated Feature Engineering for Supervised Learning | Udayan Khurana, et al. | ICDMW |
PDF
- 2016 | Cognito: Automated Feature Engineering for Supervised Learning | Udayan Khurana, et al. | ICDMW |
-
- 2017 | Learning Feature Engineering for Classification | Fatemeh Nargesian, et al. | IJCAI |
PDF
- 2017 | Learning Feature Engineering for Classification | Fatemeh Nargesian, et al. | IJCAI |
-
- 2019 | Evolutionary Neural AutoML for Deep Learning | Jason Liang, et al. | arXiv |
PDF
- 2017 | Large-Scale Evolution of Image Classifiers | Esteban Real, et al. | PMLR |
PDF
- 2002 | Evolving Neural Networks through Augmenting Topologies | Kenneth O.Stanley, Risto Miikkulainen | Evolutionary Computation |
PDF
- 2019 | Evolutionary Neural AutoML for Deep Learning | Jason Liang, et al. | arXiv |
-
- 2017 | Simple and Efficient Architecture Search for Convolutional Neural Networks | Thomoas Elsken, et al. | ICLR |
PDF
- 2017 | Simple and Efficient Architecture Search for Convolutional Neural Networks | Thomoas Elsken, et al. | ICLR |
-
- 2016 | Learning to Optimize | Ke Li, Jitendra Malik | arXiv |
PDF
- 2016 | Learning to Optimize | Ke Li, Jitendra Malik | arXiv |
-
- 2018 | AMC: AutoML for Model Compression and Acceleration on Mobile Devices | Yihui He, et al. | ECCV |
PDF
- 2018 | Efficient Neural Architecture Search via Parameter Sharing | Hieu Pham, et al. | arXiv |
PDF
- 2017 | Neural Architecture Search with Reinforcement Learning | Barret Zoph, Quoc V. Le | ICLR |
PDF
- 2018 | AMC: AutoML for Model Compression and Acceleration on Mobile Devices | Yihui He, et al. | ECCV |
-
- 2017 | Learning Transferable Architectures for Scalable Image Recognition | Barret Zoph, et al. | arXiv |
PDF
- 2017 | Learning Transferable Architectures for Scalable Image Recognition | Barret Zoph, et al. | arXiv |
-
- 2018 | Efficient Neural Architecture Search with Network Morphism | Haifeng Jin, et al. | arXiv |
PDF
- 2018 | Efficient Neural Architecture Search with Network Morphism | Haifeng Jin, et al. | arXiv |
-
- 2018 | Neural Architecture Optimization | Renqian Luo, et al. | arXiv |
PDF
- 2018 | Neural Architecture Optimization | Renqian Luo, et al. | arXiv |
- 2019 | Evolutionary Neural AutoML for Deep Learning | Jason Liang, et al. | arXiv |
PDF
- 2017 | ATM: A Distributed, Collaborative, Scalable System for Automated Machine Learning | T. Swearingen, et al. | IEEE |
PDF
- 2017 | Google Vizier: A Service for Black-Box Optimization | Daniel Golovin, et al. | KDD |
PDF
- 2015 | AutoCompete: A Framework for Machine Learning Competitions | Abhishek Thakur, et al. | ICML |
PDF
-
- 2018 | A Tutorial on Bayesian Optimization. |
PDF
- 2018 | Efficient High Dimensional Bayesian Optimization with Additivity and Quadrature Fourier Features | Mojmír Mutný, et al. | NeurIPS |
PDF
- 2018 | High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups. | PMLR |
PDF
- 2016 | Bayesian Optimization with Robust Bayesian Neural Networks | Jost Tobias Springenberg, et al. | NIPS |
PDF
- 2016 | Scalable Hyperparameter Optimization with Products of Gaussian Process Experts | Nicolas Schilling, et al. | PKDD |
PDF
- 2016 | Taking the Human Out of the Loop: A Review of Bayesian Optimization | Bobak Shahriari, et al. | IEEE |
PDF
- 2016 | Towards Automatically-Tuned Neural Networks | Hector Mendoza, et al. | JMLR |
PDF
- 2016 | Two-Stage Transfer Surrogate Model for Automatic Hyperparameter Optimization | Martin Wistuba, et al. | PKDD |
PDF
- 2015 | Efficient and Robust Automated Machine Learning |
PDF
- 2015 | Hyperparameter Optimization with Factorized Multilayer Perceptrons | Nicolas Schilling, et al. | PKDD |
PDF
- 2015 | Hyperparameter Search Space Pruning - A New Component for Sequential Model-Based Hyperparameter Optimization | Martin Wistua, et al. |
PDF
- 2015 | Joint Model Choice and Hyperparameter Optimization with Factorized Multilayer Perceptrons | Nicolas Schilling, et al. | ICTAI |
PDF
- 2015 | Learning Hyperparameter Optimization Initializations | Martin Wistuba, et al. | DSAA |
PDF
- 2015 | Scalable Bayesian optimization using deep neural networks | Jasper Snoek, et al. | ACM |
PDF
- 2015 | Sequential Model-free Hyperparameter Tuning | Martin Wistuba, et al. | ICDM |
PDF
- 2013 | Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms |
PDF
- 2013 | Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures | J. Bergstra | JMLR |
PDF
- 2012 | Practical Bayesian Optimization of Machine Learning Algorithms |
PDF
- 2011 | Sequential Model-Based Optimization for General Algorithm Configuration(extended version) |
PDF
- 2018 | A Tutorial on Bayesian Optimization. |
-
- 2018 | Autostacker: A Compositional Evolutionary Learning System | Boyuan Chen, et al. | arXiv |
PDF
- 2017 | Large-Scale Evolution of Image Classifiers | Esteban Real, et al. | PMLR |
PDF
- 2016 | Automating biomedical data science through tree-based pipeline optimization | Randal S. Olson, et al. | ECAL |
PDF
- 2016 | Evaluation of a tree-based pipeline optimization tool for automating data science | Randal S. Olson, et al. | GECCO |
PDF
- 2018 | Autostacker: A Compositional Evolutionary Learning System | Boyuan Chen, et al. | arXiv |
-
- 2017 | Global Optimization of Lipschitz functions | C´edric Malherbe, Nicolas Vayatis | arXiv |
PDF
- 2017 | Global Optimization of Lipschitz functions | C´edric Malherbe, Nicolas Vayatis | arXiv |
-
- 2009 | ParamILS: An Automatic Algorithm Configuration Framework | Frank Hutter, et al. | JAIR |
PDF
- 2009 | ParamILS: An Automatic Algorithm Configuration Framework | Frank Hutter, et al. | JAIR |
-
- 2008 | Cross-Disciplinary Perspectives on Meta-Learning for Algorithm Selection |
PDF
- 2008 | Cross-Disciplinary Perspectives on Meta-Learning for Algorithm Selection |
-
- 2017 | Particle Swarm Optimization for Hyper-parameter Selection in Deep Neural Networks | Pablo Ribalta Lorenzo, et al. | GECCO |
PDF
- 2008 | Particle Swarm Optimization for Parameter Determination and Feature Selection of Support Vector Machines | Shih-Wei Lin, et al. | Expert Systems with Applications |
PDF
- 2017 | Particle Swarm Optimization for Hyper-parameter Selection in Deep Neural Networks | Pablo Ribalta Lorenzo, et al. | GECCO |
-
- 2016 | Efficient Transfer Learning Method for Automatic Hyperparameter Tuning | Dani Yogatama, Gideon Mann | JMLR |
PDF
- 2016 | Flexible Transfer Learning Framework for Bayesian Optimisation | Tinu Theckel Joy, et al. | PAKDD |
PDF
- 2016 | Hyperparameter Optimization Machines | Martin Wistuba, et al. | DSAA |
PDF
- 2013 | Collaborative Hyperparameter Tuning | R´emi Bardenet, et al. | ICML |
PDF
- 2016 | Efficient Transfer Learning Method for Automatic Hyperparameter Tuning | Dani Yogatama, Gideon Mann | JMLR |
- 2018 | Accelerating Neural Architecture Search using Performance Prediction | Bowen Baker, et al. | ICLR |
PDF
- 2017 | Automatic Frankensteining: Creating Complex Ensembles Autonomously | Martin Wistuba, et al. | SIAM |
PDF
- 2010 | A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning |
PDF
- 2008 | Metalearning - A Tutorial |
PDF
- 2016 | Bayesian Optimization for Hyperparameter Tuning |
Link
- 2017 | Learning to learn |
Link
- 2017 | Why Meta-learning is Crucial for Further Advances of Artificial Intelligence? |
Link
- Automated Feature Engineering for Predictive Modeling | Udyan Khurana, etc al. |
PDF
- 2009 | Metalearning - Applications to Data Mining | Springer |
PDF
- AdaNet |
Python
|Open Source
|Code
- Advisor |
Python
|Open Source
|Code
- Amla |
Python
|Open Source
|Code
- ATM |
Python
|Open Source
|Code
- Auger |
Python
|Commercial
|Link
- Auto-Keras |
Python
|Open Source
|Code
- Auto-sklearn |
Python
|Open Source
|Code
- Auto-WEKA |
Java
|Open Source
|Code
- Auto_ml |
Python
|Open Source
|Code
- BayesianOptimization |
Python
|Open Source
|Code
- BayesOpt |
C++
|Open Source
|Code
- Cloud AutoML |
Python
|Commercial
|Link
- Comet |
Python
|Commercial
|Link
- DataRobot |
Python
|Commercial
|Link
- DEvol |
Python
|Open Source
|Code
- FAR-HO |
Python
|Open Source
|Code
- H2O |
Python
|Commercial
|Link
- HpBandSter |
Python
|Open Source
|Code
- HyperBand |
Python
|Open Source
|Code
- Hyperopt |
Python
|Open Source
|Code
- Hyperopt-sklearn |
Python
|Open Source
|Code
- Hyperparameter_hunter |
Python
|Open Source
|Code
- Katib |
Python
|Open Source
|Code
- MateLabs |
Python
|Commercial
|Link
- Milano |
Python
|Open Source
|Code
- MLJAR |
Python
|Commercial
|Link
- Nasbot |
Python
|Open Source
|Code
- Neptune |
Python
|Commercial
|Link
- NNI |
Python & Typescript
|Open Source
|Code
- Optunity |
Python
|Open Source
|Code
- Rbfopt |
Python
|Open Source
|Code
- RoBO |
Python
|Open Source
|Code
- Scikit-Optimize |
Python
|Open Source
|Code
- SigOpt |
Python
|Commercial
|Link
- SMAC3 |
Python
|Open Source
|Code
- TPOT |
Python
|Open Source
|Code
- TransmogrifAI |
Scala
|Open Source
|Code
- Tune |
Python
|Open Source
|Code
|Docs
- Xcessiv |
Python
|Open Source
|Code
- Frank Hutter | University of Freiburg
- Martin Wistuba | IBM Research
Special thanks to everyone who contributed to this project.
- derekflint
- endymecy
- Eric
- Randy Olson
- Richard Liaw
- Saket Maheshwary
- shaido987
- Slava Kurilyak
- sophia-wright-blue
- tengben0905
- xuehui
- Yihui He
Awesome-AutoML-Papers is available under Apache Licenses 2.0.
If you have any suggestions (missing papers, new papers, key researchers or typos), feel free to pull a request. Also you can mail to:
- hibayesian (hibayesian@gmail.com).