dtsbourg/HEP-ML-Resources

Listing of useful learning resources for machine learning applications in high energy physics (HEPML)

HEPML Resources

Listing of useful (mostly) public learning resources for machine learning applications in high energy physics (HEPML). Listings will be in reverse chronological order (like a CV).

N.B.: This listing will almost certainly be biased towards work done by ATLAS scientists, as the maintainer is a member of ATLAS and so sees ATLAS work the most. However, this is not the desired case and help to diversify this listing would be greatly appreciated.

Table of contents

• Introductory Material
  • Lectures
  • Seminar Series
  • Tutorials
  • Schools
  • Courses
  • Journals
• Software
• Papers
• Workshops
• Tweets
• People
• Contributing

Introductory Material

Lectures

• Introduction to GANs, by Luke de Oliveira (November 3, 2017)

• Frontiers with GANs, by Michela Paganini (November 3, 2017)

• Nikhef Colloquium: "Teaching machines to discover particles", by Gilles Louppe (September 29, 2017)

• CERN Academic Training Lecture Regular Programme, April 2017 (Machine Learning):

  • Machine Learning (Lecture 1) --- Michael Kagan (SLAC)
  • Machine Learning (Lecture 2) --- Michael Kagan (SLAC)
  • Deep Learning and Vision --- Jonathon Shlens (Google Research)

• Deep Learning in High Energy Physics, by Amir Farbin

Seminar Series

• CERN Data Science Seminars

• Inter-Experimental LHC Machine Learning Working Group Guest Seminars:

  • Open challenges for improving Generative Adversarial Networks (GANs), by Ian Goodfellow (October 27, 2017)

Tutorials

• PyTorch Deep Learning Minicourse - CoDaS-HEP 2018, by Alfredo Canziani

• Intro Tutorial on GANs, by Michela Paganini

• Scikit-learn Tutorial, by Gilles Louppe

• TMVA Tutorial, by Lorenzo Moneta

• Keras and TMVA interfaces Tutorial, by Stefan Wunsch

• Boosted Decision Tree Tutorial (using XGBoost), by Katherine Woodruff

• Introduction to Deep Learning with Keras Tutorial, by Luke de Oliveira

• Introduction to Deep Learning with Keras Tutorial - 2nd Developers@CERN Forum, by Michela Paganini

Schools

HEP-ML:

Upcoming:

• 4th Machine Learning in High Energy Physics Summer School 2018 (August 6-12, 2018)

Past:

• 3rd Machine Learning in High Energy Physics Summer School 2017 (July 17-23, 2017)
  • Associated Yandex School of Data Analysis repo: mlhep2017
• 1st Computational and Data Science School for High Energy Physics (July 10-13, 2017)
• 2nd Machine Learning in High Energy Physics Summer School 2016 (June 20-26, 2016)
  • Associated Yandex School of Data Analysis repo: mlhep2016
• 1st Machine Learning in High Energy Physics Summer School 2015 (August 27-30, 2015)
  • Associated Yandex School of Data Analysis repo: mlhep2015

Deep Learning:

Upcoming:

• Machine Learning Summer School 2018, Buenos Aires, Argentina (June 18-20, 2018)
• Machine Learning Summer School 2018, Madrid, Spain (August 27 - September 7, 2018)
• Deep Learning and Reinforcement Learning Summer School, Toronto, Canada (July 25 - August 3, 2018)
• PAISS: Artificial Intelligence Summer School, Grenoble, France (July 2-6, 2018)

Past:

• Deep Learning Summer School 2016 (August 1-7, 2016)

Courses

• Advanced Machine Learning, Pierre Geurts, Gilles Louppe, and Louis Wehenkel (Spring, 2018 - Université de Liège, Institut Montefiore)

• Applications of Deep Learning to High Energy Physics, Amir Farbin (Spring, 2017 - University of Texas at Arlington)

  • Associated GitHub repository: DSatHEP-Tutorial

• Tensorflow for Deep Learning Research, (Spring, 2017 - Stanford Univeristy)

• Introduction to Machine Learning and Convolutional Neural Networks for Visual Recognition:

  • Spring, 2017 - Stanford University, Fei-Fei Li, Justin Johnson, Serena Yeung
  • Winter, 2016 - Stanford University, Andrej Karpathy, Fei-Fei Li, Justin Johnson

Journals

• Distill Research Journal

Software

Common software tools and environments used in HEP for ML

• The Conda package and environment manager and Anaconda Python library collection

  • Using ROOT/PyROOT with Conda and NumPy

• scikit-learn: General machine learning package

• TMVA: ROOT's builtin machine learning package

  • TMVA-branch-adder: wrapper to add TMVA response to TTree without boiler plate code

High level deep learning libraries/framework APIs

• Keras

Deep learning frameworks

• TensorFlow
• Theano
• PyTorch
• Caffe2
• List of Conversion Tools For Saved Networks

HEP to ML bridge tools

• lwtnn: Tool to run Keras networks in C++ code

• Scikit-HEP: Toolset of interfaces and Python tools for Particle Physics

  • root_numpy: The interface between ROOT and numpy

  • root_pandas: An upgrade of root_numpy to use with pandas

  • uproot: Mimimalist ROOT to numpy converter (no dependency on ROOT)

• ttree2hdf5: Mimimalist ROOT to HDF5 converter (written in C++)

• hep_ml: Python algorithms and tools for HEP ML use cases

Papers

A .bib file for all papers listed is available in the tex directory.

• M. Andrews, M. Paulini, S. Gleyzer, and B. Poczos, "End-to-End Physics Event Classification with the CMS Open Data: Applying Image-based Deep Learning on Detector Data to Directly Classify Collision Events at the LHC," arXiv:1807.11916 [hep-ex]. (July 31, 2018)

• J. Lin, M. Freytsis, I. Moult, and B. Nachman, "Boosting H → bb̅ with Machine Learning," arXiv:1807.10768 [hep-ph]. (July 27, 2018)

• K. Albertsson et al., "Machine Learning in High Energy Physics Community White Paper," arXiv:1807.02876 [physics.comp-ph]. (July 8, 2018)

• D. Guest, K. Cranmer, and D. Whiteson, "Deep Learning and its Application to LHC Physics," arXiv:1806.11484 [hep-ex]. (June 29, 2018)

• J. Brehmer, K. Cranmer, G. Louppe, and J. Pavez, "Constraining Effective Field Theories with Machine Learning," arXiv:1805.00013 [hep-ph]. (April 30, 2018)

• J. Brehmer, K. Cranmer, G. Louppe, and J. Pavez, "A Guide to Constraining Effective Field Theories with Machine Learning," arXiv:1805.00020 [hep-ph]. (April 30, 2018)

• CMS Collaboration, A. M. Sirunyan et al., "Identification of heavy-flavour jets with the CMS detector in pp collisions at 13 TeV," arXiv:1712.07158 [physics.ins-det]. (December 19, 2017)

• V. Estrade, C. Germain, I. Guyon and D. Rousseau, Adversarial learning to eliminate systematic errors: a case study in High Energy Physics, Proceedings of the Deep Learning for Physical Sciences Workshop at NIPS (2017). (December 8, 2017)

• D. Weitekamp III, T. Nguyen, D. Anderson, R. Castello, M.Pierini, M. Spiropulu and J. Vlimant, Deep topology classifiers for a more efficient trigger selection at the LHC, Proceedings of the Deep Learning for Physical Sciences Workshop at NIPS (2017). (December 8, 2017)

• L. Hertel, L. Li, P. Baldi and J. Bian, Convolutional Neural Networks for Electron Neutrino and Electron Shower Energy Reconstruction in the NOvA Detectors, Proceedings of the Deep Learning for Physical Sciences Workshop at NIPS (2017). (December 8, 2017)

• M. Stoye, J. Kieseler, M. Verzetti, H. Qu, L. Gouskos, A. Stakia and the CMS Collaboration, DeepJet: Generic physics object based jet multiclass classification for LHC experiments, Proceedings of the Deep Learning for Physical Sciences Workshop at NIPS (2017). (December 8, 2017)

• B. Hooberman, A. Farbin, G. Khattak, V. Pacela, M. Pierini, J. Vlimant, M. Spiropulu, W. Wei, M. Zhang and S. Vallecorsa, Calorimetry with Deep Learning: Particle Classification, Energy Regression, and Simulation for High-Energy Physics, Proceedings of the Deep Learning for Physical Sciences Workshop at NIPS (2017). (December 8, 2017)

• M. Paganini, L. de Oliveira and B. Nachman, Survey of Machine Learning Techniques for High Energy Electromagnetic Shower Classification, Proceedings of the Deep Learning for Physical Sciences Workshop at NIPS (2017). (December 8, 2017)

• L. de Oliveira, M. Paganini, and B. Nachman, Tips and Tricks for Training GANs with Physics Constraints, Proceedings of the Deep Learning for Physical Sciences Workshop at NIPS (2017). (December 8, 2017)

• S. Farrell, P. Calafiura, M. Mudigonda, Prabhat, D. Anderson, J. Bendavid, M. Spiropoulou, J. Vlimant, S. Zheng, G. Cerati, L. Gray, J. Kowalkowski, P. Spentzouris, A. Tsaris and D. Zurawski, Particle Track Reconstruction with Deep Learning, Proceedings of the Deep Learning for Physical Sciences Workshop at NIPS (2017). (December 8, 2017)

• I. Henrion, K. Cranmer, J. Bruna, K. Cho, J. Brehmer, G. Louppe and G. Rochette, Neural Message Passing for Jet Physics, Proceedings of the Deep Learning for Physical Sciences Workshop at NIPS (2017). (December 8, 2017)

• T. Cheng, "Recursive Neural Networks in Quark/Gluon Tagging," arXiv:1711.02633 [hep-ph]. (November 7, 2017)

• S. Chang, T. Cohen, and B. Ostdiek, "What is the Machine Learning?," arXiv:1709.10106 [hep-ph]. (September 28, 2017)

• M. Frate, K. Cranmer, S. Kalia, A. Vandenberg-Rodes, and D. Whiteson, “Modeling Smooth Backgrounds and Generic Localized Signals with Gaussian Processes,” arXiv:1709.05681 [physics.data-an]. (September 17, 2017)

• E. M. Metodiev, B. Nachman, and J. Thaler, “Classification without labels: Learning from mixed samples in high energy physics,” arXiv:1708.02949 [hep-ph]. (August 9, 2017)

• J. Bendavid, "Efficient Monte Carlo Integration Using Boosted Decision Trees and Generative Deep Neural Networks," arXiv:1707.00028 [hep-ph]. (June 30, 2017)

• T. Cohen, M. Freytsis, and B. Ostdiek, "(Machine) Learning to Do More with Less," arXiv:1706.09451 [hep-ph]. (June 28, 2017)

• M. Paganini, L. de Oliveira, and B. Nachman, "CaloGAN: Simulating 3D High Energy Particle Showers in Multi-Layer Electromagnetic Calorimeters with Generative Adversarial Networks," arXiv:1705.02355 [hep-ex]. (May 5, 2017)

• C. Shimmin, P. Sadowski, P. Baldi, E. Weik, D. Whiteson, E. Goul, and A. Sgaard, "Decorrelated Jet Substructure Tagging using Adversarial Neural Networks," arXiv:1703.03507 [hep-ex]. (March 9, 2017)

• G. Louppe, K. Cho, C. Becot, and K. Cranmer, "QCD-Aware Recursive Neural Networks for Jet Physics," arXiv:1702.00748 [hep-ph]. (February 2, 2017)

  • Lecture: QCD-Aware Neural Networks for Jet Physics, by Kyle Cranmer

• L. M. Dery, B. Nachman, F. Rubbo, and A. Schwartzman, "Weakly Supervised Classification in High Energy Physics," JHEP 05 (2017) 145, arXiv:1702.00414 [hep-ph]. (February 1, 2017)

• L. de Oliveira, M. Paganini, and B. Nachman, "Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis," arXiv:1701.05927 [stat.ML]. (January 20, 2017)

• L.-G. Pang, K. Zhou, N. Su, H. Petersen, H. Stocker, X.-N. Wang, "An equation-of-state-meter of QCD transition from deep learning," arXiv:1612.04262 [hep-ph]. (December 13, 2016)

• P. T. Komiske, E. M. Metodiev, and M. D. Schwartz, "Deep learning in color: towards automated quark/gluon jet discrimination," JHEP 01 (2017) 110, arXiv:1612.01551 [hep-ph]. (December 5, 2016)

• MicroBooNE Collaboration, R. Acciarri et al., "Convolutional Neural Networks Applied to Neutrino Events in a Liquid Argon Time Projection Chamber," JINST 12 (2017) no. 03, P03011, arXiv:1611.05531 [physics.ins-det]. (November 16, 2016)

• M. Kagan, L. d. Oliveira, L. Mackey, B. Nachman, and A. Schwartzman, "Boosted Jet Tagging with Jet-Images and Deep Neural Networks," EPJ Web Conf. 127 (2016) 00009. (November 15, 2016)

• G. Bertone, M. P. Deisenroth, J. S. Kim, S. Liem, R. Ruiz de Austri, and M. Welling, "Accelerating the BSM interpretation of LHC data with machine learning," arXiv:1611.02704 [hep-ph]. (November 8, 2016)

• G. Louppe, M. Kagan, and K. Cranmer, "Learning to Pivot with Adversarial Networks," arXiv:1611.01046 [stat.ME]. (November 3, 2016)

  • Lecture: (Machine) Learning to Pivot with Adversarial Networks, by Gilles Louppe

• J. Barnard, E. N. Dawe, M. J. Dolan, and N. Rajcic, "Parton Shower Uncertainties in Jet Substructure Analyses with Deep Neural Networks," Phys. Rev. D95 (2017) no. 1, 014018, arXiv:1609.00607 [hep-ph] (September 2, 2016)

• A. Rogozhnikov, “Reweighting with Boosted Decision Trees,” J. Phys. Conf. Ser. 762 (2016) no. 1, 012036, arXiv:1608.05806 [physics.data-an]. (August 20, 2016)

• S. Caron, J.S. Kim, K. Rolbiecki, R. Ruiz de Austri, B. Stienen "The BSM-AI project: SUSY-AI -- Generalizing LHC limits on supersymmetry with machine learning", EPJ C (2017) 77:257, arXiv:1605.02797 [hep-ph]. (May 9, 2016)

• A. Aurisano, A. Radovic, D. Rocco, A. Himmel, M. D. Messier, E. Niner, G. Pawloski, F. Psihas, A. Sousa, and P. Vahle, "A Convolutional Neural Network Neutrino Event Classifier," JINST 11 (2016) no. 09, P09001, arXiv:1604.01444 [hep-ex]. (April 5, 2016)

• P. Baldi, K. Cranmer, T. Faucett, P. Sadowski, and D. Whiteson, “Parameterized neural networks for high-energy physics,” Eur. Phys. J. C76 (2016) no. 5, 235, arXiv:1601.07913 [hep-ex]. (January 28, 2016)

• L. de Oliveira, M. Kagan, L. Mackey, B. Nachman, and A. Schwartzman, "Jet-images deep learning edition," JHEP 07 (2016) 069, arXiv:1511.05190 [hep-ph]. (November 16, 2015)

• A. Rogozhnikov, A. Bukva, V. Gligorov, A. Ustyuzhanin, M. Williams, "New approaches for boosting to uniformity," arXiv:1410.4140 [hep-ex]. (October 15, 2014)

• P. Baldi, P. Sadowski, and D. Whiteson, "Searching for Exotic Particles in High-Energy Physics with Deep Learning," Nature Commun. 5 (2014) 4308, arXiv:1402.4735 [hep-ph]. (February 19, 2014)

• J. Stevens, M. Williams, "uBoost: A boosting method for producing uniform selection efficiencies from multivariate classifiers," arXiv:1305.7248 [nucl-ex]. (May 30, 2013)

• V. Gligorov, M. Williams, "Efficient, reliable and fast high-level triggering using a bonsai boosted decision tree," arXiv:1210.6861 [physics]. (October 25, 2012)

• B. H. Denby, “Neural Networks and Cellular Automata in Experimental High-energy Physics,” Comput. Phys. Commun. 49 (1988) 429–448. (September 20, 1987)

Workshops

Upcoming

• TBA

Past

• 2nd IML Machine Learning Workshop (2018) (April 9-12, 2018)
• Machine Learning for Phenomenology (2018) (April 3-6, 2018)
• 4th International Connecting The Dots Workshop (2018) (March 20-22, 2018)
• Machine Learning for Jet Physics (December 11-13, 2017)
• Deep Learning for Physical Sciences at NIPS (December 8, 2017)
• 18th International Workshop on Advanced Computing and Analysis Techniques in Physics Research (ACAT 2017) (August 21-25, 2017)
• Hammers & Nails - Machine Learning & HEP (July 19-28, 2017)
• CMS Machine Learning Workshop (2017) (July 5-6, 2017)
• ATLAS Machine Learning Workshop (2017) (June 6-9, 2017)
• Workshop on Machine Learning and b-tagging (May 23-26, 2017)
• DS@HEP 2017 (May 8-12, 2017)
• 2nd S2I2 HEP/CS Workshop (Parallel Session) (May 1-3, 2017)
• CERN openlab workshop on Machine Learning and Data Analytics (April 27, 2017)
• First IML Workshop on Machine Learning (March 20-22, 2017)
• DS@HEP at the Simons Foundation (July 5-7, 2016)
• ALICE Mini-Workshop 2016: Statistical Methods and Machine Learning Tutorial (May 18, 2016)
• ATLAS Machine Learning Workshop (2016) (March 29-31, 2016)
• Heavy Flavour Data Mining workshop (February 18-20, 2016)
• Data Science @ LHC 2015 (November 9-13, 2015)

Tweets

• #HEPML collection of tweets

People

• HEPML directory: Opt-in list of people working at the intersection of Machine Learning and High Energy Physics
  • Add yourself through the Google form

Contributing

Contributions to help improve the listing are very much welcome! Please read CONTRIBUTING.md for details on the process for submitting pull requests or filing issues.

Authors

Listing maintainer: Matthew Feickert

Acknowledgments

• Following PurpleBooth's README style
• All badges made by shields.io
• Inspiration for this listing came from the Awesome Machine Learning repo and Dustin Tran's Machine Learning Videos repo
• Many thanks to everyone who has contributed their time to improve this project