Code for Introduction to Deep Learning (BioeE 394E).
This course is an introduction to the machine learning technique called deep learning or deep neural networks. A focus of the class will be the mathematical formulations of this deep networks and an explanation of how these networks can be structured and “learned” from big data. Multiple application areas with a focus on biology and health will be covered, namely, image classification, text analysis, and generative modeling. The class will be taught with a mathematical foundation in addition to a practical point of view on modern coding practices in deep learning. In particular, the example codes and the homework assignments will be given in the Python programming language with heavy utilization of the scikit-learn and PyTorch packages, a necessity for developing and applying the learned techniques. Students will have homework on every unit and a final project that needs to demonstrate mastery of the topics covered.
Planned topics:
• Computation graphs and large-scale logistic regression
• Multi-layer perceptions (first “basic” deep model)
• Stochastic optimization (with a focus on stochastic gradient descent)
• Convolutional neural networks (for image and text analysis)
• Regularization techniques in deep learning
• Auto-encoders (for data cleaning and unsupervised learning)
• Generative adversarial networks and Bayesian deep Learning
• Recurrent neural networks (for text and time-series analysis)
• Embeddings and attention mechanisms (with a focus on transformer-based architectures)
Slides for the lectures can be found at Blackboard.
- Unit 1:
- Python basics: bioe394e_unit1_lecture4_coding_basics.ipynb
- Pytorch basics (MLP and MNIST): bioe394e_unit1_lecture4_intro_pytorch.ipynb
- Unit 3:
- Implementation of Convolutional Neural Networks (CCNs): bioe394e_unit3_lecture10_ham10000.ipynb
- Unit 4:
- Implementing a Variational Autoencoder (VAE): bioe394e_unit4_lecture17_ae.ipynb
- Implementing a Generative Adversarial Network (GAN): bioe394e_unit4_lecture17_gan.ipynb
- Unit 5:
- Implementing Embeddings and Recurrent Neural Networks (RNNs): bioe394e_unit5_lecture21.ipynb
- Unit 6:
- Implementing Transformers with HuggingFace: bioe394e_unit6_lecture26.ipynb