This course offers students an introduction to some of the latest state-of-the-art techniques in the field of deep learning. Throughout the course, students will delve into a wide range of advanced topics and methodologies, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative models such as large language models, and transformers, among others. The primary focus will be on comprehending the underlying principles and mathematical foundations of these techniques, as well as their practical applications in domains such as computer vision and natural language processing. The course places a strong emphasis on practical implementations, allowing students to gain hands-on experience by implementing these techniques themselves. By the end of the course, students will have a basic understanding of deep learning, equipping them with the confidence to apply cutting-edge techniques to real-world problems.
CS 165B, the Introduction to Machine Learning course, is a prerequisite for this class. We expect students to be familiar with key concepts such as vector calculus, gradient descent, supervised vs. unsupervised learning, regression, classification, training/validation/testing, hyperparameter tuning, and dimensionality reduction. Additionally, students should have experience in deriving and implementing basic machine learning algorithms using Python.
- Instructor:
- Prof. Shiyu Chang, Tuesday after the lecture
- TAs:
- Bairu Hou (Head TA): Wednesday 5 - 6 pm, HH 2118
- Yujian Liu: Friday 5 - 6 pm, HH 2118
- Jiabao Ji: Thursday 5 - 6 pm, HH 2118
- Grader:
- Guanyu Yao: Re-grading by appointment only
For any questions regarding course policies, homework assignments, late submissions, or regrading, please contact the head TAs first and include all other TAs in your email.
The course piazza can be accessed here. Please note that the primary purpose of this forum is for students to exchange ideas and clarify any concepts. TAs are not expected to respond to questions around the clock. Additionally, sharing homework solutions directly in the discussion violates the student honor code.
- Lectures will occur Tuesday/Thursday from 11:00-12:15 pm Pacific Time at ILP 2211.
- Discussion sections will occur on Friday. There are three sections, 11-12pm, 12-1pm, and 1-2pm.
Updated lecture slides will be posted here shortly before each lecture.
| Date | Description | Course Materials | Important Events |
|---|---|---|---|
| Thu, 09/26 | Introduction to deep learning | [Slides] | |
| Tue, 10/01 | CS165B Review | [Slides] [Annotated] | [HW 1 self-assessment] is out, due on Sunday, Oct 13, 11:59 pm PST |
| Thu, 10/03 | CS165B Review Con't | ||
| Fri, 10/04 | Discussion Section: Python Review | [Slides] | |
| Tue, 10/08 | Multi-class Classification | [Slides] [Annotated] | |
| Thu, 10/10 | Multi-class Classification Con't | ||
| Fri, 10/11 | Discussion Section: Logistic Regression on Imbalanced Data | [Slides] | |
| Tue, 10/15 | Multi-class Classification Con't | [Detailed notes] | [HW2] is out, due on Sunday, Oct 27, 11:59 pm PST |
| Thu, 10/17 | Fully-connected Neural Networks | [Slides] [Annotated] | |
| Fri, 10/18 | Discussion Section: A Visual Proof for Universal Approximation Theorem | [Slides] | |
| Tue, 10/22 | Fully-connected Neural Networks Con't & Back Propagation | ||
| Thu, 10/24 | Back Propagation | [Slides] [Annotated] | |
| Fri, 10/25 | Discussion Section: PyTorch Introduction | [Slides] | |
| Tue, 10/29 | Back Propagation Con't & Text Classification | [Slides] | [HW3] is out, due on Sunday, Nov 10, 11:59 pm PST |
| Thu, 10/31 | Text Classification and Midterm 1 Review | [Annotated] [Review] | |
| Fri, 11/01 | Discussion Section: Word Representation | [Slides] | |
| Tue, 11/05 | In-Class Midterm Exam | ||
| Thu, 11/07 | Convolutional Neural Networks | [Slides] | |
| Fri, 11/08 | Midterm 1 Common Mistakes Revisit | [Reference reading] | |
| Tue, 11/12 | Convolutional Neural Networks Con't | [Annotated] [Video] | [HW4] is out, due on Sunday, Nov 24, 11:59 pm PST |
| Thu, 11/14 | Residual Networks | [Slides] [Annotated] [Video] | Reference reading: [ResNet], [Batch Normalization], [Layer Normalization] |
| Tue, 11/19 | Strategies to Mitigate Overfitting | [Slides] [Annotated] | Reference reading: [Dropout] |
| Thu, 11/21 | Recurrent Neural Network | [Slides] [Annotated] | Reference reading: [LSTM], [GRU] |
| Tue, 11/26 | Attention Mechanism | [Slides] | [HW5] is out, due on Sunday, Dec 8, 11:59 pm PST & Reference reading: [Seq2Seq], [PyTorch tutorial] |
| Thu, 11/28 | No class -- Happy Thanksgiving! | ||
| Tue, 12/02 | Attention Mechanism Con't & Transformers | [Slides] | Reference reading: [Attention is all you need] |
| Thu, 12/04 | Transformers Con't | Reference reading: [GPT-1], [GPT-2], [GPT-3] |