/deep-learning

Deep learning.

Primary LanguageJupyter NotebookMIT LicenseMIT

Deep Learning course

In this github repo, I share materials corresponding to two courses that I etach at the University of Calgary:

  • ENEL645 - Data Mining & Machine Learning
  • ENSF 619.02 - Advanced Image Analysis and Machine Learning

This repo is constantly being updated. Initially, we used TensorFlow as the deep learning, but we will be adding PyTorch examples since it is the dominant deep learning framework at the moment.

ENEL 645

Assignments and Final Project

For all teamwork assigments, including the final project, your team needs an additional 1-page pdf document as described here.

-Assignment 01

-Assignment 02

-Final Project

Templates for final project:

Lectures and Tutorials

Week 0

Week 01 (09-15 January)

Week 02 (16-22 January)

Week 03 (23-29 January)

Week 04 (30 January - 05 February)

Week 05 (06-12 February)

  • Lectures TBD

Week 06 (13-19 February)

  • Lectures TBD
  • Quiz: 17 February

Week 07 (20-26 February)

  • Reading week

Week 08 (27 February - 05 March)

  • Lectures TBD
  • Quiz: 27 February

Week 09 (06-12 March)

  • Lectures TBD
  • Assignment 02 due 06 March at midnight

Week 10 (13-19 March)

  • Lectures TBD

Week 11 (20-26 March)

  • Lectures TBD
  • Wednesday and Friday - support for finalizing the projects

Weeks 12 and 13 (27 March - 12 April )

  • Project report and 10 minute presentation recording due date: 27 March at 9 am
  • Final projects' presentations

ENSF 619.02

Assignments and Final Project

-Assignment 01

-Assignment 02

-Final Project

Templates for final project:

Lectures and Tutorials

Week 0

Week 01 (09-15 January)

Week 02 (16-22 January)

Week 03 (23-29 January) Monday - paper discussion Paper discussion

[1] K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition.” arXiv, Apr. 10, 2015. Accessed: Dec. 08, 2022. [Online]. Available: http://arxiv.org/abs/1409.1556

[2] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition.” arXiv, Dec. 10, 2015. Accessed: Dec. 08, 2022. [Online]. Available: http://arxiv.org/abs/1512.03385

Presenters: Nisha Mansuri and Yashkumar Trada

  • Friday (27 January) - short quiz and paper discussion*

[3] M. Tan and Q. V. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” p. 10.

Presenters: Anik Das

[4] Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A ConvNet for the 2020s.” arXiv, Mar. 02, 2022. Accessed: Dec. 08, 2022. [Online]. Available: http://arxiv.org/abs/2201.03545

Presenters: Mouri Zakir

Week 04 (30 January - 05 February)

[5] A. Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.” arXiv, Jun. 03, 2021. Accessed: Dec. 08, 2022. [Online]. Available: http://arxiv.org/abs/2010.11929

[6] Z. Liu et al., “Swin Transformer: Hierarchical Vision Transformer using Shifted Windows,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, Oct. 2021, pp. 9992–10002. doi: 10.1109/ICCV48922.2021.00986.

Presenters: Hadi Heidarirad and Reet Ghosh

[7] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation.” arXiv, May 18, 2015. Accessed: Jan. 07, 2023. [Online]. Available: http://arxiv.org/abs/1505.04597

[8] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 4, pp. 834–848, Apr. 2018, doi: 10.1109/TPAMI.2017.2699184.

Presenters: Tariq Al Shoura

Week 05 (06-12 February)

  • Monday lecture TBD

[9] A. Hatamizadeh et al., “UNETR: Transformers for 3D Medical Image Segmentation,” in 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, Jan. 2022, pp. 1748–1758. doi: 10.1109/WACV51458.2022.00181.

Presenters: Philip Ciunkiewicz

[10] J. Bassey, L. Qian, and X. Li, “A Survey of Complex-Valued Neural Networks.” arXiv, Jan. 28, 2021. Accessed: Dec. 15, 2022. [Online]. Available: http://arxiv.org/abs/2101.12249

Presenters: Natalia Dubljevic and Paula Brandt

Week 06 (13-19 February)

  • Monday lecture TBD

[11] A. Diaz-Pinto et al., “MONAI Label: A framework for AI-assisted Interactive Labeling of 3D Medical Images.” arXiv, Mar. 23, 2022. Accessed: Dec. 08, 2022. [Online]. Available: http://arxiv.org/abs/2203.12362

[12] M. J. Cardoso et al., “MONAI: An open-source framework for deep learning in healthcare.” arXiv, Nov. 04, 2022. Accessed: Dec. 08, 2022. [Online]. Available: http://arxiv.org/abs/2211.02701

Presenters: Mahsa Dibaji and Aashka Mohite

[13] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-CAM: Visual Explanations From Deep Networks via Gradient-Based Localization”.

[14] S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. Müller, and W. Samek, “On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation,” PLoS ONE, vol. 10, no. 7, p. e0130140, Jul. 2015, doi: 10.1371/journal.pone.0130140.

Presenters: TBD

Week 07 (20-26 February)

  • Reading week

Week 08 (27 February - 05 March)

  • Quiz: 27 February
  • Assignment 01 due 27 February at midnight
  • Monday lecture TBD

[15] Y. Ganin et al., “Domain-Adversarial Training of Neural Networks,” in Domain Adaptation in Computer Vision Applications, G. Csurka, Ed. Cham: Springer International Publishing, 2017, pp. 189–209. doi: 10.1007/978-3-319-58347-1_10.

[16] N. K. Dinsdale, M. Jenkinson, and A. I. L. Namburete, “Deep learning-based unlearning of dataset bias for MRI harmonisation and confound removal,” NeuroImage, vol. 228, p. 117689, Mar. 2021, doi: 10.1016/j.neuroimage.2020.117689.

Presenters: Salma Begum Tamanna and Rubya Afrin

[17] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, Jun. 2022, pp. 10674–10685. doi: 10.1109/CVPR52688.2022.01042.

Presenters: TBD

Week 09 (06-12 March)

  • Monday lecture TBD

[18] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A Simple Framework for Contrastive Learning of Visual Representations”.

[19] I. Misra and L. van der Maaten, “Self-Supervised Learning of Pretext-Invariant Representations,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, Jun. 2020, pp. 6706–6716. doi: 10.1109/CVPR42600.2020.00674.

[20] X. Liang, L. Lee, W. Dai, and E. P. Xing, “Dual Motion GAN for Future-Flow Embedded Video Prediction,” in 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Oct. 2017, pp. 1762–1770. doi: 10.1109/ICCV.2017.194.

Presenters: Abbas Omidi and Aida Mohammadshahi

Week 10 (13-19 March)

  • Monday lecture TBD

[21] Y. Wang, Q. Yao, J. T. Kwok, and L. M. Ni, “Generalizing from a Few Examples: A Survey on Few-shot Learning,” ACM Comput. Surv., vol. 53, no. 3, pp. 1–34, May 2021, doi: 10.1145/3386252.

[22] Y. Xian, C. H. Lampert, B. Schiele, and Z. Akata, “Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 9, pp. 2251–2265, Sep. 2019, doi: 10.1109/TPAMI.2018.2857768.

Presenters: Jose Cazarin and Mohammed Adnan

Week 11 (20-26 March)

  • Monday lecture TBD
  • Friday - support for finalizing the projects

Weeks 12 and 13 (27 March - 03 April )

  • Project report and 10 minute presentation recording due date: 27 March at 9 am
  • Final projects' presentations