corleytd/PyTorchBasic

This repository provides tutorial for deep learning researchers and PyTorch beginners to learn and practice PyTorch.

PyTorch Basic Practice for Beginners

This repository provides tutorial for deep learning researchers and PyTorch beginners to learn and practice PyTorch. Introduced from the foundation of PyTorch, this repo includes five parts of model training, including data, model, loss function, optimizer and iterative training visualization. In addition to the basic use of PyTorch, there are some classic hands-on projects which be used to practice, most of which are connected to Computer Vision, and one of which is related to Natural Language Processing.

Environment

the codes have been tested in:

• Python 3.8.13
• PyTorch 1.10.1
• CUDA 11.1
• Pandas 1.5.1
• Matplotlib 3.5.0

Complete dependencies can be installed by executing the command pip install -r requirements.txt.

Table of Contents

1. Basic 🚀️
   • Hello PyTorch
   • Create Tensor
   • Tensor Operation
   • Linear Regression
   • Computational Graph
   • Auto Gradient
   • Logistic Regression
2. Data Process 🍾
   • Train LeNet for RMB
   • Train LeNet for Cat and Dog
   • Transform for Images
   • Salt Pepper Noise Transform
   • Train LeNet for RMB with Augumentation
3. Modules 🍷
   • Module Containers(Sequential, ModuleList and ModuleDict)
   • Convolution Layer
   • Pool and Linear Layer
   • Convolution Practice
4. LR, Loss and Optim 🍭
   • Gradient Disappearance and explosion
   • Loss Function
   • Methods of Optimizer
   • Learning Rate
   • Momentum of Gradient
   • LR Schedulers
5. Visualization and Hook 🍦
   • Simple Use of Tensorboard
   • Tensorboard Methods
   • Train LeNet for RMB with TensorBoard
   • Model Summary with torchsummary
   • Hook Methods(Tensor Hook, Module.register_forward/forward_pre/backward_hook)
6. Regularization 👀️
   • L2 Regularization
   • Dropout Layer
   • Dropout in Train and Eval
   • Batch Normalization
   • Train LeNet for RMB with Batch Norm
   • Batch Norm in 3 dims
   • Layer Norm, Instance Norm and Group Norm
7. Utilities 🍹
   • Save and Load Model
   • Train LeNet and Save Checkpoint
   • Train LeNet with Resuming Checkpoint
   • Use CUDA in PyTorch
   • Multi GPUs and Parallel Computation
   • Load Model in GPU
   • Common Errors in PyTorch
   • Finetuning of ResNet18
   • Inference of ResNet18
8. Applications in CV and NLP 🍖
   • Image Segmentation Example(Using DeepLabV3)
   • Train Image Segmentation Using UNet
   • Inference Image Segmentation Using UNet
   • Object Detection Example(Using FasterRCNN)
   • Object Detection Using FasterRCNN
   • Train GAN for Face Generation Using DCGAN
   • Inference of Face Generation Using DCGAN
   • Name Classification Using RNN
9. Data 🍜
   • RMB classification data
   • Cat and Dog classification data: Dogs vs. Cats Redux: Kernels Edition | Kaggle
   • Ant and Bee classification data: https://download.pytorch.org/tutorial/hymenoptera_data.zip
   • Portrait Matting Data: Deep Automatic Portrait Matting (cuhk.edu.hk)
   • Penn-Fudan Data for Pedestrian Detection: Pedestrian Detection Database (upenn.edu)
   • CelebA Data for Face Generation: CelebA Dataset (cuhk.edu.hk)
   • Name Classification data: https://download.pytorch.org/tutorial/data.zip
10. Models 👍
    • LeNet
    • UNet
    • DCGAN
    • RNN
11. Tools 🍬
    • custom datasets
    • other common tool functions

Run Example

The following are some running examples:

• Loss Curve of Cat and Dog Classification with LeNet

  

• Weight of Convolution Layer in LeNet from Tensorboard

  

• Ant and Bee Classification with ResNet

  

• Portrait Matting with UNet

  

• Object Detection with FasterRCNN

  

• Face Generation with GAN

  

• Name Classification with RNN

  

Data

The whole data has been uploaded to 2 mainstream online storage platform:

• Google Drive: https://drive.google.com/file/d/1ctoJyZB5rNJPfaNHPfDaRsyemkgTejPg/
• Baidu Netdisk: https://pan.baidu.com/s/1e0zuAyXc1ni5L9WD9cAjHQ?pwd=2cyq

Download the data form one of the platform, unzip it and rename it to data, then place it into the project home directory, and you can run the projects successfully.

Feedback

If there are some problems or good suggestions about the project, you can email me by cutercorleytd@gmail.com or submit issues on this repo.