Repository for the Workshop "Deep Learning and Visual Media", Tue 3 March 2020, DHd 2020

Installation

-1. Installing Detectron2 on Linux systems is pretty straightforward.

For Windows 10, a relatively easy solution is to install the Windows subsystem for Linux, instructions here: https://docs.microsoft.com/en-us/windows/wsl/install-win10. You will have to install the following packages as well (in the shell): python, python-dev and opencv-python.

For OS X, you can try to follow the Anaconda instructions posted here: https://medium.com/deepvisionguru/how-to-embed-detectron2-in-your-computer-vision-project-817f29149461. Please note that this is not yet compatible with our demo.py script, we hope to fix that soon.

Clone this repository and enter folder (or download, extract and enter folder):

git clone https://github.com/ghowa/dhd2020.git

cd dhd2020

Create virtual environment to make sure we don't mess with your system python install and install all needed packages:

If you have a Conda python install, try this:

conda create --nsame detectron2

conda activate detectron2

For vanilla python, try this:

python -m venv detectron2

source detectron2/bin/activate

pip install -r detectron2/requirements.txt

Install precompiled Detectron with CPU support only:

pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cpu/index.html

OR: Install precompiled Detectron for CUDA 10.1:

pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/index.html

Create a new Jupyter kernel which uses the virtual environment you have just created:

ipython kernel install --user --name=detectron2

Download Labelme2COCO converter and make it executable:

curl -JLO https://raw.githubusercontent.com/wkentaro/labelme/master/examples/instance_segmentation/labelme2coco.py

chmod 700 labelme2coco.py

Try out pretrained models

Copy your sample images to dhd2020/input
Run the following network from Detectron's model zoo:

python detectron2/demo.py --config-file detectron2/lib/python3.7/site-packages/detectron2/model_zoo/configs/COCO-PanopticSegmentation/panoptic_fpn_R_101_3x.yaml --input input/* --output output  --opts MODEL.DEVICE cpu MODEL.WEIGHTS detectron2://COCO-PanopticSegmentation/panoptic_fpn_R_101_3x/139514519/model_final_cafdb1.pkl

Other interesting models:

python detectron2/demo.py --config-file detectron2/lib/python3.7/site-packages/detectron2/model_zoo/configs/Cityscapes/mask_rcnn_R_50_FPN.yaml --input input/* --output output  --opts MODEL.DEVICE cpu MODEL.WEIGHTS detectron2://Cityscapes/mask_rcnn_R_50_FPN/142423278/model_final_af9cf5.pkl

python detectron2/demo.py --config-file detectron2/lib/python3.7/site-packages/detectron2/model_zoo/configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml   --input input/* --output output  --opts MODEL.DEVICE cpu MODEL.WEIGHTS detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl

python detectron2/demo.py --config-file detectron2/lib/python3.7/site-packages/detectron2/model_zoo/configs/COCO-Detection/faster_rcnn_R_101_FPN_3x.yaml --input input/* --output output  --opts MODEL.DEVICE cpu MODEL.WEIGHTS detectron2://COCO-Detection/faster_rcnn_R_101_FPN_3x/137851257/model_final_f6e8b1.pkl

python detectron2/demo.py --config-file detectron2/lib/python3.7/site-packages/detectron2/model_zoo/configs/COCO-Keypoints/keypoint_rcnn_R_101_FPN_3x.yaml --input input/* --output output  --opts MODEL.DEVICE cpu MODEL.WEIGHTS detectron2://COCO-Keypoints/keypoint_rcnn_R_101_FPN_3x/138363331/model_final_997cc7.pkl

python detectron2/demo.py --config-file detectron2/lib/python3.7/site-packages/detectron2/model_zoo/configs/LVIS-InstanceSegmentation/mask_rcnn_R_50_FPN_1x.yaml --input input/* --output output  --opts MODEL.DEVICE cpu MODEL.WEIGHTS  detectron2://LVIS-InstanceSegmentation/mask_rcnn_R_50_FPN_1x/144219072/model_final_571f7c.pkl

Analysis

Run Jupyter notebook

jupyter notebook

Open deep_watching.ipynb and make sure the 'detectron2' kernel is used

Annotate

Copy training images to train/ and test images to test/
Add your own categories to the text file 'labels'
Run Labelme

labelme train/ --labels labels

labelme test/ --labels labels

Convert into COCO json

./labelme2coco.py --labels labels train/ train-coco/

./labelme2coco.py --labels labels test/ test-coco/

Test annotations in Jupyter notebook Run Jupyter notebook

jupyter notebook

Open coco_test.ipynb and make sure the 'detectron2' kernel is used

Train

Create yaml for Detectron2
Register custom train/test set to Detectron2
Run training