/CNN-VIS

Primary LanguagePython

Visualization of Convolutional Neural Networks


Overview | Features | Requirements | Workflow | Slides and Report


This is the Project which we did during our internship at Computer Vision Lab Dresden under the guidance of People at CVLD and CGV.

Overview

The primary goal is to Visualize interpret the neural nets for a deeper understanding. A better understanding of these models could lead to a more handcrafted design of CNN’s, higher efficiency or better performance as achieved by Zeiler et al. We divided the Visualization tasks into two broad categories:

  • Activation of Filters
  • Gradient Based Visualization Both 2D as well as 3D visualization was performed in each case.

⬆️

Features

  • 2D VISUALIZATION OF ACTIVATION OF FILTERS run python 2d3d-unordered_vis/2Dunordered.py to get the 2d output of conv1 layer filters.
  • 3D VISUALIZATION OF OUTPUT FROM EACH NEURON Here the binary output of each of the output of conv1 filters were generated by implementing the code 2d3d-unordered_vis/layer_output_vox/layer_output_bin.py and were visualized using CGV 3D Viewer.
  • Visualizing by ordering We tried to reorder the output of conv1 layer filters in various ways:
  • The first by performing k-means clustering by taking any one slice of all 96 filters of conv1 layer.
  • The second was doing PCA on the similar rows of image slice. The rows are then reordered on the basis of the values of projection on the principal component in increasing order. This can be obtained by running the code ordering_layer1/pca_image/kmeans_pca.py.
  • Ordering for whole output images instead of rows in a slice from the volume. Here the output images are stacked column wise to form a vector and Principal Component Analysis (PCA) is done on them. The vectors are then reshaped back to images and are reordered accordingly in the volume. This volume of reordered images are visualized using the cgv-viewer. The code to perform this task is ordering_layer1/pca_image/pca_image.py. A sample screenshot of the reordered volume visualization of conv1 layer is shown below:

  • Visualization of Gradients In this method we calculated the gradients of the output of final layer with respect to the filter weights and visualized them as 3D volume. The gradient calculation can be found at gradient/grad_calc.py Demo of variation of gradient among images belonging to a reference class, the classifying neuron of this class being the neuron or classification under inspection, a similar class and a dissimilar class can be found here:
  • DEMO:

⬆️

Requirements

  • Caffe: We used pycaffe (caffe interface in python) to perform all the tasks involving CNNs.
  • For 3D visualization we used CGV 3D viewer which is proprietary of the institute

⬆️

Work Flow

We follwed the following workflow during the project:

⬆️

Slides and Report

⬆️