Deep Learning Classifier based on a Convolutional Neural Network which recognizes 5 types of paintings
The cleaned dataset contains roughly 8k usable pics in .jpg and .png format. These are divided in 5 classes:
Drawings:
Engravings:
Iconography:
Paintings:
Sculptures:
The definitive model architecture consists of 3 Convolutional layers with max pooling, a Dense final layer and Dropout with p=0.4 after every hidden layer (Convolutional and Dense both).
The model has been tested with a train set and a validation set (split of 20%) for 35 epochs, with an initial learning rate of 0.1 and a ReduceOnPlateau Scheduler to find the minima in the loss landscape. The final results are 85% accuracy on test set.
Looking at the confusion matrix, it is clear that drawings and engravings are the most mispredicted classes for the model.
Engravings get confused with drawings and vice versa, and drawings get also confused with sculptures. While the first two cases are quite understandable, since many engravings images are quite similar to drawings, it surprises that sculptures get confused with drawings. Although if we take a look at some examples, we can see that there are sculptures with pencil-like features, like this one:
and others (due to bad lighting and/or resolution) with few details and almost solid colors like this:
The accuracy score can surely get improved starting by these considerations, using data augumentation or slightly deepening the model. Also, the dataset is probably too small to properly train a CNN from scratch, so probably transfer learning (+ fine tuning) could probably give a better result.
Trained a custom model made upon MobileNetV2 (roughly 2 millions paramater, 1/3 of the CNN made from scratch) on the same dataset. It performed a 94% accuracy on evaluation, against 85% accuracy on the old model.