Deep Learning for Identifying Iran's Cultural Heritage Buildings in Need of Conservation Using Image Classification and Grad-CAM

This repository contains the code and data of an eponymous paper. In an endeavor to practically demonstrate the utilities of DL in CH literature, We developed a fully fledged DL model that classifies the images in need of conservation and even more approximately localizes the defects to help the CH practitioners identify defects in a timely manner, and as a result speed of the process of CHB conservation as well as increasing its accuracy. In spite of all the limitations, we achieved very good results with a score of at least 94% for Precision, Recall, and F1-Score, which were about 4-5% more than similar works (See Table 4 in the preprint v1).

You can download, and cite 😉, the paper from .

Requirements

Please refer to the file requirements.txt for a comprehesive list of packages and their corresponding version.

Project Dir Structure

.
├── data
│   ├── NO restoration
│   └── YES restoration
├── images
│   └── logos
├── logs
├── models
├── outputs
│   ├── Feature Maps visualization
│   ├── Grad_CAM outputs
│   └── inference
├── Reports
└── utils

13 directories

Pipeline Schema

Data

We have two/2 classes, namely (1) no restoration (class 0) and (2) need restoration (class 1) (See Fig [1] ).
Download the dataset here.

The entire dataset will be avaible to download (in the foreseeable future)!
For the detail regarding how the raw data are processed, please refer the utils/data_augmentation.py. (See Fig [2] )

Fig [1] : A few sample images which show the complexity, diversity and variation of our data.

Fig [2] : An example of applying the proposed data augmentation methods on a train image (i.e., nine times). Notice how random, realistic, and valid the augmented versions are.

Citation

@article{bahrami2023deep,
author = {Bahrami, Mahdi and Albadvi, Amir},
title = {Deep Learning for Identifying Iran’s Cultural Heritage Buildings in Need of Conservation Using Image Classification and Grad-CAM},
year = {2023},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
issn = {1556-4673},
url = {https://doi.org/10.1145/3631130},
doi = {10.1145/3631130},
journal = {J. Comput. Cult. Herit.},
month = {oct},
keywords = {deep learning, convolutional neural networks (CNN), built cultural heritage conservation, Structural health monitoring, image processing, gradient weighted class activation mapping (Grad-CAM), transfer learning}
}

Contact

Should you have any questions, feel free to contact TekBoArt @tekboart.

License

Shield:

Refer to the file LICENSE for more information regarding the license of this repository.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

tekboart/CHB-image_classification_GradCam