/do-you-see-what-AI-see

Evaluation of a simple CNN model for glaucoma detection trained on a single public dataset against complex architectures trained on multiple public/private datasets

Primary LanguageJupyter Notebook

Glaucoma detection from fundus images

For a detailed introduction into eye diseases, glaucoma, and previous literature, please refer to this presentation.

Retinal fundus image of a glaucomatous eye

About the project

This project is part of the Ironhack Data Analytics Remote Part Time bootcamp and was completed in 3 weeks. The purpose of the project is to build a neural network that detects glaucoma from retinal fundus images and compare its results on a novel dataset to already established architectures.

Technical requirements

Please refer to the requirements.txt.

Machine learning process

The project was separated in multiple Jupyter notebooks explaining each specific part of the project, which you can find below:

  1. Data Collection & Exploration
  2. Data Splitting, Augmentation, and Preprocessing
  3. Simple Neural Network Training
  4. Evaluation of Multiple Neural Networks on a New Dataset
  5. Learning Resources

Project Challenges

  1. Selecting a suitable project
  2. Acquiring the knowledge to finish the project
  3. Choosing an effective training strategy
  4. Improving simple model prediction accuracy
  5. Cropping the test images for evaluation

Lessons Learned

  1. Do your research before choosing the project - you might realize that what you set out to do is not as impactful as you initially thought.
  2. Collect data early - it takes a while for images to download.
  3. Keep backups for your data before you start processing it - e.g. `shutil.move()` can be a dangerous command if used improperly.
  4. Ensure you're able to run your project in the cloud (either via Google Colab or a different cloud provider), as training might be too taxing for your local machine.
  5. Try to test your data on a real life scenario as soon as possible, so you can discard poorly trained models.
  6. Check the changes you would undo before resetting to a previous git version - and if you don't, make sure you have backups.

Limitations

  1. The deep learning model might have trouble with images where the optic nerve head is damaged.
  2. There are few publicly available labelled datasets.

Results

The simple CNN model (G: 94%, N: 3.6%) is significantly outperformed by the more complex architectures (avg G: 63%, avg N: 97.7%), which have performed relatively well on an unseen dataset.

Conclusions

We can conclude that model architecture, data preprocessing, and an extensive training dataset are key to the performance of a disease detection algorithm, as shown by the difference in performance between a minimum viable model and more complex models.

References

[1] A global review of publicly available datasets for ophthalmological imaging: barriers to access, usability, and generalisability - Saad M Khan, Xiaoxuan Liu, Siddharth Nath, Edward Korot, Livia Faes, Siegfried K Wagner et al.
[2] ACRIMA dataset (cropped images)
[3] Cataract dataset (full images)
[4] Joint Optic Disc and Cup Segmentation Based on Multi-label Deep Network and Polar Transformation - Huazhu Fu, Jun Cheng, Yanwu Xu, Damon Wing Kee Wong, Jiang Liu, and Xiaochun Cao
[5] CNNs for Automatic Glaucoma Assessment using Fundus Images: An Extensive Validation - Diaz-Pinto, A., Morales, S., Naranjo, V. et al.

Acknowledgements

Thank you Xisca & Arek for the support, inspiration, and encouragement throughout the bootcamp and especially during the final project ⭐