by Pranav Eranki - pranav.eranki@gmail.com
- Get the images from ISIC.
- Download all the images (This will take a while).
- Format the folders so that all the images are in two folders labeled Benign and Malignant if they are from the benign and malignant folder respectively.
- Download this project, and unzip all the files into the same directory as the Benign and Malignant folders.
- Now, you should have all the .py files and your image folders in the same directory.
- First, fire up an IDE (this makes the process smooth) - I prefer Spyder from Conda.
- Navigate over to the downloaded directory
- Execute the imagePrep.py file.
- This assumes that you have already done all the necessary steps in the Installation section above.
- If you want to change the number of images desired to process, be my guest. Go into the file 'moleimages.py' and change the valuie of the variable max inside the resize_bulk function. Just please do not change the value to a number higher than 12000 if you have 16GB RAM - this gives a memory error. If you have an even smaller RAM, please change the values as needed according to your judgement.
- This might take a while depending on the number of images you set as max in the moleImages.py file.
- Run the topVGG.py file to train the optimal VGG Model for the image classification.
- Run the FullVGG.py file to get the best full CNN:
- Change the hyperparameters epochs, batch size, and learning rate as necessary.
- Change the nb_train_samples and nb_validation_samples to match the number of samples in your train and validation folders.
- Save and compile the full program.
- Please keep in mind that with 2000 images of each type, a learning rate of 0.001, 10 epochs, and a batch_size of 16, the training of the program took a little under an hour.
- If you want to test your model, run the testModel.py file
- If you want to test out a new model, delete the models folder, change the fullVGG.py file as needed, then rerun(in order) the topVGG.py file, the fullVGG.py, then the testModel.py file.
- You can delete the rocCurve.png file which is generated by the testModel.py file if you wish.
- Run app.py from your command line / terminal using the line
python app.py, which will allow you to open up a localhost with the website version of the app loaded up. The port should be 127.0.0.1:5000. - Once on the web version, upload an image and try the model all you like!
The purpose of this project is to create a tool that considering the image of a mole, can calculate the probability that the mole indicates the presence of melanoma within that user.
Skin cancer is a common disease that affect a big amount of peoples. Some facts about skin cancer:
- Every year there are more new cases of skin cancer than the combined incidence of cancers of the breast, prostate, lung and colon.
- An estimated 87,110 new cases of invasive melanoma will be diagnosed in the U.S. in 2017.
- The estimated 5-year survival rate for patients whose melanoma is detected early is about 98 percent in the U.S. The survival rate falls to 62 percent when the disease reaches the lymph nodes, and 18 percent when the disease metastasizes to distant organs.
- Early detection is critical!
The idea of this project is to construct a CNN model that can predict the probability that a specific mole is malign (indicative of skin cancer).
-
Spyder for writing code
-
Jupyter notebook for EDA
-
Git for versioning code
-
The ever-helpful StackOverflow community
-
Coding language : Python
- Modules :
- Tensorflow
- keras
- Python
- matplotlib
- scikit-learn
- Flask or Django for web
- Modules :
The following preprocessing tasks are developed for each image:
- Image resizing: Transform images to 128x128x3
- Crop images: Automatic or manual Crop
- Other to define later in order to improve model quality
The idea is to develop a CNN model from scratch, and evaluate the performance to set a baseline. The following steps to improve the model are:
- Data augmentation: Rotations, noising, scaling to avoid overfitting
- Transferred Learning: Using a pre-trained network construct some additional layer at the end to fine tuning our model. (VGG-16, or other)
- Full training of VGG-16 + additional layer.
To evaluate the different models we will use ROC Curves and AUC score. To choose the correct model we will evaluate the precision and accuracy to set the threshold level that represent a good tradeoff between TPR and FPR.
As mentioned before, the idea is to generate a tool to predict the probability of a malign mole. To do it, I'm planning to provide the following resources:
1. Web App: The web app will have the functionality that if the user uploaded a high quality image of an specific mole, the result will be a prediction about the probability that the given mole is malign in terms of percentage.
2. Iphone App: If time allows, and if my friends are willing.
3. Android App: Also if time allows.
- IoS swift + core ML (for possible iOS app)
- Android Studio (for possible android model)
The documentation for this program is fairly self explanatory.
A lot of the code which is written is fairly documented with comments.
Please keep in mind that if you wish to understand a lot of these topics through the use of the comments, you would need some ML experience. The comments are helpful, but do not explain the full extent of the code.
This project has been a bitter-sweet mix of compromises, (personal) breakthroughs in my code, and hours spent debugging. The experience gained from this project is genuinely uncomparable to any of the other projects which I have undertaken previously.
I have learned so much from this project, including:
- Image manipulation
- Reading/saving images from/to a folder in a procedural manner
- How to use Keras to construct a variety of different CNNs
- How to format documentation for a project in an official manner
- How to use .h5 files
- How to use Flask
- Scaling the data properly
- Eventually, I utilized a lot of documentation on skimage to save images
- Saving the scaled images to a folder
- This took many tries to get the image saving to work
- A lot of stack overflow was used
- Tensorflow installation issues on Windows AND Linux
- This is a funny story - I literally converted one of my computers to Ubuntu solely to work on tensorflow, and I STILL got import issues.
- Solved this by following many different threads on github until I found a fix.
- Saving bottleneck features to a byte format and reading from same format
- Solved by removing feature saving and implementing all VGG16 model training in 1 function
- Then, saved the model weights of the best VGG16 implemented to a folder for later use
- Training the full VGG model effectively
- With 10,000 images, 100 epochs, and a small learning rate, the model was projected to take about 50 hours
- Used substantially less images, 10 epochs, and a larger learning rate - model was projected to take about 1 hour
- Saving the full model effectively to be used in testing
- Used json to convert model and weights then get them again when running testModel.py
- Tried many other formats, none worked
- Utilizing Conda environment to migrate project to Windows
- Problems with using opencv
- Used skimage.io instead
- Needed to rewrite image saving code for skimage
- Create Ios App
- Create Android App
- Make a more complex documentation flow with multiple docs
This tool has been designed only for educational purposes to demonstrate the use of Machine Learning tools in the medical field. This tool does not replace advice or evaluation by a medical professional. Nothing in this project should be construed as an attempt to offer a medical opinion or practice medicine.