about data
mrzhangzizhen123 opened this issue · 11 comments
When I was running your code, I found that the data didn't match. Could you explain it to me?in eyepacs.sh.
bin2_0_cnt=48784
bin2_0_tr_cnt=40688
bin2_1_tr_cnt=16458
What is the relationship between 40688 and 48784?
bin2_0_cnt is the number of non-rDR images for both training and test.
bin2_0_tr_cnt is the number of non-rDR images for training only.
There is one more question that I need to ask you: when you were training, what was the value of AUC shown in each epoch?
Sorry, I can't help you with that. We didn't keep logs of AUC during training. The AUC also varies for each trained model, so there is no definite answer to what AUC should be after each epoch.
Hello, there is a problem in your experiment.A total of 88702.Among them, there are 65345 pictures in the 0 label, 6205 in the 1, 13153 in the 2, 2085 in the 3, and 1914 in the 4. When I executed eyepacs.sh --redistribute, I found 40374 pictures in the 0 and 16458 pictures in the 1, which did not conform to bin_0_tr_cnt=40688. May I ask how to solve this problem?
@mrzhangzizhen123 I am not able to reproduce your results. I always end up with 40688 images in the 0-folder, and 16458 images in the 1-folder, as expected. Have you modified anything in the eyepacs.sh script?
I have modified bin_0_cnt and so on according to the existing data. Other places have not moved. Now when running train.py, the first column of the 2*2 matrix is all 0.Is the data not well distributed？
@mrzhangzizhen123 What are the values of the modified parameters in eyepacs.sh?
bin_0_cnt represents the total amount of images used from the EyePacs data set (retrieved from the pool). bin_0_tr_cnt represents how many of these images are used for training. If you have modified the amount of images in the data set by separating the images for left and right eyes, these numbers will be different. The eyepacs.sh script is written specifically to fit the needs in our project. It yields an equal ratio of rDR/non-rDR images as in the original JAMA project.
When you modify the data set, in your case by separating the images for left and right eyes, I do not think eyepacs.sh will be useful for you. Please read this section in our README to preprocess your data sets correctly to be used with train.py.
Yes. You can use the Messidor-2 or Messidor-Original test sets, and run evaluate.py with the -m2 og -m parameter, respectively. If you want to use our EyePacs test set, it requires unpacking the training set too. The evaluation script loads the trained model and expects its architecture to be the same as built in the train script. Evaluating with the original EyePacs test data is done with the following steps:
- Run the eyepacs.sh script and use --output_dir
./eyepacs.sh --output_dir=./eyepacs_data - Run the evaluate.py script with your trained model:
python evaluate.py -e -lm=/your_model --data_dir=./eyepacs_data/test