valid and test about ACDC

[kathyliu579]

Hi, i have some questiones about ACDC:

1. why can do validation and test with the similar way of training? cause i saw you use batch size=1, and "test_single_volume" procedure
2. can you share the version of test that outputs the values of LV MYO RV?

thanks a lot. your work help me a lot!

[Dootmaan]

hi @kathyliu579.

1. the validation and test are actually conducted in two different ways (they are all embedded in the training process tho). validation is conducted similar to training, which accepts a 2D slice (batch_size=1 here means 1 slice one time) as input and evaluates its DSC with its 2D gt. validation is used for model selection so we just use this simple implementation. test, however, is much more formal since we have to compare with previous works. in this field, each 3D medical image is viewed as a single test case, so we must computes the DSC of the entire 3D image (batch_size=1 here means 1 3D image one time and a single 3D image may contain many slices) as the final result and thats exactly what "test_single_volume" does.
2. Current the code already outputs the values of LV MYO RV. They are marked as class 1, class 2 and class 3 in the training log.

[kathyliu579]

thanks Dootmaan. i carefully check the code. i realized what you say now. but for 1 th, i still has some comfusion.
Why test (computes the DSC of an entire 3D image) will abtain a higer DSC? like valid get 0.84, but test can get 89

[Dootmaan]

theoretically it is possible. however usually we will train the model several times to find the one with highest validation score. such model may be overfitted to validation set and tends to perform a little worse on test set. however, if the test set have very similar distribution with validation set or the model you choose happens to be more suitable for the test set, then the situation you described might just happen.

[Dootmaan]

This issue is closed since no further activity has happened for a while.