HansBambel/SmaAt-UNet

The result of test_precip_lightning.py

yyyyqy opened this issue · 6 comments

yyyyqy commented

After training UNet and other models separately, I changed the model_floder in the test_precip_lightning.py file as a way to change the different models, but I ended up with the same evaluation results. Is there a detail I'm not paying attention to?

HansBambel commented

The function get_model_losses uses every .ckpt-file and calculates the loss for each model. This was mainly used for comparing different models that were compared in the paper.

When you have only a single model (.ckpt-file) in the folder the loss will only be calculated for that model.

What do you have in your model_folder?

yyyyqy commented

When using the UNet model for prediction, my model_folder is: "F:/SmaAt_UNet_master/metric/UNet/", which has the ckpt files for 20 epochs as follow:
1676560392125
When using the UNetDS_Attention model for prediction, my model_folder is: "F:/SmaAt_UNet_master/metric/UNetDS_Attention/", which has the ckpt files for 20 epochs as follow:
1676560496806
Can you tell me what is wrong with it?
Thanks!

HansBambel commented

Aah okay. You should place the best (or latest) file from each model in a separate folder. Then link that folder in test_precip_lightning.py.

Best here was for me the one with the lower validation error.
Then only those two models are compared.

yyyyqy commented

Thanks, I know how the file should be put now. I looked at your code again and found that the code only has persistence's percision and other indicators. I will finish the rest by myself.

  • 👍1
daralevian commented

Thanks, I know how the file should be put now. I looked at your code again and found that the code only has persistence's percision and other indicators. I will finish the rest by myself.

sorry to border，if I can ask you some question in private？I can‘t find your e-mail address

yyyyqy commented

Thanks, I know how the file should be put now. I looked at your code again and found that the code only has persistence's percision and other indicators. I will finish the rest by myself.

sorry to border，if I can ask you some question in private？I can‘t find your e-mail address

Sure, you can contact me at the following email address: cychyao@outlook.com