bowang-lab/U-Mamba

Worse results of the released model on Dataset701_AbdomenCT

sms95 opened this issue · 3 comments

sms95 commented

Thank you for releasing the code and the model.

I have some questions on the testing results of Dataset701_AbdomenCT, the quantitive results are much worse than the results reported in your paper. I got 0.5408956923076923 DSC and 0.5480656923076923 by using your released model directly for inference.
I also checked and visualized the ground truth and prediction on the test set,
image

BTW, it seems the images in the train and test set of Dataset701_AbdomenCT are collected by following different settings.

image

I wonder am I doing something wrong or I missed some preprocessing steps, or I have just downloaded the wrong dataset?

Thanks!

JunMa11 commented

Hi @sms95 ,

Thanks for raising this issue.

There was a bug in resampling. We have fixed it now.

We also provide a video tutorial to show the

  • installation
  • inference
  • computing metrics

https://drive.google.com/drive/folders/1NRPOsz0hMdQKoZeJQPUFgzkkFX3phUsL?usp=sharing

Hope the answer helps. Please feel free to let us know if you have any other questions.

  • 👍2
beasthunter758 commented

I am trying to replicate this video for cell metrics evaluation but always getting this error, can you share your command line code for evaluating?
python compute_cell_metric.py -g /home/xim/Downloads/Pratik/U-Mamba/data/nnUNet_raw/Dataset705_ER/imagesVal -s /home/xim/Downloads/Pratik/U-Mamba/data/nnUNet_raw/Dataset705_ER/Predicted -o /home/xim/Downloads/Pratik/U-Mamba/evaluation/CBAM -n CBAM2
num of files: 0
compute metrics at threshold: 0.5
0it [00:00, ?it/s]
/home/xim/anaconda3/envs/umamba2/lib/python3.10/site-packages/numpy/core/fromnumeric.py:3504: RuntimeWarning: Mean of empty slice.
return _methods._mean(a, axis=axis, dtype=dtype,
/home/xim/anaconda3/envs/umamba2/lib/python3.10/site-packages/numpy/core/_methods.py:129: RuntimeWarning: invalid value encountered in scalar divide
ret = ret.dtype.type(ret / rcount)
CBAM2-0.5.csv threshold: 0.5 mean F1 Score: nan median F1 Score: nan
CBAM2-0.5.csv failed cases: []

sms95 commented

Hi @sms95 ,

Thanks for raising this issue.

There was a bug in resampling. We have fixed it now.

We also provide a video tutorial to show the

  • installation
  • inference
  • computing metrics

https://drive.google.com/drive/folders/1NRPOsz0hMdQKoZeJQPUFgzkkFX3phUsL?usp=sharing

Hope the answer helps. Please feel free to let us know if you have any other questions.

Problem solved, thanks!

  • 👍1