Reproduce the result

Question

Reproduce the result

Closed this issue 5 years ago · 4 comments

Hi @Nathanua , thanks for your great work. Recently I am working on Salient object detection and I was trying to re-train BASNet from scratch to benchmark our own work. However I found I cannot reproduce reported result. I got around 0.078 in MAE on DUTS-TE(Trained on DUTS-TR), and I used the evaluation code you provided here https://github.com/NathanUA/Binary-Segmentation-Evaluation-Tool
Could you please give me some hint or explanation? thanks in advance!

Answer 1 · 2020-01-04T06:45:58.000Z

Did you augment the DUTS-TR with horizontal flipping offline? How long did you trained?

…

On Fri, Jan 3, 2020 at 10:54 PM vandesa003 ***@***.***> wrote: Hi @Nathanua <https://github.com/NathanUA> , thanks for your great work. Recently I am working on Salient object detection and I was trying to re-train BASNet from scratch to benchmark our own work. However I found I cannot reproduce reported result. I got around 0.078 in MAE on DUTS-TE(Trained on DUTS-TR), and I used the evaluation code you provided here https://github.com/NathanUA/Binary-Segmentation-Evaluation-Tool Could you please give me some hint or explanation? thanks in advance! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#31>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADSGORLGCX7H73U2BETHE4LQ4AQDBANCNFSM4KCUP46Q> .

-- Xuebin Qin PhD Candidate Department of Computing Science University of Alberta, Edmonton, AB, Canada Homepage:https://webdocs.cs.ualberta.ca/~xuebin/

Answer 2 · 2020-01-06T01:56:28.000Z

Yes, I augment the DUTS-TR as the paper mentioned. I trained for around 24 epochs.

Answer 3 · 2020-01-06T03:04:08.000Z

The problem could be that your training iteration (24 epochs =~ 60k iterations) is much fewer than that mentioned (400k) in our paper. Please try to train with more iterations. Around 400k(125hrs on 1080ti)should be good.

…

On Jan 5, 2020, at 6:56 PM, vandesa003 ***@***.***> wrote: Yes, I augment the DUTS-TR as the paper mentioned. I trained for around 24 epochs. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

Answer 4 · 2020-01-06T09:19:09.000Z

@Nathanua Thank you for the reply! I will try it. Just astonished by the learning capability of this network, I thought this many of epochs will make the network overfitting. So is it the Refinement Module enable the further improvement? Anyway, thanks for your hint.