There is a little gap between my results and those in the paper

Question

There is a little gap between my results and those in the paper

hht1996ok opened this issue 2 years ago · 3 comments

hht1996ok commented 2 years ago

First, thank you for your work, I ran 300 epochs according to your code, however the OA was 93.85 and mAcc was 91.24, I would like to ask why there is a little gap between paper.

Answer 1 · 2022-05-10T15:24:42.000Z

@hht1996ok please see # #1 (comment)

It is common to get different results by runing same codes for ModelNet40 benchmark due to the randomness.

You can get the same results using provided checkpoint.

Answer 2 · 2022-05-10T15:27:25.000Z

@hht1996ok Also, please see News & Updates in README.md:
⏩ Questions on ModelNet40 classification reproduction
The performance on ModelNet40 of almost all methods are not stable, see (CVMI-Lab/PAConv#9 (comment)). If you run the same codes for several times, you will get different results. Also, the randomness of ModelNet40 is our motivation to experiment on ScanObjectNN benchmark, and to report the mean/std results of several runs.

Answer 3 · 2022-05-10T15:31:02.000Z

Thank you for your reply！