reproduce mIoU
changwenkai101 opened this issue · 2 comments
Thank you for your project. We deployed this project on the newly installed 8GPU server and intend to run all the demos. We did not change the configs and tested three demos: DeepLabv3_plus_resnet101_cityscape (≈78.28), DFANet_xceptionA (≈59.10), Fast_SCNN (≈60.08), the latter two are far from the original results, so the experiment was stopped.
For some well-known reasons, the results of the AI paper are not easy to reproduce, but we noticed that you mentioned in a previous reply that the configuration of a demo needs to be modified mode = 'testval', whether each demo should modify some code in order to reach the mIoU listed in the original text or the project? Or every configs need not change,the low mIoU just because the original results are difficult to reproduce?
- Fast scnn can easily reproduce the result of paper mentioned by using our configs. you just need to set total batch size=12. batch size in config is img of a single gpu. #18
- I didn't train dfanet, also can not reproduce the speed that paper mentioned. Maybe you should dive into the paper for better result.
- you do not need to pay attention to mode if you train on cityscape dataset. because the size of images are always (1024, 2048)
Thank you very much for your reply. According to your suggestion, we reduced the batch of Fast_SCNN and achieved the original accuracy. Similarly, the Deep_lab_mobile was almost the same as the original one.