cannot produce the results in table 4
Closed this issue · 4 comments
Hello
Thank you for the great work.
I am trying to re-produce the results in table 4 of the paper however I cannot.
-
should we test the model only on the validation set of CLIVE and KonIQ or on the whole dataset?
-
can you share the trained models,
P2P-BM (Sec. 4.1) , P2P-RM (Sec. 4.2), and P2P-FM (Sec. 4.4), that can produce those results?
The results that I get from running the pre-trained model are far less than what is reported on the paper.
Hi, thank you for reporting.
We faced legal issues trying to make public a considerable number of images. Therefore, we removed those images.
We have to re-split the database and retrain the model after. We didn't share the original pretrained models because the split is different.
However, the model should give a similar performance. I am writing a journal paper and more pre-trained models will be released soon. Also note the performance also vary with the split of CLIVE and KonIQ, we will report the performance on the whole database instead in the journal paper.
Thank you again and more discussion is welcome!
Thanks for the reply.
I understand that the new version will come out soon, but for the time being, can you please clarify regarding the ratio of the splits of CLIVE and KonIQ that is used for testing?
Yes, I am writing a document for that. Just uploaded the original pretrained model and will publish the google colab soon
@seyeeet Please follow this google colab to reproduce the results. Thanks!