Why kendall's τ and Spearman's ρ evaluate why Siamese regression has a negative value
Opened this issue · 2 comments
@yunshengb Hello, I'm very sorry to disturb you at this time. When I run the project, I found that Siamese classification has negative values on Kendall's_and Pearman's rho. At the same time, the effect of MSE is 10 times higher than that mentioned in the paper. Is there any parameter that needs to be adjusted?
In addition, I would like to consult prec@k and prec@k_0.005, the abscissal and vertical coordinates of these two graphs are not clear, and what is the relationship between them and the p@10 and p@20 proposed in the paper?
I really hope to hear from you. Thank you.
We have rerun our experiments several times and did not find any issue. To ensure you follow the exact same setup, please check this link:
https://drive.google.com/drive/folders/12NINRPyyNeQdhMmEV56KDJNVDN2T2gL8?usp=sharing
This time we save all the data and the complete configurations in that zip file, so you can simply run the model without any further tuning.
prec@k is the one we use, specifically, prec@10 and prec@20, i.e. we truncate the top 10 and 20 graphs retrieved by our model and compare against the ground-truth. In contrast, rho and tau are global metrics, i.e. we compare the entire ranking of all the training graphs against the ground-truth.
Let me know if you have any further issue.
Thanks,
Yunsheng
Thank you very much for your feedback.
I may not have made it clear, but the project I'm currently running is called SIMGNN, and I'd like to ask how the configuration is configured in here.
After I finished the operation, an effect drawing of MRR appeared. May I ask what is the meaning of MRR? It was not put forward in the paper.