About the evaluation scripts of this model

[fwzlaughing]

I have run the code and evaluated it on the test set. But I found that the accuracy of the result is only 0.026. The scripts of computing the accuracy are implemented by myself, and I suspect that there are some bugs in my code, could you provide your evaluation scripts？ thanks very much！

[StevenWD]

I upload my evaluation code into the repository for your reference.

[fwzlaughing]

Thanks. However, I am a little confused about your evaluation code. I think that the evaluate metric of this task is the "relation detection accuracy", i.e. #(correct relation rank first)/ #(all questions). But in your code, you calculate the "ranking accuracy" which indicate how many negative relations rank behind the correct relations.

[StevenWD]

I think either way is ok, you can easily modify the evaluation code to calculate the "relation detection accuracy" you mentioned above.

Thank you for your question. I'm closing this issue.