What relations in Chemprot are evaluated in the paper ?
yyzhuang1991 opened this issue · 0 comments
yyzhuang1991 commented
I ran the script for relation extraction in Chemprot. It looks like the accuracy includes all relations available in Chemprot, while the original publication of Chemprot and other relevant publications like bioBERT evaluated over only 5 relations ( specifically, AGONIST, ANTAGONIST, SUBSTRATE, UPREGULATOR and DOWNREGULATOR ). So I wonder whether the SciBERT paper reported the accuracy (83.64) over these 5 relations, or all relations?