Error or not : Loss about Relation Classification
HuizhaoWang opened this issue · 2 comments
Hi,
Thanks for sharing such a wonderful work. I am studying the code and found a problem about the relation calssification(may not).
The format of the relationship classification label you construct is as follows:
[ [0,1,0,0], [0,0,1,0] ... ]
If a data set contains a total of four relationships, namely [A,B,C,D], then the first line [0,1,0,0] indicates that there is a relationship B between the corresponding entity pairs, the second line [0,0,1,0] indicates that there is a relationship C between the corresponding entity pairs.
There seems to be no doubt about the above, but the first and second rows may be the same entity pair sample. At this time, the label should be in the following form：
[ [0,1,1,0], ... ]
Finally, in the above two cases, the relation calssification loss is different, and the first is wrong.
Hi,
thanks for your well described issue. I'm aware that this case is currently not handled correctly. It was not necessary for our paper, since the datasets we evaluated SpERT on did not contain multiple relations between the same entity pair. However, the model itself is build to handle such cases. Considering that our model (and code) is now also used on other datasets, I wanted to handle this case for some time, then postopened it due to other work and forgot about it. Thanks for reminding me, it is a good time to fix it now.
Should be fixed in commit 7b27b7d . Please leave a comment if the issue is still unsolved.