google-research-datasets/WebRED

Question about P0 (no_relation)

zhangdongxu opened this issue · 1 comments

Hi,

Thanks for this great dataset!

Just to double-check with you, an instance (sentence + subj + obj) will be considered as P0(no_relation) if num_pos_raters == 0. Is this correct? Or does it only mean this label is not considered as True but there may still exist other relation types in this
instance?

And a dual question, if the num_pos_raters / num_raters == 1, can we feel confident to say that this instance only indicates this relation type but not other relation types? (I guess the assumption then is relation types are exclusive)

Thanks much,
Dongxu

I'm closing this issue after a more careful reading of the paper.