The comparison with previous SOTA seems unfair
Closed this issue · 1 comments
From the original paper, the NYT and WebNLG used by CasRel (HBT) separately contain 56195 + 5000 + 5000 sentences and 5019 + 500 + 703 sentences，which are inconsistent with the ones (69710, 31511) mentioned in your paper. It is unfair to compare with the results copied from the paper when using different datasets because the improvements may come from more training data. So, I recommend that reproduce HBT results on your datasets or use the datasets from HBT, and then give a more persuasive result.
When we published this paper, HBT was not open sourced yet, so we did not know their dataset split and used Opennre's data split and original WebNLG dataset. If your insterested in this question, you can try yourself.