zjunlp/Relphormer

About entity one-hop generate and fair comparision

fuyansheng opened this issue · 5 comments

In the create_neighbor.py, the paper generated the graph structure sequence using triples from the train, valid, and test datasets. I worried this might cause test dataset leakage in the training process. Is it more reasonable to only use the training dataset in the training process?

And another question: in your comparison experiment, you cited the HittER model. However, when I read the codes, the HittER model only used the graph structure nodes from the training dataset. So, is the comparison fair with the HittER model in your experiment? Not only HittER, in another work's code(IMF: Interactive Multimodal Fusion Model for Link Prediction) using graph structure for KG, also only uses the training dataset.

Looking forward to your reply. Thx!

Hi, thank you very much for your attention! Regarding the content of create_neighbor.py, it is a mistake made during the process of organizing the code. We only use the graph structure nodes from the training dataset and we have corrected the run file accordingly.

Thanks for your reply! Will this influence the experiment result in your paper?

Hi, it does not affect the experimental results presented in the paper. The mistake occurred when we organized the code files in the Github.

zxlzr commented

Hi buddy, do you have any further questions?

Thanks a lot! I don't have questions now!