THUDM/paper-source-trace

Can't run function 'eval_test_papers_bert' from bert.py

moonlightnoodles opened this issue · 4 comments

moonlightnoodles commented

problem
can't run function 'eval_test_papers_bert' from bert.py to calculate bert average map due to './paper-source-trace/data/2023/paper_source_trace_test.json' is not given.

zfjsail commented

Hi, eval_test_papers_bert is used for the evaluation of an older dataset. Use gen_kddcup_valid_submission_bert function instead.

moonlightnoodles commented

Hi, eval_test_papers_bert is used for the evaluation of an older dataset. Use gen_kddcup_valid_submission_bert function instead.

thank you for your reply! but it seems that eval_test_papers_bert doesn't calculate MAP value to evaluate this method.
49ba2c45f0dc118c275bd718d3adc16

zfjsail commented

The outputs generated by gen_kddcup_valid_submission_bert can be submitted to https://www.biendata.xyz/competition/pst_kdd_2024/make-submission/ to obtain the MAP score on the validation set. Currently, since the competition is ongoing, the ground truths of validation set and test set are not provided.

moonlightnoodles commented

The outputs generated by gen_kddcup_valid_submission_bert can be submitted to https://www.biendata.xyz/competition/pst_kdd_2024/make-submission/ to obtain the MAP score on the validation set. Currently, since the competition is ongoing, the ground truths of validation set and test set are not provided.

作者你好，虽然比赛已经结束，但我仍然希望在这个任务上进行一些工作，所以能否请您提供实现MAP计算的代码