BeyonderXX/InstructUIE

Zero-shot NER experiment settings

Spico197 opened this issue · 2 comments

Spico197 commented

Hi, there. Thanks for the excellent work!
Since the NER datasets in zero-shot settings are also included in the IE INSTRUCTIONS dataset, I'm wondering if there are special instructions for zero-shot experiments?
Do we have to remove all the seven NER datasets and train a new model to get fair results?

Thanks very much!

BeyonderXX commented

Hi!
For the zero-shot experiments, we removed all test-related datasets from the training set, while the instruction was the same as the supervised.

Xiao Wang

  • 👍1
Spico197 commented

Got it. Thank you very much for your quick response~