需要补充以下数据文件:(tiangong暂时不需补充)
- ./BERT/BERTChinese/
- ./BERT/BERTModel/
- ./SCL/data/aol/train.pos.txt
- ./SCL/data/aol/dev.pos.txt
- ./SCL/data/tiangong/train.pos.txt
- ./SCL/data/tiangong/dev.pos.txt
- ./Ranking/data/aol/train_line.txt
- ./Ranking/data/aol/test_line.middle.txt
- ./Ranking/data/aol/test_line.txt
- ./Ranking/data/tiangong/train.point.txt
- ./Ranking/data/tiangong/dev.point.txt
- ./Ranking/data/tiangong/test.point.lastq.txt
- ./Ranking/data/tiangong/test.point.preq.txt
运行SCL和Ranking脚本:需要修改source命令后的环境参数
Ranking脚本中,pretrain_model_path需要改成SCL预训练相应的路径文件名
requirements:
- Python 3.8.0
- Pytorch 1.10.2 (with GPU support)
- pytrec-eval 0.5
- Transformers 4.2.0
- setproctitle 1.3.1
运行:
bash Ranking.sh
或
bash SCL.sh