nlpyang/PreSumm

processed data `bert_data_cnndm_final.zip` not consistent with the code

hedonihilist opened this issue · 1 comments

hedonihilist commented

After extracting bert_data_cnndm_final.zip, I got files named like cnndm.train.100.bert.pt, which is not recognized by the following code

PreSumm/src/models/data_loader.py

Line 84 in 70b810e

pts = sorted(glob.glob(args.bert_data_path + '.' + corpus_type + '.[0-9]*.pt'))

Changing the code like this can fix the issue:

pts = sorted(glob.glob(os.path.join(args.bert_data_path, 'cnndm.' + corpus_type + '.[0-9]*.pt'))
Deleted user commented

It misses a ). The fix is rather pts = sorted(glob.glob(os.path.join(args.bert_data_path, 'cnndm.' + corpus_type + '.[0-9]*.pt')))