nohup_multi_ner.out: list index out of range
mjeensung opened this issue · 2 comments
mjeensung commented
An input for the reproduction:
The invention claimed is*., 2 ,.**., 1 ,.* *., 1 ,.1. A fusion protein selected from the group consisting of SEQ ID NO. 148, SEQ ID NO. 150, SEQ ID NO. 194, SEQ ID NO. 196, SEQ ID NO. 198, SEQ ID NO. 200, SEQ ID NO. 202 and SEQ ID NO. 204.., 1 ,.* *., 1 ,.2. A fusion protein of claim 1 wherein the signal peptide has been removed.., 1 ,.* *., 1 ,.3. The fusion protein of SEQ ID NO. 148 wherein the signal peptide has been removed, SEQ ID NO. 177 ., 3 ,.QIQKAEQN DVKLAPPTDV RSGYIRLVKN VNYYIDSESI WVDNQEPQIV HFDAVVNLDK GLYVYPEPKR YARSVRQYKI LNCANYHLTQ VRTDFYDEFW GQGLRAAPKK QKKHTLSLTP DTTLYNAAQI ICANYGEAFS VDKKGGTKKA AVSELLQASA PYKADVELCV YSTNETTNCT GGKNGIAADI TTAKGYVKSV TTSNGAITVK GDGTLANMEY ILQATGNAAT GVTWTTTCKG TDASLFPANF CGSVTQ., 4 ,..., 1 ,.* *., 1 ,.4. The fusion protein of SEQ ID NO. 194 wherein the signal peptide has been removed, SEQ ID NO. 219 ., 3 ,.IQKAEQND VKLAPPTDVR SGYIRLVKNV NYYIDSESIW VDNQEPQIVH FDAVVNLDKG LYVYPEPKRY ARSVRQYKIL NCANYHLTQV RTDFYDEFWG QGLRAAPKKQ KKHTLSLTPD TTLYNAAQII CANYGEAFSV DKKGGTKKAA VSELLQASAP YKADVELCVY STNETTNCTG GKNGIAADIT TAKGYVKSVT TSNGAITVKG DGTLANMEYI LQATGNAATG VTWTTTCKGT DASLFPANFC GSVTQ., 4 ,..., 1 ,.* *., 1 ,.5. An immunogenic composition comprising the fusion protein of SEQ ID NO. 177 ., 3 ,.QIQKAEQN DVKLAPPTDV RSGYIRLVKN VNYYIDSESI WVDNQEPQIV HFDAVVNLDK GLYVYPEPKR YARSVRQYKI LNCANYHLTQ VRTDFYDEFW GQGLRAAPKK QKKHTLSLTP DTTLYNAAQI ICANYGEAFS VDKKGGTKKA AVSELLQASA PYKADVELCV YSTNETTNCT GGKNGIAADI TTAKGYVKSV TTSNGAITVK GDGTLANMEY ILQATGNAAT GVTWTTTCKG TDASLFPANF CGSVTQ., 4 ,..., 1 ,.* *., 1 ,.6. An immunogenic composition comprising the fusion protein of SEQ ID NO.219 ., 3 ,.IQKAEQND VKLAPPTDVR SGYIRLVKNV NYYIDSESIW VDNQEPQIVH FDAVVNLDKG LYVYPEPKRY ARSVRQYKIL NCANYHLTQV RTDFYDEFWG QGLRAAPKKQ KKHTLSLTPD TTLYNAAQII CANYGEAFSV DKKGGTKKAA VSELLQASAP YKADVELCVY STNETTNCTG GKNGIAADIT TAKGYVKSVT TSNGAITVKG DGTLANMEYI LQATGNAATG VTWTTTCKGT DASLFPANFC GSVTQ., 4 ,..., 1 ,.* ., 1 ,.
The error message in logs/nohup_multi_ner.out:
Found an error: list index out of range
minstar commented
This error was generated because of the NER detokenization issue given a sentence length is larger than the max sequence length (e.g., 128 in this case).
We use the truncated version in NER preprocessing (lines 302-306 in multi_ner/main.py) which could generate the following issue.
Thus, we resolve this issue by changing the code to the sliding window in preprocessing part (lines 308-414 in multi_ner/main.py) and postprocessing part (lines 236-238 in multi_ner/ops.py).
If there are any other problems please let me (@minstar) know and reopen this issue!