klamkiew opened this issue 4 years ago · 0 comments
When huge amounts of data (tested: PB2 of IAV, >60.000 sequences) are given as input, pickle complains about the size. Chunking the dataset seems to be the best idea here.