GMOD/bgzf-filehandle

Improve unzip performance

cmdcolin opened this issue · 3 comments

  1. use inflateRaw instead of inflate...skip crc32 checks which are computationally expensive
  2. use Uint8Array instead of Buffer or otherwise make the unzip more memory efficient somehow? it is basically doing "Major GC" the entire time, see screenshot. see igvutils unbgzip(MIT licensed) https://github.com/igvteam/igv-utils/blob/master/src/bgzf.js#L30

Screenshot from 2022-03-15 10-33-41

note that the unzip performance causes our jb2profiling test on high coverage short reads to timeout

(orange blocks are major gc)

Fixed by #57, might be some small additional improvements if raw is considered and the final Buffer.from(uint8array) in the unzips appears to do a copy