powturbo/TurboPFor-Integer-Compression

Benchmark: TurboPFor: IcApp 16 bits Integer Compression - Nanopore Signal Data

powturbo opened this issue · 0 comments

powturbo commented

Testing TurboPFor, I've made some experiments with nanopore signal data.
File: all 16-bits signal data extracted from multi_fast5_zip.fast5

         ./icapp sig.u16 -Elzturbo,39 -Fs -e83,77,11,39,51,30,76

file: max bits histogram:
04: 0.000% 
06: 0.000% 
07: 0.002% 
08:###### 6.0% 
09:######################################################################################### 89% 
10:##### 5.5% 
11: 0.000% 
12: 0.000% 
16: 0.001% 

file: delta max bits histogram:
00:## 2.4%
01:## 2.4%
02:##### 4.7%
03:######### 9.2%
04:################# 17%
05:########################### 27%
06:######################## 24%
07:######### 9.0%
08:#### 3.7%
09:# 0.8%
10: 0.012%
11: 0.002%
12: 0.000%
13: 0.000%
14: 0.000%

Filesize: 3.097.862 bytes    CPU: Skylake i7-6700  3.4GHz

  E MB/s     size     ratio   D MB/s function (integer size=16 bits) 
   94.48    1275037  41.16%  1740.37 Lztp4z Nibble    Transpose+zigzag+turboanx,64
   81.99    1279609  41.31%  1512.63 Lztpz Byte       Transpose+zzag+turboanx,64
    1.81    1283751  41.44%  2201.98 Lztp4z Nibble    Transpose+zigzag+lzturbo,39 
    1.13    1289797  41.64%  1912.47 Lztpz Byte       Transpose+zzag+lzturbo,39    	
    3.06    1292571  41.72%  1627.03 Lztpz Byte       Transpose+zzag+zstd,22    	
  120.73    1293228  41.75%  2010.29 lzv8zenc         TurboByte+zzag+turboanx    	
    7.32    1296883  41.86%  1946.86 lzv8zenc         TurboByte+zzag+lzturbo,39    	
    7.48    1310074  42.29%  1690.97 lzv8zenc         TurboByte+zzag+zstd,22 	
    6.44    1333780  43.05%  1103.62 vbz              vbz_compression 
  614.78    1432663  46.25%  4419.20 p4nzenc128v16    TurboPForV   zigzag  	
  547.91    1523046  49.16%   752.46 lzv8zenc         TurboByte+zzag+fse    	
    1.55    1577260  50.91%  5779.59 LztpzByte        Transpose+zzag+lzturbo,19    	
   11.68    1577173  50.91%  1789.64 LztpzByte        Transpose+zzag+lz4,12    	
   10.80    1583766  51.12%  6387.34 lzv8zenc         TurboByte+zzag+lzturbo,19    	
   21.06    1587076  51.23%  4863.21 lzv8zenc         TurboByte+zzag+lz4,12    	
  367.44    1632853  52.71%   573.68 Lztpz Byte        Transpose+zzag+fse    	
 6749.15    1676704  54.12%  8775.81 v8nzenc128v16    TByte+TPackV zigzag  	
 6992.92    1676746  54.13%  8927.56 bitnzpack128v16  TurboPackV   zigzag  	
   68.73    1705144  55.04%   959.98 lzv8enc          TurboByte+turboanx