/FileContainer

Primary LanguageC#MIT LicenseMIT

FileContainer

Key/value file container. File organized with fixed-size pages.

Run benchmark

Run Benchmark project - .net 6 targeted

Speed tests

Method BatchSize PageSize Flags Compress Avg MB/sec Lost%/Compress Gen0 Gen1 Gen2 Allocated
Single 1 256 0 None 1261.77 2.1 1000.0000 1000.0000 1000.0000 83.37 MB
Single 1 256 0 GZip 27.53 x1.8 - - - 53.03 MB
Single 1 256 0 LZ4 326.30 x1.4 1000.0000 1000.0000 1000.0000 64.34 MB
Single 1 256 WriteDirImmediately None 422.18 2.1 4000.0000 1000.0000 1000.0000 106.27 MB
Single 1 256 WriteDirImmediately GZip 20.38 x1.8 3000.0000 - - 73.14 MB
Single 1 256 WriteDirImmediately LZ4 165.95 x1.4 4000.0000 1000.0000 1000.0000 92.1 MB
Single 5 256 0 None 1234.12 2.1 1000.0000 1000.0000 1000.0000 83.41 MB
Single 5 256 0 GZip 27.58 x1.8 - - - 53.08 MB
Single 5 256 0 LZ4 316.88 x1.4 1000.0000 1000.0000 1000.0000 64.38 MB
Single 5 256 WriteDirImmediately None 886.74 2.1 1000.0000 1000.0000 1000.0000 88.09 MB
Single 5 256 WriteDirImmediately GZip 24.57 x1.8 1000.0000 - - 57.08 MB
Single 5 256 WriteDirImmediately LZ4 248.26 x1.4 2000.0000 1000.0000 1000.0000 69.9 MB
Single 10 256 0 None 1142.91 2.1 1000.0000 1000.0000 1000.0000 83.4 MB
Single 10 256 0 GZip 26.81 x1.8 - - - 53.07 MB
Single 10 256 0 LZ4 324.40 x1.4 1000.0000 1000.0000 1000.0000 64.36 MB
Single 10 256 WriteDirImmediately None 997.63 2.1 1000.0000 1000.0000 1000.0000 85.62 MB
Single 10 256 WriteDirImmediately GZip 26.56 x1.8 - - - 55.09 MB
Single 10 256 WriteDirImmediately LZ4 275.84 x1.4 1000.0000 1000.0000 1000.0000 67.16 MB
Single 25 256 0 None 1239.62 2.1 1000.0000 1000.0000 1000.0000 83.39 MB
Single 25 256 0 GZip 23.49 x1.8 - - - 53.06 MB
Single 25 256 0 LZ4 317.24 x1.4 1000.0000 1000.0000 1000.0000 64.35 MB
Single 25 256 WriteDirImmediately None 1149.97 2.1 1000.0000 1000.0000 1000.0000 84.19 MB
Single 25 256 WriteDirImmediately GZip 26.28 x1.8 - - - 53.84 MB
Single 25 256 WriteDirImmediately LZ4 309.36 x1.4 1000.0000 1000.0000 1000.0000 65.43 MB
Single 1 512 0 None 1431.55 1.5 1000.0000 1000.0000 1000.0000 83.29 MB
Single 1 512 0 GZip 27.65 x1.8 - - - 53.05 MB
Single 1 512 0 LZ4 336.82 x1.4 1000.0000 1000.0000 1000.0000 64.33 MB
Single 1 512 WriteDirImmediately None 456.74 1.5 4000.0000 1000.0000 1000.0000 104.85 MB
Single 1 512 WriteDirImmediately GZip 20.51 x1.8 2000.0000 - - 72.34 MB
Single 1 512 WriteDirImmediately LZ4 184.93 x1.4 4000.0000 1000.0000 1000.0000 89.89 MB
Single 5 512 0 None 1392.79 1.5 1000.0000 1000.0000 1000.0000 83.33 MB
Single 5 512 0 GZip 27.39 x1.8 - - - 53.1 MB
Single 5 512 0 LZ4 330.34 x1.4 1000.0000 1000.0000 1000.0000 64.37 MB
Single 5 512 WriteDirImmediately None 997.20 1.5 1000.0000 1000.0000 1000.0000 87.56 MB
Single 5 512 WriteDirImmediately GZip 25.57 x1.8 1000.0000 - - 56.93 MB
Single 5 512 WriteDirImmediately LZ4 264.87 x1.4 2000.0000 1000.0000 1000.0000 69.44 MB
Single 10 512 0 None 1413.92 1.5 1000.0000 1000.0000 1000.0000 83.31 MB
Single 10 512 0 GZip 27.47 x1.8 - - - 53.09 MB
Single 10 512 0 LZ4 338.04 x1.4 1000.0000 1000.0000 1000.0000 64.35 MB
Single 10 512 WriteDirImmediately None 1153.15 1.5 1000.0000 1000.0000 1000.0000 85.42 MB
Single 10 512 WriteDirImmediately GZip 26.32 x1.8 - - - 55.03 MB
Single 10 512 WriteDirImmediately LZ4 298.73 x1.4 1000.0000 1000.0000 1000.0000 66.92 MB
Single 25 512 0 None 1424.77 1.5 1000.0000 1000.0000 1000.0000 83.3 MB
Single 25 512 0 GZip 27.44 x1.8 - - - 53.08 MB
Single 25 512 0 LZ4 327.85 x1.4 1000.0000 1000.0000 1000.0000 64.34 MB
Single 25 512 WriteDirImmediately None 1302.05 1.5 1000.0000 1000.0000 1000.0000 84.07 MB
Single 25 512 WriteDirImmediately GZip 27.00 x1.8 - - - 53.83 MB
Single 25 512 WriteDirImmediately LZ4 318.73 x1.4 1000.0000 1000.0000 1000.0000 65.34 MB
Single 1 2048 0 None 1651.31 2.8 1000.0000 1000.0000 1000.0000 83.74 MB
Single 1 2048 0 GZip 28.46 x1.7 - - - 53.63 MB
Single 1 2048 0 LZ4 350.85 x1.4 1000.0000 1000.0000 1000.0000 64.89 MB
Single 1 2048 WriteDirImmediately None 475.31 2.8 4000.0000 1000.0000 1000.0000 105.65 MB
Single 1 2048 WriteDirImmediately GZip 21.33 x1.7 3000.0000 - - 73.38 MB
Single 1 2048 WriteDirImmediately LZ4 186.49 x1.4 4000.0000 1000.0000 1000.0000 89.95 MB
Single 5 2048 0 None 1547.96 2.8 1000.0000 1000.0000 1000.0000 83.78 MB
Single 5 2048 0 GZip 28.01 x1.7 - - - 53.69 MB
Single 5 2048 0 LZ4 349.93 x1.4 1000.0000 1000.0000 1000.0000 64.93 MB
Single 5 2048 WriteDirImmediately None 1038.62 2.8 1000.0000 1000.0000 1000.0000 88.12 MB
Single 5 2048 WriteDirImmediately GZip 26.59 x1.7 1000.0000 - - 57.61 MB
Single 5 2048 WriteDirImmediately LZ4 299.11 x1.4 2000.0000 1000.0000 1000.0000 69.9 MB
Single 10 2048 0 None 1595.03 2.8 1000.0000 1000.0000 1000.0000 83.77 MB
Single 10 2048 0 GZip 28.20 x1.7 - - - 53.68 MB
Single 10 2048 0 LZ4 341.46 x1.4 1000.0000 1000.0000 1000.0000 64.92 MB
Single 10 2048 WriteDirImmediately None 1267.33 2.8 1000.0000 1000.0000 1000.0000 85.96 MB
Single 10 2048 WriteDirImmediately GZip 27.20 x1.7 - - - 55.66 MB
Single 10 2048 WriteDirImmediately LZ4 307.71 x1.4 1000.0000 1000.0000 1000.0000 67.43 MB
Single 25 2048 0 None 1635.09 2.8 1000.0000 1000.0000 1000.0000 83.76 MB
Single 25 2048 0 GZip 28.16 x1.7 - - - 53.67 MB
Single 25 2048 0 LZ4 354.46 x1.4 1000.0000 1000.0000 1000.0000 64.91 MB
Single 25 2048 WriteDirImmediately None 1413.27 2.8 1000.0000 1000.0000 1000.0000 84.6 MB
Single 25 2048 WriteDirImmediately GZip 28.06 x1.7 - - - 54.43 MB
Single 25 2048 WriteDirImmediately LZ4 327.84 x1.4 1000.0000 1000.0000 1000.0000 65.88 MB
Single 1 8192 0 None 1790.71 11.8 1000.0000 1000.0000 1000.0000 86.32 MB
Single 1 8192 0 GZip 34.55 x1.4 - - - 56.23 MB
Single 1 8192 0 LZ4 383.06 x1.2 1000.0000 1000.0000 1000.0000 67.48 MB
Single 1 8192 WriteDirImmediately None 544.50 11.8 5000.0000 1000.0000 1000.0000 113.19 MB
Single 1 8192 WriteDirImmediately GZip 25.50 x1.4 4000.0000 - - 80.99 MB
Single 1 8192 WriteDirImmediately LZ4 220.52 x1.2 5000.0000 1000.0000 1000.0000 97.34 MB
Single 5 8192 0 None 1310.99 11.8 1000.0000 1000.0000 1000.0000 86.36 MB
Single 5 8192 0 GZip 34.50 x1.4 - - - 56.29 MB
Single 5 8192 0 LZ4 389.52 x1.2 1000.0000 1000.0000 1000.0000 67.52 MB
Single 5 8192 WriteDirImmediately None 1225.40 11.8 2000.0000 1000.0000 1000.0000 91.68 MB
Single 5 8192 WriteDirImmediately GZip 32.08 x1.4 1000.0000 - - 61.2 MB
Single 5 8192 WriteDirImmediately LZ4 305.83 x1.2 2000.0000 1000.0000 1000.0000 73.45 MB
Single 10 8192 0 None 1682.20 11.8 1000.0000 1000.0000 1000.0000 86.34 MB
Single 10 8192 0 GZip 34.42 x1.4 - - - 56.28 MB
Single 10 8192 0 LZ4 390.39 x1.2 1000.0000 1000.0000 1000.0000 67.51 MB
Single 10 8192 WriteDirImmediately None 1214.28 11.8 2000.0000 1000.0000 1000.0000 89.03 MB
Single 10 8192 WriteDirImmediately GZip 32.89 x1.4 1000.0000 - - 58.75 MB
Single 10 8192 WriteDirImmediately LZ4 345.94 x1.2 2000.0000 1000.0000 1000.0000 70.5 MB
Single 25 8192 0 None 1755.22 11.8 1000.0000 1000.0000 1000.0000 86.33 MB
Single 25 8192 0 GZip 34.32 x1.4 - - - 56.27 MB
Single 25 8192 0 LZ4 389.40 x1.2 1000.0000 1000.0000 1000.0000 67.5 MB
Single 25 8192 WriteDirImmediately None 1605.03 11.8 1000.0000 1000.0000 1000.0000 87.37 MB
Single 25 8192 WriteDirImmediately GZip 33.74 x1.4 1000.0000 - - 57.22 MB
Single 25 8192 WriteDirImmediately LZ4 373.21 x1.2 2000.0000 1000.0000 1000.0000 68.65 MB
Single 1 32768 0 None 2251.95 39.5 3000.0000 1000.0000 1000.0000 96.76 MB
Single 1 32768 0 GZip 59.17 x0.8 3000.0000 2000.0000 1000.0000 98.68 MB
Single 1 32768 0 LZ4 561.09 x0.7 3000.0000 2000.0000 1000.0000 109.93 MB
Single 1 32768 WriteDirImmediately None 775.53 39.5 8000.0000 1000.0000 1000.0000 144.39 MB
Single 1 32768 WriteDirImmediately GZip 44.40 x0.8 8000.0000 1000.0000 1000.0000 144.2 MB
Single 1 32768 WriteDirImmediately LZ4 395.66 x0.7 9000.0000 1000.0000 1000.0000 160.49 MB
Single 5 32768 0 None 2250.52 39.5 3000.0000 1000.0000 1000.0000 96.8 MB
Single 5 32768 0 GZip 59.08 x0.8 3000.0000 1000.0000 1000.0000 98.74 MB
Single 5 32768 0 LZ4 545.82 x0.7 3000.0000 2000.0000 1000.0000 109.97 MB
Single 5 32768 WriteDirImmediately None 1609.78 39.5 4000.0000 1000.0000 1000.0000 106.24 MB
Single 5 32768 WriteDirImmediately GZip 56.06 x0.8 4000.0000 1000.0000 1000.0000 107.76 MB
Single 5 32768 WriteDirImmediately LZ4 473.64 x0.7 4000.0000 1000.0000 1000.0000 120 MB
Single 10 32768 0 None 2122.37 39.5 3000.0000 1000.0000 1000.0000 96.79 MB
Single 10 32768 0 GZip 58.90 x0.8 3000.0000 1000.0000 1000.0000 98.73 MB
Single 10 32768 0 LZ4 565.35 x0.7 3000.0000 2000.0000 1000.0000 109.96 MB
Single 10 32768 WriteDirImmediately None 1943.12 39.5 3000.0000 1000.0000 1000.0000 101.53 MB
Single 10 32768 WriteDirImmediately GZip 58.14 x0.8 3000.0000 1000.0000 1000.0000 103.26 MB
Single 10 32768 WriteDirImmediately LZ4 502.83 x0.7 3000.0000 1000.0000 1000.0000 115 MB
Single 25 32768 0 None 2111.64 39.5 3000.0000 1000.0000 1000.0000 96.78 MB
Single 25 32768 0 GZip 58.91 x0.8 3000.0000 2000.0000 1000.0000 98.72 MB
Single 25 32768 0 LZ4 535.42 x0.7 3000.0000 2000.0000 1000.0000 109.95 MB
Single 25 32768 WriteDirImmediately None 2080.35 39.5 3000.0000 1000.0000 1000.0000 98.61 MB
Single 25 32768 WriteDirImmediately GZip 57.09 x0.8 3000.0000 1000.0000 1000.0000 100.47 MB
Single 25 32768 WriteDirImmediately LZ4 514.10 x0.7 3000.0000 1000.0000 1000.0000 111.89 MB