- X-Reed-Solomon Erasure Code engine in pure Go.
- Fast and Efficient Data Reconstruction in Erasure-code
- Saving about 30% I/O in reconstruction
- Based on papers: <A “Hitchhiker’s” Guide to Fast and Efficient Data Reconstruction in Erasure-coded Data Centers> & <A Piggybacking Design Framework for Read-and Download-efficient Distributed Storage Codes>
To get the package use the standard:
go get github.com/templexxx/xrs
See the associated GoDoc
- All arch are supported
- Go1.11(for AVX512)
And we must know the benchmark test is quite different with encoding/decoding in practice.
Because in benchmark test loops, the CPU Cache will help a lot. In practice, we must reuse the memory to make the performance become as good as the benchmark test.
Example of performance on my MacBook Pro (Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz) DataCnt = 10; ParityCnt = 4
Vector size | AVX512 (MB/S) | AVX2 (MB/S) |
---|---|---|
4KB | -- | 8632 |
64KB | -- | 7978 |
1MB | -- | 5967 |