wirehair: A C++ repository from dumpforjunk

	Wirehair FEC - Fast Forward Error Correction (FEC) Codec

	Wirehair FEC produces a stream of error correction blocks from a
data source.  When enough of these blocks are received, the original
data can be recovered.

	Wirehair is an FEC codec used to improve reliability of data sent
over a Binary Erasure Channel (BEC) such as satellite Internet or WiFi.
The data to transmit over the BEC is encoded into equal-sized blocks.
When enough blocks are received at the other end of the channel, then
the original data can be recovered.

	How many additional blocks are needed is random, though the number
of additional blocks is low and does not vary based on the size of the
data.  Typical overhead is between 0.015 and 0.03 additional blocks.
This may seem appalling to you, but think of it this way: The channel
is already lossy, so this is just like adding < 3% / N loss - a very
very small amount of additional loss (e.g. 0.003%) - to the channel,
down in the noise.

	Wirehair is released under the BSD license, which means that I ask
only that if you use my software that in the binary distribution of your
software you include the copyright notice in WIREHAIR.LICENSE and maybe
it would be nice to say thank you in an Email. :}

-Christopher A Taylor (chris@gameclosure.com) http://catid.org


News Update: 3/28/2012


TODO:

+ Fix any seeds that have >3% overhead by extending the exception list.
+ C and Objective-C ports.
+ Optimizations for very large files > 1GB.


Further improvements on the horizon:

+ SIMD memxor() and prefetch is being worked on by Nate Tinkler.
+ Multi-threading is being worked on by Brandon McKune.
+ Java port is being worked on by Steve Sheppard.
+ C# / R port


Network File Transfer Benchmarks:

Test software compiled in 64-bit Release mode in MSVC.

Message block size set at 1500 bytes.  Simulated packet loss rate is 50%.
Running on my laptop with a Core i7 @ 2.67 GHz:

>> OKAY! N=64(0.096 MB) encoder.BeginEncode in 445.777 usec, 215.355 MB/s
N=64 Decode in 426.823 usec, 224.918 MB/s.  Avg overhead = 0.016 (24 bytes)
>> OKAY! N=1064(1.596 MB) encoder.BeginEncode in 4301.47 usec, 371.036 MB/s
N=1064 Decode in 4697.76 usec, 339.736 MB/s.  Avg overhead = 0.017 (25.5 bytes)
>> OKAY! N=2064(3.096 MB) encoder.BeginEncode in 9464.09 usec, 327.131 MB/s
N=2064 Decode in 11165 usec, 277.296 MB/s.  Avg overhead = 0.016 (24 bytes)
>> OKAY! N=3064(4.596 MB) encoder.BeginEncode in 17301.4 usec, 265.644 MB/s
N=3064 Decode in 18475.5 usec, 248.762 MB/s.  Avg overhead = 0.027 (40.5 bytes)
>> OKAY! N=4064(6.096 MB) encoder.BeginEncode in 23452.5 usec, 259.929 MB/s
N=4064 Decode in 26123.7 usec, 233.352 MB/s.  Avg overhead = 0.02 (30 bytes)
>> OKAY! N=5064(7.596 MB) encoder.BeginEncode in 30726.6 usec, 247.212 MB/s
N=5064 Decode in 35306.1 usec, 215.147 MB/s.  Avg overhead = 0.015 (22.5 bytes)
>> OKAY! N=6064(9.096 MB) encoder.BeginEncode in 39403.9 usec, 230.84 MB/s
N=6064 Decode in 44974.7 usec, 202.247 MB/s.  Avg overhead = 0.029 (43.5 bytes)
>> OKAY! N=7064(10.596 MB) encoder.BeginEncode in 45753.7 usec, 231.588 MB/s
N=7064 Decode in 53796.4 usec, 196.965 MB/s.  Avg overhead = 0.022 (33 bytes)
>> OKAY! N=8064(12.096 MB) encoder.BeginEncode in 54760.5 usec, 220.889 MB/s
N=8064 Decode in 66865.6 usec, 180.9 MB/s.  Avg overhead = 0.02 (30 bytes)
>> OKAY! N=9064(13.596 MB) encoder.BeginEncode in 65359.4 usec, 208.019 MB/s
N=9064 Decode in 76243.7 usec, 178.323 MB/s.  Avg overhead = 0.022 (33 bytes)


2.0 GB Large File Benchmarks:

Message block size set at ~2 MB.  Simulated packet loss rate is 50%.
Running on a desktop with an Intel processor @ 3.07 GHz:

>> OKAY! N=1024(2000 MB) encoder.BeginEncode in 7.079e+006 usec, 282.539 MB/s
>> OKAY! N=1024 decoder.Decode in 9.18e+006 usec, 217.734 MB/s after 0 extra.
>> OKAY! N=1024 decoder.Decode in 8.14e+006 usec, 245.49 MB/s after 0 extra.
>> OKAY! N=1024 decoder.Decode in 8.03e+006 usec, 249.073 MB/s after 0 extra.
>> OKAY! N=1024 decoder.Decode in 7.75+006 usec, 258.032 MB/s after 0 extra.
>> OKAY! N=1024 decoder.Decode in 8.09e+006 usec, 247.205 MB/s after 0 extra.
>> OKAY! N=1024 decoder.Decode in 8.15e+006 usec, 245.223 MB/s after 0 extra.
>> OKAY! N=1024 decoder.Decode in 8.32e+006 usec, 240.273 MB/s after 0 extra.
>> OKAY! N=1024 decoder.Decode in 8.21e+006 usec, 243.544 MB/s after 0 extra.
>> OKAY! N=1024 decoder.Decode in 8.23e+006 usec, 243.093 MB/s after 0 extra.
>> OKAY! N=1024 decoder.Decode in 8.28e+006 usec, 241.404 MB/s after 0 extra.
dumpforjunk/wirehair
