whatyouhide/xandra

Figure out how to test compression

Closed this issue · 0 comments

Erlang/Elixir have no okay libraries for either the Snappy or lz4 compression algorithms.

We used to use snappy-erlang-nif but it doesn't compile easily anymore.

Also, doesn't support Snappy anymore (since native protocol v4 or v5, can't remember). It only supports lz4 for now.

This issue requires a few steps:

  1. Figure out if C*'s lz4 compression needs a given size when decompressing. The native protocol provides the size of the frames to decompress, but I’m not sure if they also prepend the size with lz4 or not.
  2. c:Xandra.Compressor.decompress/1 doesn't take a size, so we'll need to either deprecated it or leave it, but add c:Xandra.Compressor.decompress/2 maybe.

I have locally a hacked-together lz4 library (that uses Rust NIFs) that we can use to test. If someone picks this issue up, contact me and I'll hook you up with the library 😄