[QST] CPU compression for decompression with high-level interface

Question

[QST] CPU compression for decompression with high-level interface

technillogue opened this issue a year ago · 9 comments

How can I generate the metadata needed for decompression?

Answer 1 · 2023-07-19T18:01:27.000Z

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

Answer 2 · 2023-08-22T16:21:50.000Z

Any update on this? I have a bunch of old data that needs to be compressed and it would be unfortunate to spin up GPUs only for compression

Answer 3 · 2023-08-22T18:53:19.000Z

Hi @technillogue, we're looking at producing a binary that would do this for you (i.e. produce an HLIF buffer using the CPU). Which formats are you interested in?

If you have a significant amount of old data, why not do this on GPU?

Answer 4 · 2023-08-22T18:58:35.000Z

Mostly Gdeflate, though Zstd would be good too. Is the HLIF format proprietary? I have some terabytes of data that sometimes needs to be loaded very quickly. Compression is cost-sensitive, not time-sensitive.

…

On Tue, Aug 22, 2023, 8:53 PM eschmidt-nvidia ***@***.***> wrote: Hi @technillogue <https://github.com/technillogue>, we're looking at producing a binary that would do this for you (i.e. produce an HLIF buffer using the CPU). Which formats are you interested in? If you have a significant amount of old data, why not do this on GPU? — Reply to this email directly, view it on GitHub <#86 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHG4G2LAEAW5XTBPXQJNNDXWT52XANCNFSM6AAAAAAZMHPKC4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

Answer 5 · 2023-08-22T21:13:56.000Z

I see. If I understand correctly, based on my experiments GPU and CPU will provide similar throughput/$. I've tested this on H100 compared to Genoa / SPR CPUs.

Given that you're not time sensitive, have you investigated the GDeflate high-compression mode? This could provide cost savings if you're storing the data for a long time.

We're looking at adding a similar mode to ZSTD.

The format isn't proprietary but we haven't had time to produce a public document that fully describes it.

Answer 6 · 2023-08-23T11:18:36.000Z

Interesting, I'm not sure what prices you have access to that make that work out but I can give it a shot and do compression a little more efficiently

As of June, GDeflate high-compression mode was broken for HLIF #81 (comment). I'm mostly interested in compressing model finetunes, not datasets, so almost all gains are from entropy coding and not dictionary compression. When trying the LLIF benchmark entropy-only had about the same compression ratio as high compression. I imagine there's ways to tune specifically entropy coding to be higher compression though.

Answer 7 · 2023-08-23T21:54:00.000Z

I responded to the earlier issue. This should be fixed.

Interesting regarding entropy-only being better. Have you tried our ANS, bitcomp, or Cascaded formats?

Answer 8 · 2023-09-22T22:01:19.000Z

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

Answer 9 · 2023-12-21T23:01:17.000Z

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.