Excessive memory consumption when using ZSTD to receive compressed data
Opened this issue · 2 comments
djluck commented
Describe the bug
We've seen excessive memory consumption that leads to OOMs when receiving ZSTD-compressed data.
Steps to reproduce
- Enable zstd compression for the HTTP exporter in an upstream collector
- Observe OOMs in your receiving collector!
You can observe excessive memory via pprof:
What version did you use?
v0.114.0
atoulme commented
Might this be something to report upstream to the zstd lib? https://github.com/klauspost/compress
Please attach the full pprof dump?
djluck commented
Unfortunately I can't seem to locate a copy of the original trace dump. Due to time commitments I'll be unlikely to recreate this issue, sorry that the above isn't as much to go on as would be ideal but hopefully others can provide dumps when/ if they run into this issue.