JavaScript heap out of memory
Opened this issue · 2 comments
braco commented
Anyone else encounter this?
<--- Last few GCs --->
[35779:0x103001800] 204288 ms: Mark-sweep 1372.5 (1449.5) -> 1372.5 (1449.5) MB, 1296.8 / 0.0 ms allocation failure GC in old space requested
[35779:0x103001800] 205584 ms: Mark-sweep 1372.5 (1449.5) -> 1372.5 (1418.0) MB, 1296.0 / 0.0 ms last resort GC in old space requested
[35779:0x103001800] 206876 ms: Mark-sweep 1372.5 (1418.0) -> 1372.5 (1418.0) MB, 1292.4 / 0.0 ms last resort GC in old space requested
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0x2223aa7255e9 <JSObject>
1: f [/private/var/www/data/node_modules/unbzip2-stream/index.js:~25] [pc=0x3638d944dae9](this=0x2223759866f1 <JSGlobal Object>,b=97)
2: /* anonymous */ [/private/var/www/data/node_modules/unbzip2-stream/lib/bzip2.js:~163] [pc=0x3638d945853e](this=0x222364102249 <Object map = 0x222302a76661>,bits=0x2223b85b5889 <JSFunction f (sfi = 0x2223d86f2a51)>,stream=0x2223fe782441 <JSFunction f (sf...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [/Users/c/.nvm/versions/node/v9.11.1/bin/node]
2: node::FatalTryCatch::~FatalTryCatch() [/Users/c/.nvm/versions/node/v9.11.1/bin/node]
3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/Users/c/.nvm/versions/node/v9.11.1/bin/node]
4: v8::internal::Factory::NewUninitializedFixedArray(int) [/Users/c/.nvm/versions/node/v9.11.1/bin/node]
5: v8::internal::(anonymous namespace)::ElementsAccessorBase<v8::internal::(anonymous namespace)::FastPackedSmiElementsAccessor, v8::internal::(anonymous namespace)::ElementsKindTraits<(v8::internal::ElementsKind)0> >::GrowCapacity(v8::internal::Handle<v8::internal::JSObject>, unsigned int) [/Users/c/.nvm/versions/node/v9.11.1/bin/node]
6: v8::internal::Runtime_GrowArrayElements(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/c/.nvm/versions/node/v9.11.1/bin/node]
7: 0x3638d92842fd
Abort trap: 6
yanosh-igor commented
I have a similar issue. Looks like this module doesn't properly implement backpressure.
regular commented
Yes, lack of implementing back pressure could very well be the reason. If you write fast to the stream and consume the decompressed data slowly, decompressAndQueue will quickly build up a buffer queue of decompressed data. There probably should be a high watermark that causes the stream to pause. I don't actually use this module, so if one of you feels inclined to implement and test this, I'll certainly be willing to merge a pull request.