can't process big log : out of memory
lalop opened this issue · 6 comments
I launch my app with --prof and the result is a 1G file.
When I try to process it with nprof I get this error :
FATAL ERROR: CALL_AND_RETRY_0 Allocation failed - process out of memory
Try running nprof like this: node --max_old_space_size=4096 nprof <args>. Replace 4096 (that's 4 GB) with a value that works for you.
Thank for the trick no value work for me, I tried with 4 GB, 8 GB and more but I get the same error.
Hm, okay. I don't think there's much you can do in that case.
I have to say, that's a truly massive log file, maybe try slimming it down. As a quick hack, try head -100000 v8.log > v8-head.log.
Ok so I will make some smaller file.
Thank you
@bnoordhuis thanks for all your awesome work!
Is there any update on this? When I split my log with that (or other commands) only the 1st log works with node --prof-process --preprocess -j, all the other "later" logs don't process correctly - but those are the ones I need to look at.
I regularly have 1GB+ logs that I need to inspect only the tail end.
I'm guessing there is a simple fix I'm missing now in 2020? Any links/tips/pointers would be super appreciated.
Thank you!!!
@amark With modern versions of Node.js, the --max_old_space_size=... trick should work.
Old Node versions had a bug where values beyond 1.5G didn't really work but nowadays setting it to 4096 or 8192 (or even bigger) should Just Work.
If that doesn't help, can you open an issue over at nodejs/node and cc me? Cheers.