soheilpro/catj

"JavaScript heap out of memory" exception on large json file

jackharrhy opened this issue · 1 comments

to reproduce:

(warning: 180mb json file)

curl "https://github.com/zemirco/sf-city-lots-json/blob/master/citylots.json" > citylots.json
cat citylots.json | catj # <- out of mem.
catj citylots.json # <- also out of mem.

Since it loads the whole JSON file in memory, you have to have enough RAM available for it to run. On my machine with 16 GB of RAM it works fine.

If there's enough demand, I'll investigate rewriting it using a streaming JSON parser to lower the memory footprint.