silverwind/rrdir

Maximum callstack exceeded

borestad opened this issue · 4 comments

borestad commented

I have a test folder containing 249351 files.
node v12.16.1

const rrdir = require('rrdir')

rrdir.async('.').then((res) => {
  console.log(res, res.length)
})


(node:71696) UnhandledPromiseRejectionWarning: RangeError: Maximum call stack size exceeded
    at /Users/johan/netent/netent-repos/node_modules/rrdir/index.js:134:26
    at async Promise.all (index 11)
    at async AsyncGeneratorFunction.module.exports.async (/Users/johan/test-repos/node_modules/rrdir/index.js:111:3)

Running the code like this works though.
node --stack-size=65500 async.js

silverwind commented

Can you try with the async iterator version? I assume this might be a limitation of .async, thought it's possible that variant also runs into that issue with deeply nested directories because each subdirectory does cause recursion.

silverwind commented

Also, it might be interesting at which nesting level this issue occurs. Node's --stack-size is limited by an amount of bytes and we can either push Node.js to raise their default (it may be a unoptimized v8 default) or recommend that flag in the README.

silverwind commented

Reading some more into this, the issue is probably caused by V8 not implementing tail call optimization. There might a a workaround possible via process.nextTick, but I'm not sure how to apply yet.

silverwind commented

Should definitely be solvable by using the async iterator.