TooTallNate/node-time

Possible race condition when loading node-time and immediately using it

Opened this issue · 6 comments

My application uses node-time to look up timezone offsets. It uses node's cluster module to start 5 workers. When we're experiencing high throughput and try to bring up a new process, many times the workers die with errors related to being unable to load timezones:

Uncaught Exception: Error: Unknown Timezone: 'Europe/Moscow'

{ message: 'Unknown Timezone: \'Europe/Moscow\'',
 name: 'Error',
 stack: 
  [ 'Error: Unknown Timezone: \'Europe/Moscow\'',
    '    at Function.tzset (.../node_modules/time/index.js:139:15)',
    '    at Date.setTimezone (.../node_modules/time/index.js:241:18)',
    '    at Object.localTime (.../lib/geo_timezone.js:70:11)',
 ...
    '    at HTTPServer.Server.handle (.../node_modules/connect/lib/http.js:217:3)',
    '    at HTTPServer.EventEmitter.emit (events.js:98:17)',
    '    at HTTPParser.parser.onIncoming (http.js:2108:12)',
    '    at HTTPParser.parserOnHeadersComplete [as onHeadersComplete] (http.js:121:23)',
    '    at Socket.socket.ondata (http.js:1966:22)',
    '    at TCP.onread (net.js:525:27)' ] }

After a while, the load balancer backs off a bit and the process is able to start. It looks to me like most things in node-time are synchronous, but is there a way for me to ensure the timezones are fully loaded before I start listening for requests?

It appears that I have the same behavior here:

Error: Unknown Timezone: 'Europe/Madrid'
at Function.tzset (/home/ubuntu/retarget/node_modules/time/index.js:149:15)
at Date.setTimezone (/home/ubuntu/retarget/node_modules/time/index.js:251:18)
at Date (/home/ubuntu/retarget/node_modules/time/index.js:584:7)
at Object.exports.getTimezoneOffset (/home/ubuntu/retarget/src/lib/util/index.js:129:20)
at Object.exports.getInitialHourForTimezone (/home/ubuntu/retarget/src/lib/util/index.js:147:31)

I'm requiring node-time in an util library were we use it. I've added the require to my app.js to try to leave enough time to node-time in order to initialize what it needs to work.

EDIT: No luck , errors continue appearing. :(

I experienced the same error.. the problem was that the module was looking for timezones in /usr/share/zoneinfo/ while I had incorrect tzdata package installed, which resulted in missing the timezone in the mentioned directory.

I eventually discovered that https://github.com/gsmcwhirter/node-zoneinfo was a better fit for our use case.

Same error here. Seems to be intermittent.

It's definitely a race condition of some sort. However, I've been unable to reproduce it outside of our production app in a simple script. We've added try catch logic around the calls to setTimezone and will be moving to moment and moment-timezone.

It looks like this is not a race condition between cluster processes. I was unable to reproduce a race condition failure even with spinning up 8 processes and switching timezones on all 8 processes for 10000 iterations.

I was however able to reproduce this error by hitting the limit on the number of files open. Here's a gist that reproduces the issue consistently. Your mileage may vary on the number of files you can have open on a single process and the processes' user. One solution here is to up the number of files the process and process user can have open. https://gist.github.com/Zambonilli/f03f6d5a723a4b403b959f1348809933