Errors when starting up Polyglot before servers are created
Closed this issue · 10 comments
This has been mentioned previously, but want to record it here to be fixed at some point. I see these error when starting polyglot when it gets a request for a server that it has not yet started up.
ERROR [03-13-2016 08:59:49] tornado.application: Uncaught exception GET /api/server/3wb9F (192.168.1.61)
HTTPServerRequest(protocol='http', host='192.168.1.77:8080', method='GET', uri='/api/server/3wb9F', version='HTTP/1.1', remote_ip='192.168.1.61', headers={'Accept-Language': 'en-US,en;q=0.5', 'Accept-Encoding': 'gzip, deflate', 'Host': '192.168.1.77:8080', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0', 'Connection': 'keep-alive', 'Referer': 'http://192.168.1.77:8080/static/index.html', 'Authorization': 'Basic YWRtaW46YWRtaW4='})
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado/web.py", line 1443, in _execute
result = method(_self.path_args, *_self.path_kwargs)
File "polyglot/element_manager/api.py", line 173, in get
self.send_json(PGLOT.nodeservers.servers[base_url].definition)
KeyError: u'3wb9F'
Hi jimboca,
Thanks so very much for the trace. Quite helpful!
With kind regards,
Michel Kohanim
CEO
(p) 818.631.0333
(f) 818.436.0702
http://www.universal-devices.comhttp://www.universal-devices.com/
From: jimboca [mailto:notifications@github.com]
Sent: Sunday, March 13, 2016 9:07 AM
To: UniversalDevicesInc/Polyglot Polyglot@noreply.github.com
Subject: [Polyglot] Errors when starting up Polyglot before servers are created (#7)
This has been mentioned previously, but want to record it here to be fixed at some point. I see these error when starting polyglot when it gets a request for a server that it has not yet started up.
ERROR [03-13-2016 08:59:49] tornado.application: Uncaught exception GET /api/server/3wb9F (192.168.1.61)
HTTPServerRequest(protocol='http', host='192.168.1.77:8080', method='GET', uri='/api/server/3wb9F', version='HTTP/1.1', remote_ip='192.168.1.61', headers={'Accept-Language': 'en-US,en;q=0.5', 'Accept-Encoding': 'gzip, deflate', 'Host': '192.168.1.77:8080', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0', 'Connection': 'keep-alive', 'Referer': 'http://192.168.1.77:8080/static/index.html', 'Authorization': 'Basic YWRtaW46YWRtaW4='})
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado/web.py", line 1443, in _execute
result = method(self.path_args, *self.path_kwargs)
File "polyglot/element_manager/api.py", line 173, in get
self.send_json(PGLOT.nodeservers.servers[base_url].definition)
KeyError: u'3wb9F'
—
Reply to this email directly or view it on GitHubhttps://github.com//issues/7.
I assume this comes from several things, but should we connect to the ISY and make sure that is active before loading the node servers? Also I noticed thread cleanup on exit needs to be done a little cleaner. Is that approach acceptable?
I think this happens because the ISY pings the node server before polyglot has actually started the node server when it's starting up?
That makes more sense. Unless we start the node-servers first, but then that seems like the node server would have issues. I guess we could thread start both, but node manager already threads. Thread in thread nesting? Interesting thought. Perhaps we just let it be for now and see how much push back the end-users have.
I think that polyglot can just ignore the requests coming from the ISY until it is done starting up?
I think this is the best approach: Polyglot ignoring requests from ISY till it’s ready.
With kind regards,
Michel Kohanim
CEO
(p) 818.631.0333
(f) 818.436.0702
http://www.universal-devices.comhttp://www.universal-devices.com/
From: jimboca [mailto:notifications@github.com]
Sent: Saturday, March 19, 2016 5:53 PM
To: UniversalDevicesInc/Polyglot Polyglot@noreply.github.com
Cc: Michel Kohanim michel@universal-devices.com
Subject: Re: [Polyglot] Errors when starting up Polyglot before servers are created (#7)
I think that polyglot can just ignore the requests coming from the ISY until it is done starting up?
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHubhttps://github.com//issues/7#issuecomment-198816573
I am not actively dealing with Polyglot at this time but have written many systems that interface to the ISY over TCP. I offer the following commenst:
1: Until the Polyglot system is ready to deal with all ISY transmissions of any type including a ping, it should disregard them.
2: The Polyglot system must assume that the ISY will disappear at any time and reappear later. In this respect I always time out heartbeats from the ISY Subscription service.
From: Michel Kohanim [mailto:notifications@github.com]
Sent: Sunday, March 20, 2016 2:05 PM
To: UniversalDevicesInc/Polyglot
Subject: Re: [Polyglot] Errors when starting up Polyglot before servers are created (#7)
I think this is the best approach: Polyglot ignoring requests from ISY till it’s ready.
With kind regards,
Michel Kohanim
CEO
(p) 818.631.0333
(f) 818.436.0702
http://www.universal-devices.comhttp://www.universal-devices.com/
From: jimboca [mailto:notifications@github.com]
Sent: Saturday, March 19, 2016 5:53 PM
To: UniversalDevicesInc/Polyglot Polyglot@noreply.github.com
Cc: Michel Kohanim michel@universal-devices.com
Subject: Re: [Polyglot] Errors when starting up Polyglot before servers are created (#7)
I think that polyglot can just ignore the requests coming from the ISY until it is done starting up?
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHubhttps://github.com//issues/7#issuecomment-198816573
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view #7 (comment) it on GitHub https://github.com/notifications/beacon/ABE23DkieB92TquicXrdCpQWH4Bwqb5rks5pvYxMgaJpZM4HvkJ5.gif
1: Until the Polyglot system is ready to deal with all ISY transmissions of any type including a ping, it should disregard them.
2: The Polyglot system must assume that the ISY will disappear at any time and reappear later. In this respect I always time out heartbeats from the ISY Subscription service.
- It does deal with all ISY transmissions, the communication between Polyglot and the ISY is documented in UDI node server documentation.
- It does a good job of handling timeouts, and it doesn't use a Subscription service. The ISY communicates with Node servers thru a pipe.
The only issue here is a startup ordering issue. Polyglot starts up, but during that startup time ISY asks for a node server status, but Polyglot hasn't started the node server yet, so it prints a traceback in the log. Later, polyglot starts the node server and everything is ok.
Adding the following instance of this error (culled from the forum) -- I suspect there are a lot of places in the code that we need to check the existence of the key (node server) before using.
ERROR [05-10-2016 17:30:35] tornado.application: Uncaught exception GET /api/server/KSjQN (192.168.1.88)
HTTPServerRequest(protocol='http', host='192.168.1.118:8080', method='GET', uri='/api/server/KSjQN', version='HTTP/1.1', remote_ip='192.168.1.88', headers={'Accept-Language': 'en-US,en;q=0.8', 'Accept-Encoding': 'gzip, deflate, sdch', 'Host': '192.168.1.118:8080', 'Accept': '*/*', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36', 'Connection': 'keep-alive', 'Referer': 'http://192.168.1.118:8080/static/index.html''If-None-Match': '"4cee6c984f617ddc3f8499d5088b1b56a3019206"', 'Authorization': 'Basic YWxleGI6RmFkZTJibGFjayE='})
Traceback (most recent call last):
File "/tmp/polyglot_jc6cBB/tornado/web.py", line 1413, in _execute
result = method(*self.path_args, **self.path_kwargs)
File "/tmp/polyglot_jc6cBB/polyglot/element_manager/api.py", line 173, in get
self.send_json(PGLOT.nodeservers.servers[base_url].definition)
KeyError: u'KSjQN'
ERROR [05-10-2016 17:30:35] tornado.access: 500 GET /api/server/KSjQN (192.168.1.88) 75.62ms
Added test of base_url before using it, and return a 404 if the node server referenced does not yet exist in the dict. This resolves the error in the log.