ritstudentgovernment/petitions

Intermittent Authentication Failure (502 Bad Gateway)

Closed this issue · 3 comments

Original User Complaint:
Current PawPrints login is not working...if someone could investigate that would be appreciated. This is a big deal to get fixed ASAP. Thank you!

Nginx Error Log 2/6/2016 17:00PM-17:59PM Retrieved

Update 2/7/2017

Of particular interest are error-log entries that look like this:
2017/02/06 17:45:02 [error] 22266#0: *2291058 connect() failed (111: Connection refused) while connecting to upstream, [REMOVED]

After some cursory research on the Google, I have discovered an SELinux boolean that has been known to raise errors similar to those listed above. A DigitalOcean user reported that by setting httpd_can_network_connect to true, (s)he was able to eliminate both the connnect() failed and 502 Bad Gateway errors.

[REMOVED]

EDIT: This should be considered in future SG Services efforts, but in retrospect I do not believe SELinux was the root cause.

Update 2/7/2017 (End of Business)

At this time, I believe we have resolved this issue. No more 502 Bad Gateway on Pawprints Rochester.

Root Cause

One bad line in an Nginx config. Default listen 80; assumes IPv6 is in use, and then nginx gets confused.

Resolution

added listen 127.0.0.1; to port-80 server block.

Example:

server {
    listen 127.0.0.1;
    listen 80;

   [. . .]
}

Validation of Resolution

  • Apache Bench Tests
  • Manual Refresh
  • Chrome Console
  • Nginx Error Logs

Okay, I just hit Pawprints-ROC with 1 million requests, 1 thousand requests at at time. Everything came up clear in AB and manual refreshing like a maniac has yielded no 502s. I'm gonna call this one closed.

We ought to be able to take care of 502 errors in all other and future Meteor implementations.