http-party/node-http-proxy

Error: socket hang up

Closed this issue · 17 comments

trusktr commented

I'm getting errors that look like this happening every hour or two, causing systemd to restart my nodeProxyServer.js script:

Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: throw arguments[1]; // Unhandled 'error' event
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: ^
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: Error: socket hang up
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at createHangUpError (http.js:1360:15)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at ServerResponse.OutgoingMessage._writeRaw (http.js:507:26)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at ServerResponse.OutgoingMessage._send (http.js:476:15)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at ServerResponse.OutgoingMessage.write (http.js:740:18)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at ClientRequest.proxyError (/srv/nodeProxyServer/node_modules/http-proxy/lib/node-http-proxy/http-proxy.js:211:13)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at ClientRequest.g (events.js:192:14)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at ClientRequest.EventEmitter.emit (events.js:96:17)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at Socket.socketCloseListener (http.js:1411:9)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at Socket.EventEmitter.emit (events.js:126:20)
Nov 10 16:49:17 fountainsunique nodeProxyServer[24009]: at Socket._destroy.destroyed (net.js:358:10)
Nov 10 16:49:17 fountainsunique nodeProxyServer[27651]: Proxy server started on port 80.
Nov 10 16:51:17 fountainsunique nodeProxyServer[27651]: res.end error: socket hang up

May it be related to node-http-proxy, or something else?

yawnt commented

can you paste me an issue that demonstrates the problem? also are you using caronte or the master branch?

jondubois commented

I also encounter the issue.

I'm using sample code from the README:

var http = require('http'),
    httpProxy = require('http-proxy');
//
// Create your proxy server and set the target in the options.
//
httpProxy.createProxyServer({target:'http://localhost:9000'}).listen(8000);

//
// Create your target server
//
http.createServer(function (req, res) {
  res.writeHead(200, { 'Content-Type': 'text/plain' });
  res.write('request successfully proxied!' + '\n' + JSON.stringify(req.headers, true, 2));
  res.end();
}).listen(9000);

One way to reproduce is to refresh the page many times really quickly (hold down refresh keys). The 'socket hang up' error kills the process if not handled correctly.

EDIT The issue appears to be caused by this line: https://github.com/nodejitsu/node-http-proxy/blob/master/lib/http-proxy/passes/web-incoming.js#L127 - It looks like Node.js always throws an error when you call req.abort().
I think ideally Node.js shouldn't raise an error there because the abort() in this case is fully intentional - Errors should only be raised when an unexpected condition occurs.

jcrugzz commented

@jondubois You should listen on the error event for the proxy so that you can handle the errors and return proper responses when hitting this case. It is most likely triggered because there are not anymore sockets to be used and there is no socket pooling by default.

seglo commented

I noticed that when I revert back to the 0.10.x version I don't encounter this problem. It looks like 1.x.x was a fairly significant rewrite, so it's tough to say what may have changed that could be causing this issue. Perhaps someone more familiar with codebase could comment?

jcrugzz commented

@seglo in 0.10.x there was a proxyError event that was emit when something happened with the conenction so there could be a response sent to the client. This is now an error event. For example usage, see here. The error event should not go unhandled, its the default behavior of the EventEmitter to throw when it has no listeners.

  • ❤️1
seglo commented

@jcrugzz I see, OK I'll start handling that. Are you saying that with 0.10.x a proxyError event would be raised, but is internally handled?

jcrugzz commented

@seglo in 0.10.x a proxyError event would be raised but it wouldn't crash the server if it wasn't handled. You just wouldn't properly respond to clients who disconnect if you didn't handle the event

jondubois commented

I feel that this error isn't very useful at all for the following reasons:

  • It happens predictably every time a connection is intentionally aborted by the client (can be quite often)
  • There is no way to 'properly respond to clients' since the connection is now closed so when this error happens, there is no way to inform the client anyways (all we can do is log it).

Also using an agent for socket pooling doesn't fix the issue.

In my particular project (which makes use of http-proxy internally), for a while, I was just logging the error (not doing anything else) but it seemed to cause confusion among some developers who thought that it was an actual error that they needed to handle.

Unlike other Node.js errors, this error appears to be completely harmless - It doesn't cause any side effects if you ignore it (I've tested this for a while) - No memory leaks. I feel that exceptions are not exceptions if they happen predictably during a common use case of the system (in this case, aborting the connection). I think it's more of a Node.js issue than an http-proxy issue - I was able to reproduce this issue in Node by creating a client request and then aborting the connection (it would throw an error whenever I called req.abort()).

seglo commented

@jcrugzz Thanks for the example. I've made the appropriate updates to my project.

jcrugzz commented

@jondubois my explanation was how the error event in general should be handled. In node the semantics in various libraries are never consistent. Socket hangups don't only happen with that particular edge case you ran into. Do you have a gist of that consistent reproducible case? Im curious how its hitting the code paths in node core.

ashu-daffodil commented

Hi I just upgraded to version 1.7.0 of http-proxy and getting following error at random occasions.

Error: socket hang up at createHangUpError (http.js:1472:15) at Socket.socketCloseListener (http.js:1522:23) at Socket.EventEmitter.emit (events.js:95:17) at TCP.close (net.js:465:12)

Node.js : 0.10.28
Express : 4.4.2

I want to avoid these failed calls.

pgasiorowski commented

Same here.

Error: socket hang up
    at createHangUpError (http.js:1472:15)
    at Socket.socketOnEnd [as onend] (http.js:1568:23)
    at Socket.g (events.js:180:16)
    at Socket.EventEmitter.emit (events.js:117:20)
    at _stream_readable.js:920:16
    at process._tickCallback (node.js:415:13)

Node.js: 0.10.26
Express: 4.10.4 and 4.8.5

jcrugzz commented

@ashu-daffodil can you specify what your code is doing? There was nothing in the latest release that affects how errors are handled. To me, both you and @WooDzu seem to not be handling errors when the destination you are proxying to kills the socket for whatever reason. I cannot really reason about this without a small reproducible code sample.

I general, the expectation should be to get these types of errors when there are legitimate network failures or the server you are proxying to goes down. Errors need to be handled in these cases. Please see my example proxy-server to see how errors are handled. This ensures clients are responded to when the destination is having issues.

ashu-daffodil commented

@jcrugzz
I'm even getting hangups while loading images and css besides queries and other services.

REQUESTS :
req >> /images/user.png
req >> /js/lib/text-angular.min.js

ERROR :
Error: socket hang up at createHangUpError (http.js:1472:15) at Socket.socketCloseListener (http.js:1522:23) at Socket.EventEmitter.emit (events.js:95:17) at TCP.close (net.js:465:12)

Error handling CODE :

var proxy = httpProxy.createProxyServer({});
proxy.on('error', function (err, req, res) {
maintainErrorLogs(err, req, res);
});
process.on('uncaughtException', function (err) {
maintainErrorLogs(err);
});
function maintainErrorLogs(error, req, resp) {
var reqInfo = "";
if(req){
reqInfo = "req >> "+req.url+">>>host>>>"+req.headers.host;
}
getCollection(Config.LOGTABLE, Config.LOG_DB, function (err, logCollection) {
if (err) {
printError(error, err, reqInfo,req, resp);
} else {
logCollection.insert({"errorTime": new Date(),reqInfo:reqInfo, error: error.stack || error.message || error}, function (err) {
printError(error, err, reqInfo,req, resp);
})
}
})
}
function printError(mainError, dbError, reqInfo,req, resp) {
if(reqInfo && req){
console.error(reqInfo);
}
if (mainError) {
console.error("Error in ProxyServer : " + mainError.stack || mainError.message || mainError);
}
if (dbError) {
console.error("Error in ProxyServer (DB): " + dbError.stack || dbError.message || dbError);
}
if(resp){
resp.writeHead(500, {
'Content-Type': 'text/plain'
});
resp.end('Something went wrong during redirection. We are reporting an error message.');
}
}

jcrugzz commented

@ashu-daffodil this is most likely a path issue, you should log the path that is trying to be requested through the proxy and debug backwards from there. This is not a bug with http-proxy itself. Closing this issue as there is no bug related, errors just need to be handled.

bitliner commented

Same problem here: Error { [Error: socket hang up] code: 'ECONNRESET' }.

The cose is:

app.post('/export/bis', function(req, res) {

        proxy.web(req, res, {
            target: 'http://localhost:40010/export/bis',
        });
    });
NightFury2 commented

const targetUrl = config.apiHost + config.apiPort; const proxy = httpProxy.createProxyServer({ target: targetUrl }); // Proxy to API server app.use('/api', (req, res) => { console.log(req.originalUrl); console.log(targetUrl); proxy.web(req, res, {target: targetUrl}); });

Why http proxy send a request to the right address, and receives 404, and if you open the same address through brouser gets json.
checked through console.log the issue the required address