This socket has been ended by the other party
kikar opened this issue · 21 comments
Why is this happening? I connect my server to db with pg.connect, and execute few queries with the client given, and after every query, i call done(). What's wrong?
at Socket.writeAfterFIN as write
at Connection.query (/node_modules/pg/lib/connection.js:189:15)
at Query.submit (/node_modules/pg/lib/query.js:106:16)
at Client._pulseQueryQueue (/node_modules/pg/lib/client.js:293:24)
at Client.query (/node_modules/pg/lib/client.js:321:8)
Did you get this figured out? I need a larger code sample to tell you
what's wrong. I also don't know how long it takes this error to occur?
Sometime after connecting and running a couple successful queries? Probably
some kind of network partition disconnecting your clients, but I've not
really seen this error happen before.
On Mon, Feb 9, 2015 at 7:13 PM, Gady notifications@github.com wrote:
Why is this happening? I connect my server to db with pg.connect, and
execute few queries with the client given, and after every query, i call
done(). What's wrong?at Socket.writeAfterFIN as write
at Connection.query (/node_modules/pg/lib/connection.js:189:15)
at Query.submit (/node_modules/pg/lib/query.js:106:16)
at Client._pulseQueryQueue (/node_modules/pg/lib/client.js:293:24)
at Client.query (/node_modules/pg/lib/client.js:321:8)—
Reply to this email directly or view it on GitHub
#725.
Hey @brianc I'm observing this as well about 50% of the time I attempt to roll back a transaction. Here's the stack trace I'm seeing. Running pg version 4.3.0.
reassign: Error rolling back transaction: Error: This socket has been ended by the other party
Jul 29 16:21:34 shyp-api-prod app/web.2: at Socket.writeAfterFIN [as write] (net.js:275:12)
Jul 29 16:21:34 shyp-api-prod app/web.2: at [object Object].Connection.query (/app/node_modules/pg/lib/connection.js:189:15)
Jul 29 16:21:34 shyp-api-prod app/web.2: at [object Object].Query.submit (/app/node_modules/pg/lib/query.js:106:16)
Jul 29 16:21:34 shyp-api-prod app/web.2: at [object Object].Client._pulseQueryQueue (/app/node_modules/pg/lib/client.js:293:24)
Jul 29 16:21:34 shyp-api-prod app/web.2: at [object Object].Client.query (/app/node_modules/pg/lib/client.js:321:8)
Jul 29 16:21:34 shyp-api-prod app/web.2: at [object Object].wrapped (/app/node_modules/newrelic/lib/instrumentation/pg.js:183:23)
Jul 29 16:21:34 shyp-api-prod app/web.2: at [object Object].<anonymous> (/app/node_modules/newrelic/node_modules/continuation-local-storage/context.js:74:17)
Jul 29 16:21:34 shyp-api-prod app/web.2: at [object Object].wrapSegmentInvocation [as query] (/app/node_modules/newrelic/lib/transaction/tracer/index.js:259:63)
Jul 29 16:21:34 shyp-api-prod app/web.2: at DBConnection.module.exports.DBConnection.query (/app/api/domain_objects/DBConnection.coffee:101:5)
Here's that DBConnection class - planning on open sourcing it soon as a thing you could point to re: transactions if you wish.
https://gist.github.com/kevinburkeshyp/edadef3bd85b6dd0dddf
Let me know if I can provide any more debugging information - I haven't dug into this at all.
The new relic stuff is unfortunate but I'm not sure I can tear it out.
(Probably the next thing you will ask me to do is set up a reproducible test case and unfortunately this will take more time. I will try my best but it may take a while.)
I had an idea! @brianc I am wondering what happens if you are in a transaction and you try a DB write that fails a check constraint. I believe that automatically aborts the transaction in most scenarios... maybe there's a race between the transaction aborting/pgbouncer killing the connection, and the client issuing a rollback request?
We are using node-postgres (PostgreSQL client for node.js) to connect Redshift from lambda function. The connection to Redshift works properly for 80% of times, but fails randomly. The error message we got was,
2015-09-30T04:53:57.259Z 1d9361fc-672f-11e5-a760-31ab2e36abea Error: This socket has been ended by the other party at Socket.writeAfterFIN as write at Connection.query (/var/task/node_modules/pg/lib/connection.js:189:15) at Query.submit (/var/task/node_modules/pg/lib/query.js:115:16) at Client._pulseQueryQueue (/var/task/node_modules/pg/lib/client.js:298:24) at Client.query (/var/task/node_modules/pg/lib/client.js:326:8) at /var/task/index.js:98:20 at null. (/var/task/node_modules/pg/lib/client.js:184:7) at g (events.js:180:16) at emit (events.js:92:17) at Socket. (/var/task/node_modules/pg/lib/connection.js:66:10)
We are using multiple/parallel connections to Redhsift from Lambda functions.
Let us know if you have some ideas about this issue
@iamvp I don't have any ideas off the top of my head, but I'm happy to discuss deeper via email. brian.m.carlson@gmail.com
any news?
I'm receiving this error after each insert:
Error: This socket has been ended by the other party] code: 'EPIPE'
Means, first time i'm inserting it fine, but when i'm attempting to do insert another data, or retrieve data etc -i'm getting this error.
insert query example:
var query = "insert into public.comment (text,owner,owner_name,resource_type, resource_id,parent) values ('some text','40c28840-4241-4b7a-b339-29d9759693eb','UserName','note','33695e8b-5cff-42a1-9c64-7c0a3cd0f411','8a416406-c848-46e0-ac14-6aa6bcdb5044') returning owner,owner_name,text,parent,created,id"
client.query(query, function(err, result) {
done();
if (err) {
socket.emit('userError', 0) // unknownError
return console.error('ERROR '+query, err);
}
console.log(result)
});
I am also facing similar errors almost 50% of the times - while connecting to redshift from AWS lambda.
This socket has been ended by the other party at Socket.writeAfterFIN [as write] (net.js:276:12) at Connection.query (/var/task/node_modules/pg/lib/connection.js:189:15) at Query.submit (/var/task/node_modules/pg/lib/query.js:115:16) at Client._pulseQueryQueue (/var/task/node_modules/pg/lib/client.js:298:24) at Client.query (/var/task/node_modules/pg/lib/client.js:326:8) at exports.handler (/var/task/index.js:49:24)
I'm getting this too. I'm on OSX, and I think it's related to #122
Edit: Okay, this consistently fails about 30 seconds after the first connection is made. Have a feeling there's a timeout somewhere.
I'm having this problem too! Any help would be greatly appreciated! ✋
@tuckerconnelly the Wiki example you have linked to does not actually solve the problem...
It is still creating a new Postgres connection pool for each HTTP request. Those connections will need to be closed at some point else they will build up over time and slow down the server...
As for using co-pg that makes sense given that you are using Koa, but most people are not using co-routines. Also, your pseudo code will only work for a single-file app... how do you pass the pg connection to other files/handlers in your app?
I'm not too familiar with the lower-level node server stuff, but I'd take a look at the req object, and if the browser is requesting twice. I also don't see how that's related lol.
You could use a higher-order function to pass it around.
const myMiddleware = require('myMiddleware');
app.use(myMiddleware(pg))koa has a context (this) that gets passed around. Express might too, I'm not sure.
I think I've found a repro steps. Looks like the error happens in the following sequence, but when you skip client.end() then it seems working:
fails: pg.connect -> client1.query -> client1.end -> pg.connect -> client2.query
works: pg.connect -> client1.query -> client1.end -> pg.connect -> client2.query
repro sample:
var run = function(sql, cb) {
pg.connect("postgres://postgres:pwd@localhost/test", function (err, client, done) {
if (err)
return console.error('could not connect to postgres', err);
client.query(sql, function (err, result) {
done();
client.end(function(){
if (err)
return cb('error running query. ' + err);
else
return cb(null, result);
});
});
});
};
var sql = 'select 1';
run(sql, function(err, result) {
console.log('1', err || 'OK');
//output: 1 OK
run(sql, function(err, result) {
console.log('2', err || 'OK');
//output: error running query. Error: This socket has been ended by the other party
});
});the error happens on second run()
I was having the same issue, but after removing this:
client.end( function(err) {
if(err) return err
)}
this error did not occur anymore. From the README.md, if I am understanding it correctly, it seems that when calling client.end() you are disconnecting the client from the database and in order for me to get a new client, I need to restart my server. I think that what we should be doing is calling done() when completing the query, in order to send the client back to the pool of clients in order for it to be used again. Calling client.end() should only happen when there is no need for database access anymore from that client. I could be wrong but this was my understanding. Hope this helps those with issues.
when calling client.end() you are disconnecting the client from the database
@acucciniello yeap this makes perfect sense, however it should not matter, because after client.end() I'm calling pg.connect() again, and I assume this is not expected to fail :)
As acucciniello said, when using pg.connect, you’re making use of a pool, and should only release clients back into the pool with done(). Don’t call client.end().
Creating pools explicitly with the pg.Pool constructor is also preferable over the pg.connect implicit pool and will make this behaviour more obvious.
I'm not using client.end() anywhere in my code and I'm still getting this error like 9 times in a row on every login!
idle client error This socket has been ended by the other party Error: This socket has been ended by the other party
at Socket.writeAfterFIN [as write] (net.js:355:12)
at Connection.end (/Users/aa/Documents/Fonk/Node-Web/node_modules/pg/lib/connection.js:313:22)
at global.Promise (/Users/aa/Documents/Fonk/Node-Web/node_modules/pg/lib/client.js:410:23)
at Promise (<anonymous>)
at Client.end (/Users/aa/Documents/Fonk/Node-Web/node_modules/pg/lib/client.js:409:12)
at Pool._remove (/Users/aa/Documents/Fonk/Node-Web/node_modules/pg-pool/index.js:135:12)
at Timeout.setTimeout (/Users/aa/Documents/Fonk/Node-Web/node_modules/pg-pool/index.js:38:12)
at ontimeout (timers.js:488:11)
at tryOnTimeout (timers.js:323:5)
at Timer.listOnTimeout (timers.js:283:5)
Hey people,
I just went to index.js which is inside node_modules/pg-ppl and commented out the line client.end() and the error stopped showing!
But then I got into "too many clients already" error :(