Implementation of ReconnectingConnectionPool ?
Closed this issue · 11 comments
Hi,
I'm a newbie with txpostgres, but a feature is missing for me, it's an automatic reconnection pool, as described here for adbapi:
http://www.gelens.org/2009/09/13/twisted-connectionpool-revisited/
From my point of view, it's an important feature in a daemon, I don't understand isn't standard in adbapi.
With txpostgres, I've read some discussions about this in #18 and #20 but nothing to implement this directly inside txpostgres.
Why it isn't implemented in txpostgres ? Lack of time to do this, or it's a mistake to implement a generic reconnecting connection pool ?
Thanks for your answer.
Regards.
Lack of time, really, and it's difficult to implement a solution that's generic enough to fit everyone.
If someone would contribute a nice reconnecting pool, I'd be happy to review and merge it (or I might get around to extracting the internal solution I have and putting it directly in txpostgres).
If you can extract your internal solution, even if it isn't generic for everyone with some hardcoded things, I should help to clean-up and add more generic stuff inside.
You should publish in a separe branch if you want.
I'd definitely love to see this!
@wulczer If you don't have the time to integrate this in txpostgres, could you publish a code snippet to help others ?
Thank you.
@GMLudo unfortunately it's part of a proprietary codebase, so I'll have to find a free evening to extract it, generalise the code and port the tests to the suite txpostgres uses.
The most important insight I can offer right now is that the correct place for the reconnection mechanism to hook into is the connectionLost method. It should also implement some kind of reconnection interval policy.
Let me see if I'll find the time to abstract out the important bits and commit them this week.
Hi folks!
I just pushed a bunch of commits implementing automatic reconnection.
The docs are here: http://txpostgres.readthedocs.org/en/latest/usage.html#automatic-reconnection
Please give it a try and let me know if you find it useful, not too cumbersome to use etc. I've been using code similar to this in production for over two years and it's been holding up all right.
I'll probably make some small adjustments based on feedback,fix the inevitable Travis unit test errors and wrap a 1.2.0 release to put it on PyPI.
Hi wulczer!
I've been playing around with your implementation of the automatic reconnection, it works well, thanks for the work!
There is just one thing that would be nice to have, which would be to relaunch the queries that failed during the loss of connection when the connection is recovered (so that the connection loss is invisible for the user).
At the moment, i don't see a clean way to implement that, but maybe you have an idea :)
Hi!
Thanks, I'm glad you're finding it useful.
Retrying queries as a general feature isn't something I could add: it's always what you want, in fact. Suppose I service an API request and find that the connection is down. I return an error code and the client assumes that it has failed and could retry later. If I replay that query once the connection recovers, I would end up in a wrong state, where I told the client their query failed and then went ahead and applied it.
If you'd like to implement something like this in your own code, where you explicitly choose that behaviour, my first attempt would be to add a wrapper class that would act as a buffer. All queries would be added to the buffer and successful queries removed from it. If a query fails because the connection is down, it's kept in the buffer until connection restitution and then replayed.
@wulczer Any hope of getting your auto-reconnecting pool in a release on PyPi? Sure would love to not have to pull from git with pip!
@gtaylor Version 1.2.0 is on PyPI (https://pypi.python.org/pypi/txpostgres/1.2.0) and it includes the reconnection features, or am I confused?
I just released 1.4.0 (1.3.0 was a little disaster, I tagged a commit that's not reachable from master and also included some misguided code there).
Closing this issue, since reconnection is now easy to implement in user code, using the primitives I described (look at tests for guidance on exactly how).