Killing an application with a subscriber factory yields a non clean shutdown

[ansman]

If you run the subscriber.py example and kill the server with ctrl+c you get the following:

2013-09-11 23:29:43-0700 [-] Log opened.
2013-09-11 23:29:43-0700 [-] twistd 13.0.0 (/Users/nicklas/.wrapp/virtualenvs/wrapport/bin/python2.7 2.7.5) starting up.
2013-09-11 23:29:43-0700 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2013-09-11 23:29:43-0700 [-] Starting factory <__builtin__.myFactory instance at 0x10a8850e0>
2013-09-11 23:29:43-0700 [Uninitialized] waiting for messages...
2013-09-11 23:29:43-0700 [Uninitialized] use the redis client to send messages:
2013-09-11 23:29:43-0700 [Uninitialized] $ redis-cli publish zz test
2013-09-11 23:29:43-0700 [Uninitialized] $ redis-cli publish foo.bar hello world
2013-09-11 23:29:43-0700 [-] pattern=subscribe, channel=zz message=1
2013-09-11 23:29:43-0700 [-] pattern=psubscribe, channel=foo.* message=2
^C2013-09-11 23:29:45-0700 [-] Received SIGINT, shutting down.
2013-09-11 23:29:45-0700 [myProtocol,client] lost connection: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion: Connection lost.
2013-09-11 23:29:45-0700 [myProtocol,client] ]
2013-09-11 23:29:45-0700 [myProtocol,client] <twisted.internet.tcp.Connector instance at 0x10a515b90> will retry in 3 seconds
2013-09-11 23:29:45-0700 [myProtocol,client] Stopping factory <__builtin__.myFactory instance at 0x10a8850e0>
2013-09-11 23:29:45-0700 [-] Main loop terminated.
2013-09-11 23:29:45-0700 [-] Server Shut Down.

Is there any way to prevent this "non clean" shutdown?
It would be kinda nice to be able to wait for all pending requests to finish before closing it.

I can add that this doesn't happen when using redis.Connection.

[fiorix]

I think one way to prevent this non-clean shutdown would be to handle signals and make your software wait for all redis commands to be finished before closing the connection. It shouldn't be handled in the library.

Regarding redis.Connection, it's the very same code therefore should be no difference.

[ansman]

I don't really agree.
When stopping a factory you can gracefully stop all connections, it's all built in to twisted.

Also, perhaps it should be the same with redis.Connection but it's not if you try it.

[fiorix]

What's your suggestion to fix it?

[ansman]

I'll do some investigation and get back to you. Thanks for being open minded :)

[ansman]

Ok, I'm going to close this one.

Here is the solution I came up with (you have to use a service instead of an application):

import txredisapi as redis

from twisted.internet import reactor, defer
from twisted.application import service


class myProtocol(redis.SubscriberProtocol):
    def connectionMade(self):
        # This is important, otherwise  self.factory.handler.disconnect() is a noop
        self.factory.addConnection(self)
        # ...

    def messageReceived(self, pattern, channel, message):
        pass

    def connectionLost(self, reason):
        self.factory.delConnection(self)
        print "lost connection:", reason


class myFactory(redis.SubscriberFactory):
    # SubscriberFactory is a wapper for the ReconnectingClientFactory
    maxDelay = 120
    continueTrying = True
    protocol = myProtocol


class SubscriberService(service.Service):
    @defer.inlineCallbacks
    def startService(self):
        self.factory = myFactory()
        yield reactor.connectTCP('localhost', 6379, self.factory)

    def stopService(self):
        # This should return a deferred that fires when the shutdown is complete
        return self.factory.handler.disconnect()


application = service.Application("subscriber")
SubscriberService().setServiceParent(application)

Which yields the output:

2013-09-12 17:15:50-0700 [-] Log opened.
2013-09-12 17:15:50-0700 [-] twistd 13.0.0 (/Users/nicklas/.wrapp/virtualenvs/wrapport/bin/python2.7 2.7.5) starting up.
2013-09-12 17:15:50-0700 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2013-09-12 17:15:50-0700 [-] Starting factory <__builtin__.myFactory instance at 0x10fda2dd0>
^C2013-09-12 17:15:51-0700 [-] Received SIGINT, shutting down.
2013-09-12 17:15:51-0700 [myProtocol,client] lost connection: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionDone'>: Connection was closed cleanly.
2013-09-12 17:15:51-0700 [myProtocol,client] ]
2013-09-12 17:15:51-0700 [myProtocol,client] Stopping factory <__builtin__.myFactory instance at 0x10fda2dd0>
2013-09-12 17:15:52-0700 [-] Main loop terminated.
2013-09-12 17:15:52-0700 [-] Server Shut Down.

[fiorix]

Cool. Feel free to send a pull request if you'd like to add that to the readme.

On Sep 12, 2013, at 8:16 PM, Nicklas Ansman Giertz notifications@github.com wrote:

Ok, I'm going to close this one.

Here is the solution I came up with (you have to use a service instead of an application):

import txredisapi as redis

from twisted.internet import reactor, defer
from twisted.application import service

class myProtocol(redis.SubscriberProtocol):
def connectionMade(self):
self.factory.addConnection(self)
# ...

def messageReceived(self, pattern, channel, message):
    pass

def connectionLost(self, reason):
    self.factory.delConnection(self)
    print "lost connection:", reason

class myFactory(redis.SubscriberFactory):
# SubscriberFactory is a wapper for the ReconnectingClientFactory
maxDelay = 120
continueTrying = True
protocol = myProtocol

class SubscriberService(service.Service):
@defer.inlineCallbacks
def startService(self):
self.factory = myFactory()
yield reactor.connectTCP('localhost', 6379, self.factory)

def stopService(self):
    # This should return a deferred that fires when the shutdown is complete
    return self.factory.handler.disconnect()

application = service.Application("subscriber")
SubscriberService().setServiceParent(application)
Which yields the output:

2013-09-12 17:15:50-0700 [-] Log opened.
2013-09-12 17:15:50-0700 [-] twistd 13.0.0 (/Users/nicklas/.wrapp/virtualenvs/wrapport/bin/python2.7 2.7.5) starting up.
2013-09-12 17:15:50-0700 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2013-09-12 17:15:50-0700 [-] Starting factory <builtin.myFactory instance at 0x10fda2dd0>
^C2013-09-12 17:15:51-0700 [-] Received SIGINT, shutting down.
2013-09-12 17:15:51-0700 [myProtocol,client] lost connection: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionDone'>: Connection was closed cleanly.
2013-09-12 17:15:51-0700 [myProtocol,client] ]
2013-09-12 17:15:51-0700 [myProtocol,client] Stopping factory <builtin.myFactory instance at 0x10fda2dd0>
2013-09-12 17:15:52-0700 [-] Main loop terminated.
2013-09-12 17:15:52-0700 [-] Server Shut Down.
—
Reply to this email directly or view it on GitHub.