tiagocoutinho/multivisor

Backend server hot reload in development mode

Closed this issue · 3 comments

guy881 commented

Hello, when running multivisor web server with command python -m multivisor.server.web -c multivisor.conf and with flask env variable FLASK_ENV=development I don't get a server reload after changes in .py files. Which is painful of course, because I keep on forgetting about reloading it manually. I can see that project is using WSGI server from gevent.pywsgi module:

    http_server = WSGIServer(bind, application=app)
    logging.info('Start accepting requests')
    try:
        http_server.serve_forever()
    except KeyboardInterrupt:
        log.info('Ctrl-C pressed. Bailing out')

I tried to replace this with flask run function:

    if app.debug:
        host, port = bind.split(':')
        app.run(host=host, port=port)
    else:
        http_server = WSGIServer(bind, application=app)
...

But unfortunately I get the following exception from time to time:

File "/home/stevens/PycharmProjects/multivisor/multivisor/multivisor.py", line 430, in _do_processes
joinall(tasks)
File "src/gevent/greenlet.py", line 899, in gevent._greenlet.joinall

File "src/gevent/greenlet.py", line 909, in gevent._greenlet.joinall

File "src/gevent/_hub_primitives.py", line 217, in gevent.__hub_primitives.wait_on_objects

File "src/gevent/_hub_primitives.py", line 254, in gevent.__hub_primitives.wait_on_objects

File "src/gevent/_hub_primitives.py", line 152, in gevent.__hub_primitives._WaitIterator.__next__

File "src/gevent/_hub_primitives.py", line 143, in gevent.__hub_primitives._WaitIterator.__next__

File "src/gevent/_waiter.py", line 192, in gevent.__waiter.MultipleWaiter.get

File "src/gevent/_waiter.py", line 151, in gevent.__waiter.Waiter.get

File "src/gevent/_greenlet_primitives.py", line 60, in gevent.__greenlet_primitives.SwitchOutGreenletWithLoop.switch

File "src/gevent/_greenlet_primitives.py", line 60, in gevent.__greenlet_primitives.SwitchOutGreenletWithLoop.switch

File "src/gevent/_greenlet_primitives.py", line 64, in gevent.__greenlet_primitives.SwitchOutGreenletWithLoop.switch

File "src/gevent/__greenlet_primitives.pxd", line 35, in gevent.__greenlet_primitives._greenlet_switch

LoopExit: This operation would block forever
Hub: <Hub '' at 0x7fec429e5680 epoll pending=0 ref=0 fileno=34 thread_ident=0x7fec4170c700>
Handles:
[]
Traceback (most recent call last):
File "src/gevent/queue.py", line 667, in gevent._queue.Channel._unlock
File "src/gevent/_waiter.py", line 116, in gevent.__waiter.Waiter.switch
AssertionError: Can only use Waiter.switch method from the Hub greenlet

Do you think there is any way we could solve this?

Also, I noticed that I cannot run the server the way it is described in README.md:
python -m multivisor.server -c multivisor.conf
returns
No module named multivisor.server.__main__; 'multivisor.server' is a package and cannot be directly executed
So I use: python -m multivisor.server.web -c multivisor.conf

tiagocoutinho commented

I understand your frustration. It is completely reasonable.
The thing is you definitively need to run a gevent WSGI. Main reason is the underlying ZeroRPC library is based on gevent. When you implement a coroutine based solution you need to go coroutine 100%.

Searching a bit on the web the closest solution I could find was this gist.
I didn't give it a try so I don't know if it works.
Let me know if you have time to try it out. If not I might have a look.

guy881 commented

Thanks a lot for your explaination @tiagocoutinho. I will have a look and let you know.

tiagocoutinho commented

Issue fixed in develop branch
Will become visible in next release

  • 🎉1