gevent not compatible in flask

[cppla]

Hello，

gevent not compatible， in flask + gevent + multiprocess + loguru

loguru don't work , and python logging library is ok

example:

# coding: utf-8

from gevent import monkey
from gevent.pywsgi import WSGIServer
monkey.patch_all()

import sys
import loguru
import datetime
import os
from multiprocessing import cpu_count, Process
from flask import Flask, jsonify, request
from kafka_handle import kafka_client
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor

logger = loguru.logger
_format = '{{"time":<green>"{time:YYYY-MM-DD HH:mm:ss}"</green>,"level":<level>"{level}"</level>,"pos":<cyan>"{name}</cyan>:<cyan>{function}</cyan>:<cyan>{line}"</cyan>,"msg":<level>"{message}"</level>}}'
_file = {
    'sink': "/tmp/log.txt",
    'format': _format,
    'level': "INFO",
    'rotation': "128 MB",
    "compression": "tar.gz",
    "enqueue": True,
    "encoding": "utf-8"
}
logger.configure(handlers=[_file])

app = Flask(__name__)

def mirrorMSG():
    for i in kafka_client.get():
        print(i)


@app.route("/putmsg", methods=['GET'])
def function_benchmark():
    msg = request.args.get("msg")
    kafka_client.send(msg)
    logger.info(msg)
    return jsonify(
        {
            "status": "ok",
            "time": datetime.datetime.now().strftime('%Y-%m-%d %H:%M'),
            "pid": os.getpid(),
            "put": msg
        }
    ), 200

def run(MULTI_PROCESS):
    if MULTI_PROCESS == False:
        WSGIServer(('0.0.0.0', 8080), app, log = None).serve_forever()
    else:
        mulserver = WSGIServer(('0.0.0.0', 8080), app, log = None)
        mulserver.start()

        def server_forever():
            mulserver.start_accepting()
            mulserver._stop_event.wait()

        for i in range(cpu_count()):
            p = Process(target=server_forever)
            p.start()

if __name__ == "__main__":
    with ProcessPoolExecutor(max_workers=3) as executor:
        executor.submit(mirrorMSG)
        executor.submit(run, (True))
    run(True)

[Delgan]

Hi @cppla. Could you please describe the bug you are facing: no output, hanging, error, etc.?

Note: You have to surround your snippet with three back-ticks if you want to create a code block: ``` (I edited your message accordingly). 👍

[cppla]

not any output in console , it can't listen port . @Delgan

telnet 127.0.0.1 8080

Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused
telnet: Unable to connect to remote host

[Delgan]

Ok, this seems caused by monkey.patch_all(). More precisely, this happens because gevent modifies the Thread class in such a way thread.start() while the logger is configured.

See also: Using gevent monkey patching with threading makes thread work serially

If possible, you should avoid call monkey.patch_all(thread=False) instead. Otherwise, you have to implement an handler which supports both gevent and multiprocessing-safety in a different way (that is, without using threads). You can also set enqueue=False but this may result in interlaced log messages in your file as processes will not be synchronized.

[cppla]

i see ， thank you very much. @Delgan