Accessing to application context in a background task with Blueprints

[j2logo]

Hi again @miguelgrinberg !

In my last issue miguelgrinberg/Flask-SocketIO#651 I wasn't clear enough.

I am working on an application with the same archetype as this one (Flask-SocketIO-Chat).
I want to request the database inside a background task but if I invoke the code below, no notifications are sent. I am using eventlet.

This is my events.py

import os
from threading import Lock

from .. import socketio, create_app
from app.emc_core.models import Alert

thread = None
thread_lock = Lock()

def notifications_job():
    last_alert_id = None
    
    app = create_app(os.getenv('FLASK_SETTINGS_MODULE'))

    with app.app_context():
        while True:
            socketio.sleep(app.config.get('NOTIFICATIONS_TIMER', 60))
            # Check for new alerts
            last_id = Alert.get_last_alert_id()
            if last_alert_id is not None and last_id != last_alert_id:
                notifications['last_alert_id'] = last_id
                socketio.emit('new_notification', {'msg': 'new alert', 'last_id': last_id}, namespace='/rt/notifications/')
            last_alert_id = last_id


@socketio.on('connect', namespace='/rt/notifications/')
def start_notifications_thread():
    global thread
    with thread_lock:
        if thread is None:
            thread = socketio.start_background_task(target=notifications_job)

And this is my init.py for app:

from flask import Flask
from flask_marshmallow import Marshmallow
from flask_socketio import SocketIO
from flask_sqlalchemy import SQLAlchemy

db = SQLAlchemy()
ma = Marshmallow()

# Set this variable to "threading", "eventlet" or "gevent" to test the
# different async modes, or leave it set to None for the application to choose
# the best option based on installed packages.
async_mode = 'eventlet'
socketio = SocketIO(async_mode=async_mode)


def create_app(settings_module):
    app = Flask(__name__, instance_relative_config=True)
    # Load the config file specified by the FLOODSERV_SETTINGS_MODULE environment variable
    app.config.from_object(settings_module)
    # Load the configuration from the instance folder
    app.config.from_pyfile('config.py')

    db.init_app(app)
    ma.init_app(app)
    socketio.init_app(app)

    # Blueprints registration
    from app.emc_core.api_1_0 import emc_1_0
    app.register_blueprint(emc_1_0)

    from app.pois.api_1_0 import pois_1_0
    app.register_blueprint(pois_1_0)

    from app.sensor.api_1_0 import sensor_api_1_0
    app.register_blueprint(sensor_api_1_0)

    from app.ui import ui
    app.register_blueprint(ui)

    # Here events blueprint is registered
    from app.rt import real_time_events
    app.register_blueprint(real_time_events)

    return app

Any suggestions?

[miguelgrinberg]

Did you monkey patch the standard library?

[j2logo]

Miguel, sorry for my late reply.

No, I didn't. I will try it. Nevertheless, I'm also using celery. Would be better using a scheduled celery task or a socketio background task for this purpose?

[miguelgrinberg]

That really depends. But regardless of your choice, you need to monkey patch, so start with that.

[j2logo]

Hi Miguel, I have monkey patched the app with no success.

Here you can find my own repository with a code example https://github.com/j2logo/Flask-SocketIO-AppContext

As you can see, I have followed the same approach as in your chat example with the exception of accessing to the app context.

This code doesn't work (no event is sent):

from threading import Lock
from flask import Flask
from app import create_app, socketio

thread = None
thread_lock = Lock()

def notifications_job():
    app = create_app()
    count = 0
    with app.app_context():
        while True:
            step = int(app.config.get('STEP', 1))
            count += step
            print("Count: {}".format(count))
            socketio.emit('my_response', {'count': count}, namespace='/rt/notifications/')
            socketio.sleep(10)


@socketio.on('connect', namespace='/rt/notifications/')
def start_notifications_thread():
    global thread
    with thread_lock:
        if thread is None:
            thread = socketio.start_background_task(target=notifications_job)

However, this another one does:

from threading import Lock
from flask import Flask
from app import create_app, socketio

thread = None
thread_lock = Lock()

def custom_create_app():
    app = Flask(__name__)
    app.config.from_object('config')

    # NO socketio initialization

    # Blueprints registration (only real time events blueprint)
    from app.rt import real_time_events
    app.register_blueprint(real_time_events)

    return app


def notifications_job():
    app = custom_create_app()
    count = 0
    with app.app_context():
        while True:
            step = int(app.config.get('STEP', 1))
            count += step
            print("Count: {}".format(count))
            socketio.emit('my_response', {'count': count}, namespace='/rt/notifications/')
            socketio.sleep(10)


@socketio.on('connect', namespace='/rt/notifications/')
def start_notifications_thread():
    global thread
    with thread_lock:
        if thread is None:
            thread = socketio.start_background_task(target=notifications_job)

I think that socketio.init_app(app) in init.py is the key.

What do you think? Is custom_create_app a good approach? Do you have any example accessing to the app context?

[miguelgrinberg]

Three comments:

1. I don't see any monkey patching in your application.
2. You are creating a different app instance for your background task. You need to use the same app instance.
3. You are creating a real_time_events blueprint that is completely unnecessary, since Socket.IO events are not attached to a blueprint.

[j2logo]

Miguel sorry for my trouble.

I have committed a new version of my app https://github.com/j2logo/Flask-SocketIO-AppContext. The only way I have found to run the socketio thread is defining it in the run.py module but I don't like this approach. I would like having it in the events.py module.

Any suggestion?

[miguelgrinberg]

What is the problem if you move the Socket.IO code down to a events.py? I did that in the application in this repo and had no issues. In fact, this repo exists to demonstrate how to do it because a lot of people needed an actual example.

[j2logo]

The problem is accessing to the app context.

If I follow your example, I have no trouble:

This would be the events.py:

from threading import Lock
from .. import socketio


thread = None
thread_lock = Lock()


def notifications_job():
    count = 0
    #with app.app_context():
    #    step = int(app.config.get('STEP', 1))
    while True:
        socketio.sleep(10)
        count += 1
        print("Count: {}".format(count))
        socketio.emit('my_response', {'count': count}, namespace='/rt/notifications/')


@socketio.on('connect', namespace='/rt/notifications/')
def start_notifications_thread():
    global thread
    with thread_lock:
        if thread is None:
            thread = socketio.start_background_task(target=notifications_job)

But if I change the notifications_job function as below I think that some socketio or app import is wrong:

from threading import Lock
from .. import socketio
from run import app


thread = None
thread_lock = Lock()


def notifications_job():
    count = 0
    with app.app_context():
        step = int(app.config.get('STEP', 1))
        while True:
            socketio.sleep(10)
            count += 1
            print("Count: {}".format(count))
            socketio.emit('my_response', {'count': count}, namespace='/rt/notifications/')

Have you ever access to app context within socketio?

[miguelgrinberg]

You can pass the app instance as an argument into your thread. I actually answered a question on Stack Overflow about this same thing yesterday: https://stackoverflow.com/questions/49252601/flask-socketio-context-for-flask-sqlalchemy/49271277?noredirect=1#comment85550170_49271277

[j2logo]

🎉😊

That was great!! Just what I was looking for!! I have tried it and it runs ok. So you can close this issue.

But there is a bug in the code of stackoverflow. If you pass both keyword arguments, then the next exception is raised:

TypeError: notifications_job() got an unexpected keyword argument 'args'

So you have to pass only

thread = socketio.start_background_task(notifications_job, (current_app._get_current_object()))

Thank you very much!!!

[miguelgrinberg]

This issue will be automatically closed due to being inactive for more than six months. Seeing that I haven't responded to your last comment, it is quite possible that I have dropped the ball on this issue and I apologize about that. If that is the case, do not take the closing of the issue personally as it is an automated process doing it, just reopen it and I'll get back to you.