celery apply_async() can not send app_context in args

[buptmuye]

When i use celery and apply_async(), i got such error:

EncodeError: Can't pickle class 'werkzeug.datastructures.ImmutableMultiDict': it's not the same object as werkzeug.datastructures.ImmutableMultiDict

@celery.task
def send_async_email(app, msg):
with app.app_context():
mail.send(msg)

def send_email(to, subject, template, *_kwargs):
app = current_app._get_current_object()
with app.app_context():
msg = Message(app.config['FLASKY_MAIL_SUBJECT_PREFIX'] + ' ' + subject,
sender=app.config['FLASKY_MAIL_SENDER'], recipients=[to])
msg.body = render_template(template + '.txt', *_kwargs)
msg.html = render_template(template + '.html', **kwargs)
send_async_email.apply_async(args=[current_app, msg])

but if i don't send current_app to send_async_email, then i got "RuntimeError: working outside of application context" in celery terminal outputs.

Can you help me ?

[miguelgrinberg]

You don't have to send the app as an argument, the Celery worker has access to an application instance of its own. Just pass the message as an argument to the remote task.

[mcsquaredjr]

How do I get access to application context from a worker?

[miguelgrinberg]

@mcsquaredjr you can pass any specific the information that you need from the application and request contexts as arguments into your function, as this will ensure this information is available to the worker process, even after the app/request contexts in the main process are destroyed. For stuff that is not request specific, such as configuration values, the worker can create its own application instance and application context. This is what I did to get Flask-Mail to work in the worker.

[mcsquaredjr]

Thanks for your prompt response, Miguel. Makes sense.

[ar-anvd]

@miguelgrinberg I am having a similar issue I think. The error on celery console is

NameError: global name 'app' is not defined.

Can you help please? I already try to pass app = current_app._get_current_object() to send_async_email_celery, but then i will get EncodeError

from app import db, celery

def send_emails(date_to_find):            
    msg = Message('Hello from Flask', recipients=['xxxx@hotmail.com'])
    msg.body = 'This is a test email sent from a background Celery task.'

    send_async_email_celery.delay(msg)
    return True

@celery.task
def send_async_email_celery(msg):
    with app.app_context():
        mail.send(msg)

[miguelgrinberg]

@DmonteiroD the Flask application and the celery worker do not share an application object. Each has its own. In your Celery worker you have to create an app instance. If you define your app instance as a global object, then just import it in the Celery task that you need it. If you are using a factory function, create an app when the Celery worker starts.

[ar-anvd]

@miguelgrinberg thanks, i followed your example in flasky-with-celery, and now works correctly. thanks for your great help to the community.

[sijeesh]

@miguelgrinberg
Nicely explained, I was looking for a solution for the same problem.

[miguelgrinberg]

This issue will be automatically closed due to being inactive for more than six months. Seeing that I haven't responded to your last comment, it is quite possible that I have dropped the ball on this issue and I apologize about that. If that is the case, do not take the closing of the issue personally as it is an automated process doing it, just reopen it and I'll get back to you.