How can you use schemas in this project with celery

[0xecute]

Hello,
I have to implement some scheduled tasks for my project. I am doing it with Celery.
The tasks have to connect to the database, read and write data.
The thing is, I have all my schemas with the flask rest server, using flask-sqlalchemy.

Which solution you think would be better? I don't want to duplicate schemas to have one version with and one without flask-sqlalchemy.

So, my question is, how can you share models between the flask server and another non-flask application?

Thank you in advance for your ideas!

[frol]

I design my services so every piece of data is owned by only one application. This means that either scheduled tasks own their own data in their own DB (well, the DB server can be shared, but services use separate databases), OR scheduled tasks make queries to the API server, OR introduce a separate "internal" API service, so the public API server and the scheduled tasks operate via this internal API service.

I have had the experience implementing API server and scheduled tasks in a single code base, and, mind you, you don't want to make that mistake; that goes cumbersome really quickly especially with Celery [I also strongly recommend avoid Celery; take a look at Dramatiq or Kuyruk].

I usually go with the second option (a single API server completely owns the data, and I make regular API calls from the scheduled tasks using an "internal" role for that).

P.S. I close this issue to mark it as answered, but feel free to leave your comments, and let us discuss the topic if necessary.

[0xecute]

Thanks for the quick answer
I use Celery because I need to schedule tasks every 15 minutes, and to have a simple dashboard (flower does it perfectly) to know if a task has failed etc... I m not sure Kuyruk does this, it looks more like a distributed task manager. Right?

Having an "internal" API service may be an acceptable solution, I think I will go for it.

THank you

[frol]

it looks more like a distributed task manager. Right?

You are right. It is just like Celery in that respect. If you are just running some regular tasks, you may just as well run a thread inside your Flask application and do the job using something like https://github.com/dbader/schedule.

[0xecute]

Thanks for the advice.
I don't think it is good to mix businesses. Schedule job has nothing to do on a web Rest API for me and should be done aside.
But as you said, I will call the "internal" api from celery. And thanks to flower I will have all my tasks status and logs accessible easily.

Thank you for the help.