BinChengZhao/delicate

can this project handle million of one time task?

vidyli opened this issue · 2 comments

I got 30 million users, every one has one task specified on at specific day, note not the same date, for example,

user1, got a task , run at 2021-10-3 23:19:00
user2, got a task , run at 2021-11-3 13:19:00
...

30 million tasks, which means we have no way to load all tasks into memory, does this project support this case please ?

Hi, friend @vidyli

I've dealt with a requirement similar to yours, kind of like sending birthday wishes to a user~

I handled this requirement by delicate like this.

Design two execution tasks that

Task A: keep processing the user's tasks to aggregate them in time on a BTree on disk or in a database.
Task B: keep getting the machine time, fetching the corresponding data from the BTree or the database, and executing the user's tasks.

I don't know if this is for you, but if you have any questions you can contact me :) .

@BinChengZhao , thanks for the clue,it is also what I did.