riverqueue/river

[FR]: Job ordering

ewhauser opened this issue · 1 comments

In some systems, you need to guarantee order across a given partition key while still having multiple queue consumer threads. SQS has the concept of FIFO queues that we currently use for this.

The canonical use case a high volume SMS system where you are sending multiple messages to a specific customer and always want to send them in order.

This does seem like something that might be useful in the project — I've seen the same kind of thing elsewhere too.

I'm not sure we're likely to get to this right away, so I'd suggest this workaround in the meantime: using the SMS example again, instead of putting the SMS messages directly in a River job, hold them instead as a separate database model that's ordered naturally on insert order. Then, your River job runs and just clears whatever SMS messages it finds in the database:

func (w *SendSMSWorker) Work(ctx context.Context, job *river.Job[SendSMSArgs]) error {
    ...

    messagesToSend, err := queries.SMSMessageLoadByUser(job.UserID)
    ...

    for _, message := range messagesToSend {
        // send SMS message
    }

    return nil
}