Let the driver determine the appropriate payload storage format
tonyarnold opened this issue · 3 comments
I'm frustrated that when I use the excellent Fluent Driver for Queues, the payload/JobData enforces storage as raw Data. This ends up meaning that for Postgres and MySQL, it's a column containing [UInt8], which is far from ideal.
I assume that this was done for the early drivers, which require storage this way, but it would be good to unpick this.
Offering the payload as Codable and expecting the driver to handle persistence would be a more flexible approach, and perhaps allow the use of things like Postgres' jsonb type for far easier inspection and debugging of in-flight jobs.
@tonyarnold thanks for the suggestion! On mobile so apologies for the quick response/possibly inaccurate information, but if I remember correctly we had to store the data inside of JobData as [UInt8] instead of Codable because we ran into problems with Codable not being allowed to be used as an existential, and some other problem when trying to use it as a generic. Sorry for the vagueness, when I'm back at a computer I'll see if I can dig in and remember exactly why we made that choice.
Surely that's the concern of the driver though, right — shouldn't Queues itself only be saying "here's what I need you to persist"?
@tonyarnold agreed, but I believe the only way we were able to coerce the type system into doing that was dropping down to bytes. Will have to carve out some time this week to take a look, in the meantime @gwynne I believe you helped me and @tanner0101 when we were writing that part of it, let me know if you recall any of this