nervous-systems/hildebrand

Build a simple schema migration system

Opened this issue · 0 comments

moea commented

I haven't thought much about this, but I think it would be pretty easy and super useful. We'd probably be reading partial create/update statements from disk (?), or whatever, and storing the current version numbers in an internal Dynamo table. If we were worried about lack of atomicity it would be possible to encode the version number in the name an attribute which is added to the table each time an update is issued. Anyway generally you could have a sequence of files on disk like this:

0 (create):

{:name :xyz
 :keys [:x :y]
 :attrs [:z]
 :throughput {:read 1 :write 1}}
 ...}

1 (update throughput):

{:throughput {:read 2}}

2 (add/remove index):

{:indexes [:global [:add {:name :excellent ...}] 
                   [:remove {:name :terrible ...}]] ...}

(I'm imagining the table name is encoded in the file name, so not required for updates - or not)

There are some constraints imposed by Dynamo, e.g. inability to add local index after table creation time, and we would want to coalesce successive throughput adjustments.

The whole thing would be exposed really unobtrusively, like maybe a version of ensure-table! which brings the table up to date