bryanlabs/cosmos-indexer-sdk

Preprocess BlockEventType and BlockEventAttributeKey relations before BlockEvent bulk insertion

Closed this issue · 1 comments

The way the new models work here:

https://github.com/DefiantLabs/cosmos-indexer/blob/b0e4a9897e1db4a7340f4dd27504b92af95c4eb2/db/models/block.go#L26-L58

will take a bit of preprocessing to make sure the model relationships exist before BlockEvent Bulk insertion.

The basic workflow will be:

  1. Keep a mapping unique of BlockEventTypes and BlockEventAttributeKey
  2. Insert the unique items first with an OnConflict do nothing returning id clause
  3. Loop through the BlockEvent and apply the correct model items

This will be possible in a brute force manner, but we should attempt to see if this is possible in GORM as a one-liner.

Doing the above will allow the app to do bulk insertion in GORM rather than doing a naive insertion loop that will hit the database much harder.

This will also be a similar principal that can be applied to Message Event insertion.

Closed by #27