application-research/estuary

Offload Shuttle data

10d9e opened this issue · 2 comments

10d9e commented

Depends on #855

Once a root SP has been established, all of the shuttle data should be offloaded to it, and shuttle data should be emptied.
At this point, shuttles have to be reconfigured as temporary stores that immediate delete their data as soon as the 6+1 deals have been executed.

jcace commented

I wonder if we should do this in a phased approach, by first offloading to our existing SP network. We could examine:

  • Which CIDs are least-frequently retrieved
  • Of those infrequently-accessed CIDs, how many are presently retrievable from SPs (if 5/6 or 6/6, perhaps we can already offload them)
  • 👍1
alvin-reyes commented

It should be in phases.

We need to ensure that the CIDs are retrievable in any other "retrieval" providers before we offload. If we can prove this (by pinging?) the retrieval providers, we can delete the CID/content from the blockstore and store the meta information on our database. We should store where the user can look up the CID and we can present it on the frontend.

  • 👍1