Constant-time manifest lookup on object storage
wjones127 opened this issue · 0 comments
wjones127 commented
Alternative to #1362
If we change the naming scheme of manifests to be zero-padded descending numbers, we can get O(1) lookup of the latest manifest via object store list operations. So the numbers would be {999999999999999999999-version}.manifest.
(Local filesystems would still have to list the fully directory, as they are not ordered, but it is already much faster and there are additional optimizations we could provide to improve performance.)