Draw up catalog.json format version 3
Opened this issue · 0 comments
piroor commented
Known problems in v2 (should be fixed at version 3):
-
"replicas" appears in a dataset definition directly. It should be placed under "volume".
-
We cannot put replicas under a slice. (note: actually this can be done on the latest implementation for v2, but it is invalid as a v2 catalog.)
-
"volume" should be optional. If a slice doesn't have "volume", the slice itself should be parsed as a volume.
Otherwise, we'll see deeply nested JSONs like: "dataset-volume-replicas-slices-slice-volume-replicas-..." -
Replicas according to application by use. For example:
- Replica 1: There is only one slice. This should be used for generic search.
- Replica 2: There are time-sliced slices. This should be used for search with year.
- Replica 3: There are category-sliced slices. This should be used for search with category.
For these replicas, search requests should be delivered only for suitable replica for the use case.