'Verbose' parameters for dset reads returns inaccurate until scan
mattjala opened this issue · 1 comments
The 'verbose' parameter for a dataset GET returns information based on all of the datasets' chunks, which is only aggregated whenever a bucket scan occurs. These scans occur once a minute by default, so requests will return inaccurate info until the next scan occurs.
This is relevant to HDFGroup/vol-rest#55, since that API call should return the total number of allocated bytes, which must be obtained by the verbose parameter.
Closing this as it should be resolved in master.
Since the task that collect storage statistics runs asynchronously and only looks at chunks in the storage system, clients need to do a domain PUT with rescan and flush params, followed by a dataset GET with a verbose param. See checkVerbose in tests/integ/value_test.py for an example.