datacommonsorg/mixer

[Feature Request] /stat/collection API latest date

intrepiditee opened this issue · 6 comments

intrepiditee commented

This is a feature request to make the /stat/collection API able to return the latest date (perhaps by default if date is not specified) and to include the dates in the response.

shifucun commented

@pradh Is it possible we produce the date information in the flume job as a separate cache?

pradh commented

Yeah, doable. So, would the cache key have everything as obs collection but the date?

shifucun commented

Yeah, doable. So, would the cache key have everything as obs collection but the date?

Yeah, that would work. Thanks!

shifucun commented

@pradh Can you hold for the new cache. We wonder if it's better to have the cache without date in the key, ie, the cache will have ALL data for all the dates.

This way, mixer can respond with mixed date data of different places.

pradh commented

Do we really need ALL data? For example, US states have population data from 1920. Or, do we restrict time-series to up to N latest data points?

shifucun commented

This is solved in #512 by using the ObsSeries cache, scan through each sources and fetch the latest stat