meraki-analytics/datapipelines-java

DataPipeline.getMany should support merging partial results from several DataSources

robrua opened this issue · 0 comments

Currently, DataPipeline.getMany will only try the next DataSource if the current DataSource returns null. Otherwise it will faithfully return the result, even if the results are incomplete. Because of this, the only reasonable DataSource behavior is to return null unless it can provide all of the results specified in the query. Adding a mechanism to merge partial results from several DataSources would allow DataSources to return partial results, minimizing downstream work for later DataSources.