DataPipeline.getMany should support merging partial results from several DataSources
robrua opened this issue · 0 comments
robrua commented
Currently, DataPipeline.getMany will only try the next DataSource if the current DataSource returns null. Otherwise it will faithfully return the result, even if the results are incomplete. Because of this, the only reasonable DataSource behavior is to return null unless it can provide all of the results specified in the query. Adding a mechanism to merge partial results from several DataSources would allow DataSources to return partial results, minimizing downstream work for later DataSources.