Delta Standalone Reader - Java; to read from delta tables and pass filters

[smmenon]

I have an analytical use case where I do some pre-aggregation of data into Delta Tables. Now while visualising these delta tables, I would need to apply filters on the data so that users can drill down further on the data. Is there a better possibility to read the delta tables with filters, other than reading and iterating over the data?

[allisonport-db]

Currently we don't support filtering for the Delta Standalone data-reader, although this sounds like a reasonable request. Can you elaborate on the use case you're using this for?

There is also https://github.com/delta-io/delta-rs which does support partition filtering (and does not have any spark dependencies.)

[smmenon]

Thanks @allisonport-db for the response. As mentioned above, this is an analytical scenario that deals with a retail volume of data. For e.g. a comparison between forecast vs. sales for a retail store. The plain data is available as parquets and I shall create an abstraction/pre-aggregation/pre-join based on scenario into Delta Tables.
These tables acts as virtual data models that could be used projected into an UI of choice. But these shall further be filtered based on any of the dimensions available. For e.g. just the time factor as to show based on days to weeks to months.

[smmenon]

@allisonport-db Is there something that we can expect from Delta Standalone Reader. What is the general approach if we need to use the delta tables to be displayed in a custom UI application? Considerations are huge data volume with filtering capabilities.

[vkorukanti]

This repo has been deprecated and the code is moved under connectors module in https://github.com/delta-io/delta repository. Please create the issue in repository https://github.com/delta-io/delta. See #556 for details.