xitongsys/parquet-go

[Question] Read Perfomances

Opened this issue · 0 comments

Hello Folks, thanks for the great library!
I was wondering if there are any resource on tuning the library for max read performances.
I need to read a lot of parquet files and store the data in a redis cluster, my code resemble mostly the code in the example folder.
I wonder if and how readParallelism and the pr.Read(&records) are related to each other and how to tune them.
Thanks in advance!