Becksteinlab/zarrtraj

More efficient buffering algorithms

Opened this issue · 0 comments

  1. Setup ASV benchmarks to get a baseline of performance in current solution
  2. Experiment to find the fastest on-average fixed chunk size
  3. In reader, use trajectory slicing to pull the next needed chunks rather than allowing LRU cache to handle it
  4. In writer, experiment with parallelizing writes across different datasets (IE step, time, etc) to see if engaging more cores improves performance in IO operations