Reduce memory requirements for segy cutting
Closed this issue · 2 comments
markspec commented
mdio_to_segy requires a significant amount of memory for large datasets. This seems related to dask/dask#10535
Test dask release: 2023.10.0
fjetter commented
Is it possible to have a minimal reproducer for this memory increase? Note that we just merged another change to the relevant code sections, see dask/dask#10660 which I hope may be more stable. I would still be interested in a reproducer for your problem
Edit: I misread the issue. I thought #10535 was the cause for a regression in your code base but it appears to be the opposite. Still, I recommend checking out the new version as well. I believe this should be released this Friday. Please ping me if it doesn't work for you