Reorder dimension for SPI in Windows
Closed this issue · 2 comments
Usually I use climate-indices in Mac to calculate SPI and always worked. Now I need to write all the procedure from preparing the data to SPI calculation in Windows machine for training purpose, and experiencing error during calculation.
My error is similar with #387
Seems I must modify the dimension order from "time,lat,lon" to "lat,lon,time" in order to run in Windows machine.
I follow your suggestion in above issue by using ncpdq tool: ncpdq -a lat,lon,time input.nc -o output.nc
Unfortunately after waiting 3 hours, it failed.
My data is here (5.93GB): https://www.dropbox.com/s/m31hshoqtj069no/wld_cli_chirps_1months_1981_2020_d1.nc?dl=0
Do you have any suggestion or other tool/script to reorder the dimension?
Desktop:
- OS: Windows Server 2019
- Miniconda3, python=3.8.7
NCO is not reliable on Windows. Almost 90% of the errors that are reported here are a result of the NCO code that I have cooked into this package's main processing script. As a result I plan to remove the NCO-related code here, I just haven't found the time yet. The main thing to do is to remove that section of the code and then prominently document the fact that this package will not perform well unless we order the dimensions as (lat, lon, time).
My only advice is to use NCO for this task on Linux or Mac. Windows will cause you headaches, as you've discovered. I have tried to run the NCO commands on your file on my Linux laptop with 32GB of RAM and it croaked, so you'll probably need to split the file and reassemble after the re-order unless you have a machine with lots of RAM. Maybe ask about this on the NCO help forum, almost certainly they'll have good guidance for you.
BTW the requirement to order the dimensions is simply for performance reasons and is helpful on any OS. This is a result of how we do looping internally, in that the order prescribed is the optimal order of dimensions.