TypeError: expected str, bytes or os.PathLike object, not NoneType
Closed this issue · 10 comments
How to craft a useful, minimal bug report
I am on windows 10 and try to calculate SPEI monthly as described in the image below. I tried forward, backslashes, and double backslashes since I'm aware that NCO is for Linux. Both of my NC files have one time as a dimension, not lat or on. Both variables are expressed in mm. I still get this error as the script does not recognize the parameters in the correct units. Does it have to do with the time units?
With CDOS I looked into the timestamps; they look like this:
cdo showtimestamp mm_monthly_1979_2019_tp_celle.nc
1979-01-16T00:00:00 until 2019-12-16T00:00:00
From the documentation is not clear to me if mm is the correct unit and if the NetCDF can be a field average as in my case. That's why I do not have lat, lon info.
Attached you find the 2 NetCDF files that I used for this test.
Screenshots
Desktop (please complete the following information):
- OS: \Windows 10
I solved the error above by switching to Linux, something did not work with nco on Windows. But this lead me to a new error:
$ process_climate_indices --index spei --periodicity monthly --netcdf_precip /home/chris/Documents/climate_indices/prec/mm_monthly_1979_2019_tp_celle.nc --var_name_precip tp --netcdf_pet /home/chris/Documents/climate_indices/et/ET_monthly_celle.nc --var_name_pet ET --output_file_base /home/chris/Documents/climate_indices/spei/output --scales 12 --calibration_start_year 1979 --calibration_end_year 2019 --multiprocessing all
2021-03-15 20:00:22 INFO Start time: 2021-03-15 20:00:22.904565
2021-03-15 20:00:23 INFO Computing 12-month SPEI/Pearson
2021-03-15 20:00:23 ERROR Failed to complete
Traceback (most recent call last):
File "/home/chris/miniconda3/envs/mypython3/lib/python3.9/site-packages/climate_indices/main.py", line 1749, in main
_compute_write_index(kwrgs)
File "/home/chris/miniconda3/envs/mypython3/lib/python3.9/site-packages/climate_indices/main.py", line 1021, in _compute_write_index
_parallel_process(keyword_arguments["index"],
File "/home/chris/miniconda3/envs/mypython3/lib/python3.9/site-packages/climate_indices/main.py", line 1234, in _parallel_process
params["sub_array_end"] = split_indices[i + 1]
IndexError: list index out of range
Traceback (most recent call last):
File "/home/chris/miniconda3/envs/mypython3/bin/process_climate_indices", line 8, in
sys.exit(main())
File "/home/chris/miniconda3/envs/mypython3/lib/python3.9/site-packages/climate_indices/main.py", line 1749, in main
_compute_write_index(kwrgs)
File "/home/chris/miniconda3/envs/mypython3/lib/python3.9/site-packages/climate_indices/main.py", line 1021, in _compute_write_index
_parallel_process(keyword_arguments["index"],
File "/home/chris/miniconda3/envs/mypython3/lib/python3.9/site-packages/climate_indices/main.py", line 1234, in _parallel_process
params["sub_array_end"] = split_indices[i + 1]
IndexError: list index out of range
Sorry you had to work out that NCO issue on your own, @andiefajardo . Most of the issues we get are related to that and there are plans to excise the NCO code from this package's main driver script as it has turned out to be more of a hindrance than a help.
If you can please post a link to your datasets so I can try this out on my machine to see if I can diagnose the issue. I'm quite busy these days so I may not have time to look at it soon, but maybe another user will chime in with some help in the meantime.
Dear James,
thanks for your reply. I attach the datasets that I used to test the calculation of SPEI. After reading a bit, I wanted to ask if the NetCDF files should have information on lat/lon or they could be spatial average of precipitation and PET for example? Maybe that's why I am getting this error.
Looking forward to hearing from you with your insights.
https://github.com/monocongo/climate_indices/files/6172680/nc_files.zip
nc_files.zip
Yes, this is most likely at play here -- it is expecting a time/lat/lon in that script so if you only have a single averaged point then try creating the NetCDF with a single lat and single lon to represent that point, it shouldn't matter what you use as long as it's not too far north since that will monkey with the PET calculation. You could also read the data in as a numpy array and use the lower level API in index.py, which is what the main driver script does essentially. But that might be too complex for you to get into and it's probably easier to cook up the NetCDF in a way that'll satisfy the current processing script. So try creating the single lat and lon dimensions/coordinate and make the precip/temp data have the shape (1, 1, timesteps). If this is unclear or doesn't work out please let me know and I'll try to help more but maybe this is a good first thing to try.
My dimensions in the files above are time/lat/lot (time,1,1). I tried with your suggestion of the shape (1,1,timesteps) but I get the same error as described above sadly. Could the error be in the size o lat/lot?
@andiefajardo I get the same error using a daily netcdf precipitation to compute the SPI. Did you fix the bug ?
@andiefajardo I get the same error using a daily netcdf precipitation to compute the SPI. Did you fix the bug ?
Sorry, not at the moment!
I am also using Windows 10/11. And here is my experience for calculating SPEI successfully after encountering the same question as Author did.
First, I think we should first install necessary packages (https://climate-indices.readthedocs.io/en/latest/#download-the-code, including NCO!!) as the tutorial by @monocongo.
Then, try calculating climate indices with the example files provided by @monocongo. (https://github.com/monocongo/example_climate_indices/). This step is to make sure that the calculation code of climate indices have no problem.
Third, download or create your own *.nc files and replace the example files and calculate climate indices.
I am lucky that the first two steps have no obstacles. But the problem of Author showed up when I was trying the third step. And this step took me serveral hours and I nearly give up. The last thing I try to do is imitating almost every format (dimensions, attributes number, variables name and so on) of the example *.nc files. Especially the order of ["time", "lat", "lon"], which is the core reason for my failure to process the thrid step.
Wish my experience would help somebody.
Thanks for this explanation @levisocool
The issue is that we re-orient the data within our processing script using NCO. This usually works well on Linux/OSX but usually croaks on Windows. The workaround, for now, is to re-orient the input NetCDF's variable dimension order to (lat, lon, time), so that once our code sees that it'll know it doesn't need to run the re-orientation task using NCO.