monocongo/climate_indices

Encountering `nco.nco.NCOException: (returncode:3221225477) b''` exception when running `process_climate_indices `

Closed this issue · 2 comments

richpsharp commented

Describe the bug
Encountering a nco.nco.NCOException: (returncode:3221225477) b'' when running on command line:

>>> C:\Users\richp\miniconda3\envs\gwsc\Library\bin\ncpdq -a lat,lon,time -O --output=C:\Users\richp\AppData\Local\Temp\ncpdq_nclimgrid-prcp.ncs3f6yrd_.tmp .\nclimgrid-prcp.nc <<<
Inputs: .\nclimgrid-prcp.nc
b''
2022-11-07  13:57:47 ERROR Failed to complete
Traceback (most recent call last):
  File "C:\Users\richp\miniconda3\envs\gwsc\lib\site-packages\climate_indices\__main__.py", line 1764, in main
    netcdf_precip = _prepare_file(
  File "C:\Users\richp\miniconda3\envs\gwsc\lib\site-packages\climate_indices\__main__.py", line 1536, in _prepare_file
    netcdf_file = nco.ncpdq(input=netcdf_file,
  File "C:\Users\richp\miniconda3\envs\gwsc\lib\site-packages\nco\nco.py", line 342, in get
    raise NCOException(**retvals)
nco.nco.NCOException: (returncode:3221225477) b''
Traceback (most recent call last):
  File "C:\Users\richp\miniconda3\envs\gwsc\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\richp\miniconda3\envs\gwsc\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\richp\miniconda3\envs\gwsc\Scripts\process_climate_indices.exe\__main__.py", line 7, in <module>
  File "C:\Users\richp\miniconda3\envs\gwsc\lib\site-packages\climate_indices\__main__.py", line 1764, in main
    netcdf_precip = _prepare_file(
  File "C:\Users\richp\miniconda3\envs\gwsc\lib\site-packages\climate_indices\__main__.py", line 1536, in _prepare_file
    netcdf_file = nco.ncpdq(input=netcdf_file,
  File "C:\Users\richp\miniconda3\envs\gwsc\lib\site-packages\nco\nco.py", line 342, in get
    raise NCOException(**retvals)
nco.nco.NCOException: (returncode:3221225477) b''

To Reproduce
Steps to reproduce the behavior:

  1. Fetch precipitation netcat file nclimgrid-prcp.nc from https://www.ncei.noaa.gov/pub/data/nidis/indices/nclimgrid-monthly/base-files/nclimgrid-prcp.nc
  2. Run the following command: process_climate_indices --index spi --index spei --index palmers --index pnp --periodicity monthly --output_file_base spi --netcdf_precip .\nclimgrid-prcp.nc --var_name_precip prcp --scales 3 6 --calibration_start_year 1950 --calibration_end_year 2010
  3. Approximately 10 seconds later observe the exception above with no other output.

Expected behavior
Expected SPI to be generated.

Desktop (please complete the following information):

  • OS: Windows 11
  • Version: Python 3.10.6
monocongo commented

NCO is notoriously buggy on Windows. we get most of our issues around that. I plan to excise our use of NCO within the main processing script but for now, the workaround is usually to go ahead and do the reorientation of the data before running an SPI command. I.e. convert from (time, lat, lon) to (lat, lon, time) which should bypass the section of code where NCO comes into play. Try that and if it doesn't work we'll look into it further...

richpsharp commented

Thank you, that is helpful @monocongo. Rather than swizzle the netcdf, I made a Docker image that I can use to run process_climate_indicies at the command line and that worked just fine. I've put the Docker file with embedded documentation here in case any future user finds that useful: https://github.com/Global-Water-Security-Center/data-exploration/blob/main/Dockerfile-climate-indicies

  • 🎉1