Unable to run the code properly

Question

Unable to run the code properly

Closed this issue 6 months ago · 4 comments

Untyped global name '_norm_fitdict': Cannot determine Numba type of <class 'function'
When I run it with process_climate_indices --index spei..., the program reports an error, which seems to be a problem with the numba package

To Reproduce
Steps to reproduce the behavior:

virtualenv venv
venv\Scripts\activate
pip install climate_indices netCDF4 h5netcdf setuptools

process_climate_indices --index spei --periodicity monthly --netcdf_precip example/input/nclimgrid_lowres_prcp.nc --var_name_precip prcp --netcdf_pet example/input/nclimgrid_lowres_pet.nc --var_name_pet pet --output_file_base out/nclimgrid_lowres --scales 9 18 --calibration_start_year 1951 --calibration_end_year 2010 --multiprocessing all

and then

2024-03-26  18:28:09 INFO Start time:    2024-03-26 18:28:09.014857
2024-03-26  18:28:09 INFO Computing 9-month SPEI/Pearson
2024-03-26  18:28:13 ERROR Failed to complete
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "D:\DevTools\Python\Lib\multiprocessing\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "D:\DevTools\Python\Lib\multiprocessing\pool.py", line 48, in mapstar
    return list(map(*args))
           ^^^^^^^^^^^^^^^^
  File "E:\ILRoad\ICodes\Github\climate_indices\venv\Lib\site-packages\climate_indices\__main__.py", line 1421, in _apply_along_axis_double
    computed_array[i, j] = func1d(x[j], y[j], parameters=params["args"])
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ILRoad\ICodes\Github\climate_indices\venv\Lib\site-packages\climate_indices\__main__.py", line 1144, in _spei
    return indices.spei(precips_mm=precips,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ILRoad\ICodes\Github\climate_indices\venv\Lib\site-packages\numba\core\dispatcher.py", line 468, in _compile_for_args
    error_rewrite(e, 'typing')
  File "E:\ILRoad\ICodes\Github\climate_indices\venv\Lib\site-packages\numba\core\dispatcher.py", line 409, in error_rewrite
    raise e.with_traceback(None)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Untyped global name '_norm_fitdict': Cannot determine Numba type of <class 'function'>

File "venv\Lib\site-packages\climate_indices\indices.py", line 262:
def spei(
    <source elided>
    # Normalize fitting param keys
    fitting_params = _norm_fitdict(fitting_params)
    ^

"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "E:\ILRoad\ICodes\Github\climate_indices\venv\Lib\site-packages\climate_indices\__main__.py", line 1775, in main
    _compute_write_index(kwrgs)
  File "E:\ILRoad\ICodes\Github\climate_indices\venv\Lib\site-packages\climate_indices\__main__.py", line 1040, in _compute_write_index  
    _parallel_process(keyword_arguments["index"],
  File "E:\ILRoad\ICodes\Github\climate_indices\venv\Lib\site-packages\climate_indices\__main__.py", line 1317, in _parallel_process     
    pool.map(_apply_along_axis_double, chunk_params)
  File "D:\DevTools\Python\Lib\multiprocessing\pool.py", line 367, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\DevTools\Python\Lib\multiprocessing\pool.py", line 774, in get
    raise self._value
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Untyped global name '_norm_fitdict': Cannot determine Numba type of <class 'function'>

File "venv\Lib\site-packages\climate_indices\indices.py", line 262:
def spei(
    <source elided>
    # Normalize fitting param keys
    fitting_params = _norm_fitdict(fitting_params)
    ^

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "D:\DevTools\Python\Lib\multiprocessing\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "D:\DevTools\Python\Lib\multiprocessing\pool.py", line 48, in mapstar
    return list(map(*args))
           ^^^^^^^^^^^^^^^^
  File "E:\ILRoad\ICodes\Github\climate_indices\venv\Lib\site-packages\climate_indices\__main__.py", line 1421, in _apply_along_axis_double
    computed_array[i, j] = func1d(x[j], y[j], parameters=params["args"])
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ILRoad\ICodes\Github\climate_indices\venv\Lib\site-packages\climate_indices\__main__.py", line 1144, in _spei
    return indices.spei(precips_mm=precips,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\ILRoad\ICodes\Github\climate_indices\venv\Lib\site-packages\numba\core\dispatcher.py", line 468, in _compile_for_args
    error_rewrite(e, 'typing')
  File "E:\ILRoad\ICodes\Github\climate_indices\venv\Lib\site-packages\numba\core\dispatcher.py", line 409, in error_rewrite
    raise e.with_traceback(None)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Untyped global name '_norm_fitdict': Cannot determine Numba type of <class 'function'>

File "venv\Lib\site-packages\climate_indices\indices.py", line 262:
def spei(
    <source elided>
    # Normalize fitting param keys
    fitting_params = _norm_fitdict(fitting_params)
    ^

"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "E:\ILRoad\ICodes\Github\climate_indices\venv\Scripts\process_climate_indices.exe\__main__.py", line 7, in <module>
  File "E:\ILRoad\ICodes\Github\climate_indices\venv\Lib\site-packages\climate_indices\__main__.py", line 1775, in main
    _compute_write_index(kwrgs)
  File "E:\ILRoad\ICodes\Github\climate_indices\venv\Lib\site-packages\climate_indices\__main__.py", line 1040, in _compute_write_index  
    _parallel_process(keyword_arguments["index"],
  File "E:\ILRoad\ICodes\Github\climate_indices\venv\Lib\site-packages\climate_indices\__main__.py", line 1317, in _parallel_process     
    pool.map(_apply_along_axis_double, chunk_params)
  File "D:\DevTools\Python\Lib\multiprocessing\pool.py", line 367, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\DevTools\Python\Lib\multiprocessing\pool.py", line 774, in get
    raise self._value
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Untyped global name '_norm_fitdict': Cannot determine Numba type of <class 'function'>

File "venv\Lib\site-packages\climate_indices\indices.py", line 262:
def spei(
    <source elided>
    # Normalize fitting param keys
    fitting_params = _norm_fitdict(fitting_params)

Expected behavior
I would like to be able to run the sample code

Screenshots

Desktop (please complete the following information):

OS: Win10-64
python:3.12.2
netCDF4:1.6.5
numba:0.59.1
numpy:1.26.4
scipy:1.12.0

Answer 1 · 2024-03-26T16:18:05.000Z

Are you using the latest release from PyPI? I'm looking through the current code from the master branch and don't see any usage of numba.

Answer 2 · 2024-03-27T02:08:03.000Z

I'm using climate_indices 1.0.12, which I downloaded directly using pip

Answer 3 · 2024-03-27T02:35:07.000Z

There's your issue then. Start with a new virtual environment, then install the package with poetry. Here's what I just did on my local machine and it seemed to install fine:

# create and activate a new virtual env with conda
% conda create -n ci545 python="3.10" poetry
% conda activate ci545

# install the latest version from PyPI by "adding" the package to the environment
% python -m poetry add climate-indices

# display the installed version
% conda list | grep climate
climate-indices           2.0.0                    pypi_0    pypi

Please give that a whirl and let us know how it goes.

Answer 4 · 2024-03-27T03:03:00.000Z

Yes, it works all right, I think I automatically downloaded version 1.0 when I downloaded pip install climate-indices, and when I manually specified pip install climate-indices==2.0.0, the program works. Thank you for your help.