ZeroDivisionError
Closed this issue · 13 comments
Hi! I run version 4.2.11 (although, by the way, I downloaded the latest package, named 4.2.12). It works fine with some netcdf files, until I get this fatal error with files from a specific model.
Traceback (most recent call last): File "ICCLIMscriptPI.py", line 36, in <module> icclim.indice(in_files=ncfile[0], var_name=var_name, out_file=out_file, indice_name=indice_name, slice_mode='DJF', callback = callback.defaultCallback) File "/home/scussoli/.local/lib/python2.7/site-packages/icclim-4.2.11-py2.7.egg/icclim/icclim.py", line 678, in indice user_indice=user_indice) ## tuple: (dt_centroid_arr, dt_bounds_arr, indice_arr) File "/home/scussoli/.local/lib/python2.7/site-packages/icclim-4.2.11-py2.7.egg/icclim/icclim.py", line 870, in get_indice_from_dict_temporal_slices percentage_slice = (callback_percentage_total*1.0)/(nb_t_slices*nb_chunks) ZeroDivisionError: float division by zero
A snippet of the input nc file and the python script can be found at: https://surfdrive.surf.nl/files/index.php/s/wu7MFtLwoVGv560
Thanks!
Hi Pauli,
Thanks for the files - I will have a look on your problem within 3 days max.
Xavier
Hi Pauli,
Could you try to give a 'None' value to your callback function in icclim.indice(....,callback = None)
This might do the trick for now - not really sure.
I think (nb_t_slices*nb_chunks) is equal to 0 due to the var_name on your netcdf: "totp". Icclim expect the variable "pr" for pressure index calculation.
Thanks for looking into this!
I set callback = None, but now I get an IndexError:
Traceback (most recent call last): File "ICCLIMscriptPI.py", line 36, in <module> icclim.indice(in_files=ncfile[0], var_name=var_name, out_file=out_file, indice_name=indice_name, slice_mode='DJF', callback = None) File "/home/scussoli/.local/lib/python2.7/site-packages/icclim-4.2.11-py2.7.egg/icclim/icclim.py", line 678, in indice user_indice=user_indice) ## tuple: (dt_centroid_arr, dt_bounds_arr, indice_arr) File "/home/scussoli/.local/lib/python2.7/site-packages/icclim-4.2.11-py2.7.egg/icclim/icclim.py", line 879, in get_indice_from_dict_temporal_slices any_slice = vars_dict[vars_dict.keys()[0]]['temporal_slices'].keys()[0] #### arrays in all slices have the same shape IndexError: list index out of range
I tried on my side and it's working if I remove a raising error due to var_name. I did a new pull request disabling a certain type of error related to var_name - an error you would get with the latest version.
Once my pulling request is accepted, could you try to update your version and let me know if the problem is still here
Xavier
Thanks again Xavier,
I cloned the new master version (which, I have checked, includes your last changes), and now I get a different type of error:
Traceback (most recent call last): File "ICCLIMscriptPI.py", line 35, in <module> icclim.indice(in_files=ncfile[0], var_name=var_name, out_file=out_file, indice_name=indice_name, slice_mode='year', callback = None) File "/home/scussoli/.local/lib/python2.7/site-packages/icclim-4.2.11-py2.7.egg/icclim/icclim.py", line 483, in indice dict_files_years_to_process = files_order.get_dict_files_years_to_process_in_correct_order(files_list=VARS_in_files[v], time_range=time_range) File "/home/scussoli/.local/lib/python2.7/site-packages/icclim-4.2.11-py2.7.egg/icclim/util/files_order.py", line 57, in get_dict_files_years_to_process_in_correct_order list_files_correct_order = get_files_correct_order(files_list, time_range) File "/home/scussoli/.local/lib/python2.7/site-packages/icclim-4.2.11-py2.7.egg/icclim/util/files_order.py", line 27, in get_files_correct_order dict_file_years = get_dict_file_years_glob(files_list) File "/home/scussoli/.local/lib/python2.7/site-packages/icclim-4.2.11-py2.7.egg/icclim/util/files_order.py", line 14, in get_dict_file_years_glob dates_list_current_file = util_dt.get_list_dates(filename, 'dt') File "/home/scussoli/.local/lib/python2.7/site-packages/icclim-4.2.11-py2.7.egg/icclim/util/util_dt.py", line 83, in get_list_dates arr_dt = t.num2date(var_time[:]) File "netcdftime/_netcdftime.pyx", line 1188, in netcdftime._netcdftime.utime.num2date File "/home/scussoli/.conda/envs/test_env/lib/python2.7/site-packages/numpy/core/fromnumeric.py", line 232, in reshape return _wrapfunc(a, 'reshape', newshape, order=order) File "/home/scussoli/.conda/envs/test_env/lib/python2.7/site-packages/numpy/core/fromnumeric.py", line 57, in _wrapfunc return getattr(obj, method)(*args, **kwds) ValueError: cannot reshape array of size 1 into shape (1095,)
what are your netcdftime and numpy version?
I have:
numpy.version
'1.15.1'
netcdftime.version
'1.0.0a2'
netcdftime is the same.
I upgraded numpy to your version (a bit of messing around, as any netcdf4 package I have found wants to downgrade my numpy to 1.13.X).
I get the same ValueError as above.
There is something suprising in the log error:
File "/home/scussoli/.local/lib/python2.7/site-packages/icclim-4.2.11-py2.7.egg/icclim/util/util_dt.py", line 83, in get_list_dates arr_dt = t.num2date(var_time[:])
If you check at https://github.com/cerfacs-globc/icclim/blob/master/icclim/util/util_dt.py there is no such operation on line 83; the arr_dt = t.num2date(var_time[:]) doesn't appear at all on the new icclim
Could you try to "git clone" again the new icclim repository, reinstall the library and see if the problem persists. If it does, could you send me the script within the function get_list_dates(filename, 'dt') in /home/scussoli/.local/lib/python2.7/site-packages/icclim-4.2.11-py2.7.egg/icclim/util/util_dt.py
Many thanks
I have removed icclim and reinstalled from cloning the present master branch. I now get a different error indeeed:
Traceback (most recent call last): File "ICCLIMscriptPI.py", line 37, in <module> icclim.indice(in_files=ncfile[0], var_name=var_name, out_file=out_file, indice_name=indice_name, slice_mode='DJF', callback = None) File "/home/scussoli/.conda/envs/test_env/lib/python2.7/site-packages/icclim-4.2.11-py2.7.egg/icclim/icclim.py", line 523, in indice save_percentile=save_percentile) ## tuple: (dt_centroid_arr, dt_bounds_arr, indice_arr) File "/home/scussoli/.conda/envs/test_env/lib/python2.7/site-packages/icclim-4.2.11-py2.7.egg/icclim/icclim.py", line 712, in get_indice_from_dict_temporal_slices any_slice = [temp_slice for temp_slice in t_slices][0] IndexError: list index out of range
Here is the function (but it is in /home/scussoli/.local/lib/python2.7/site-packages/icclim-4.2.11/icclim/util, not under the icclim-4.2.11-py2.7.egg directory, if that matters at all):
`def get_list_dates(ifile, type_dates):
'''
Returns list of dates from one file.
:param ifile: NetCDF file
:type ifile: str
:param type_dates: type of dates ('dt' for datetime objects, 'num' for float objects)
:type type_dates: str
:rtype: list of datetime/float
'''
try:
nc = Dataset(ifile, 'r')
except RuntimeError:
raise MissingIcclimInputError("Failed to access dataset: " + ifile)
var_time = nc.variables['time']
time_units = var_time.units # str (ex.: 'days since 1850-01-01 00:00:00')
try:
time_calend = var_time.calendar # str (ex.: 'standard'/'gregorian'/...)
except:
time_calend = 'gregorian'
if type_dates == 'num':
arr_dt = var_time[:]
list_dt = arr_dt.tolist() # numpy array -> list
if type_dates == 'dt':
t = netcdftime.utime(time_units, time_calend)
arr_dt = t.num2date(var_time[:])
list_dt = arr_dt.tolist() # numpy array -> list
del arr_dt
nc.close()
return list_dt`
Ok I see. The problem comes from the slice_mode DJF, the slice_mode year run successfully. I'll have a look tomorrow on that and should be able to solve the issue.
Your icclim version on your /.local/lib is out of date. If you work in a research center you might ask to the admin to update the icclim version. If this is your personal computer, you might want to upgrade it as well.
I let you know how it goes tomorrow
Thanks Xavier!
Strange that my icclim is out of date, as I just cloned the master branch and reinstalled this morning. But maybe I am doing something wrong.
I'll wait to hear from you ;)
Hi Paulo,
Really sorry for the delay. I've checked everything and the problem comes from your netcdf file. The 3 years time frame in your dataset have all the same value, i.e all the year are equal to 1. It implies that icclim can't know which month december should be related to the next year.
Do you think you could recreate your netcdf file with incremented year value in the time vector ? I tried with other dataset and it works
Hi Xavier,
thanks for finding out! I had not noticed that the year keeps repeating in that dataset, and it makes perfect sense that that's the problem. I will do as you suggest and apply fix the years. I'll report in case it still doesn't work, and also if it does ;)
Thank you again for your time and effort.