dgilford/tcpyPI

Can we change file for "run_sample.py" from a pickle file to a more portable format?

aramirezreyes opened this issue · 4 comments

aramirezreyes commented

When running run_sample.py on python 3.7 with:

netcdf4                   1.5.8                    pypi_0    pypi
numba                     0.55.1                   pypi_0    pypi
numpy                     1.21.6                   pypi_0    pypi
xarray                    0.20.2                   pypi_0    pypi

...PI computation complete and saved

Performing PI analyses...
Traceback (most recent call last):
  File "run_sample.py", line 148, in <module>
    ds2 = run_sample_analyses(ds,_mdrF,CKCD=0.9)
  File "run_sample.py", line 73, in run_sample_analyses
    basins = pickle.load( open( _mdrF, "rb" ) )
AttributeError: Can't get attribute 'PandasIndexAdapter' on <module 'xarray.core.indexing' from '/Users/arreyes/miniconda3/envs/mytcpypi/lib/python3.7/site-packages/xarray/core/indexing.py'>

The error occurs because the pickled file was saved with an unknown version of pickle and they are not supposed to be compatible (see e.g. pydata/xarray#5642) a better Idea could be to redistribute the file as netcdf or hdf5.

dgilford commented

This is a good idea. I'll need to create the file that avoids compatibility issues as you suggest. Please stay tuned.

aramirezreyes commented

Actually now that I notice, mdr.pk1 is not used anywhere as far as I can see. It is loaded in run_sample.py but its contents are never used. That line could be removed. If there is some functionality for the basins dataset we can use netCDF (I could contribute a PR if you think this is a good idea.).

Wamashudu commented

HEY PLEASE HELP I AM ENCOUNTERING THE SAME PROBLEM

Wamashudu commented

@aramirezreyes DID YOU SOLVE IT