--save_params / --load_params options not available

[JulianGiles]

When trying to use the --save_params / --load_params options they are not available, I get the following error:

usage: process_climate_indices [-h] --index
                               {spi,spei,pnp,scaled,pet,palmers,all}
                               --periodicity {monthly,daily}
                               [--scales [SCALES [SCALES ...]]]
                               [--calibration_start_year CALIBRATION_START_YEAR]
                               [--calibration_end_year CALIBRATION_END_YEAR]
                               [--netcdf_precip NETCDF_PRECIP]
                               [--var_name_precip VAR_NAME_PRECIP]
                               [--netcdf_temp NETCDF_TEMP]
                               [--var_name_temp VAR_NAME_TEMP]
                               [--netcdf_pet NETCDF_PET]
                               [--var_name_pet VAR_NAME_PET]
                               [--netcdf_awc NETCDF_AWC]
                               [--var_name_awc VAR_NAME_AWC]
                               --output_file_base OUTPUT_FILE_BASE
                               [--multiprocessing {single,all_but_one,all}]
process_climate_indices: error: unrecognized arguments: --save_params

[monocongo]

It looks like this feature is included in the __spi__.py script.

Hopefully, this may at least steer you in the right direction until we can get this code rolled into the main processing script (or we can modify the documentation to reflect the actual state of affairs, probably a simpler solution for now).

[monocongo]

Possibly related to #447

[JulianGiles]

I have coded the --save_params / --load_params options into __main__.py. Now these options are working for SPI and SPEI and they take a base file name to save or load a file of fitting parameters for each index (SPI or SPEI), distribution (gamma or pearson) and scale combination. For example, the code:

process_climate_indices --index scaled --periodicity monthly --scales 12 --netcdf_precip pre_1982-2012_monsum.nc --var_name_precip var62 --netcdf_pet pet_thornthwaite.nc --var_name_pet pet_thornthwaite --output_file_base INDEX --save_params INDEX2

Generates the following files:

INDEX2_fitting_params_gamma_spei_12.nc
INDEX2_fitting_params_gamma_spi_12.nc
INDEX2_fitting_params_pearson_spei_12.nc
INDEX2_fitting_params_pearson_spi_12.nc
INDEX_pnp_12.nc
INDEX_spei_gamma_12.nc
INDEX_spei_pearson_12.nc
INDEX_spi_gamma_12.nc
INDEX_spi_pearson_12.nc

Then, you can run another calculation calling the option --load_params INDEX2 and it will look for the files with the fitting parameters that start with INDEX2 and use them to fit the new indices.

LIMITATIONS:

• I only managed to implement the options for the case of data in the form of arrays, and I did not mess with the case of US Climate Divisions.
• I am not an expert coder and the original code is complex. I did my best but I am sure that this is not the most clean nor efficient implementation. That being said, it is working and it may serve as a guide for someone willing to polish the code.

You can check it out here:
https://github.com/JulianGiles/climate_indices/tree/issue_439_save_params

[monocongo]

@JulianGiles This is excellent work, thanks a lot for this contribution!

I think generalizing this to also cover non-gridded datasets such as climate divisions won't be too much extra work, but that's easy for me to say now, let's see.

[xyliu6666]

Hi, I am using climate-indices 1.0.9 and facing the same problem. I also tried to install the package from https://github.com/JulianGiles/climate_indices but the problem remained. The error information I get is the same, "process_climate_indices: error: unrecognized arguments: --save_params ../results/". This package is brilliant but I really need to store the parameters and use for another dataset. Hope anyone can help! Thanks!

[JulianGiles]

Hi @xyliu6666, you have to install the "issue_439_save_params" branch of my forked code, and not the main branch. Could you check that and try again? it should work.

[xyliu6666]

@JulianGiles Thank you for your response! I tried but it resulted in: "fatal: repository 'https://github.com/JulianGiles/climate_indices/tree/issue_439_save_params/' not found". It is a strange error if my command is correct: "pip install -e git+https://github.com/JulianGiles/climate_indices/tree/issue_439_save_params#egg=climate-indices"

[JulianGiles]

@xyliu6666 I just tried your command and it worked for me. Then I tried again and it did not. I do not understand what is happening but maybe just keep trying.

You can also try:
pip install -e git+https://github.com/JulianGiles/climate_indices@issue_439_save_params#egg=climate-indices

(it has an @ instead of /tree/)

[xyliu6666]

@JulianGiles Sure, thank you!