prepare_sector_network.py: pandas.errors.IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integer
Opened this issue · 5 comments
Describe the Bug
I tried to run
python scripts/prepare_sector_network.py
and got the error below
Error Message
(pypsa-eur) projekt-resilient03@ubuntu-22-04-lts-temp:~/pypsa-eur$ python scripts/prepare_sector_network.py
Output_
/home/projekt-resilient03/pypsa-eur/results/prenetworks/elec_s_37_lv1.0__730H-T-H-B-I-A-dist1_2050.nc
INFO:pypsa.io:Imported network elec_s_37_ec_lv1.0_.nc has buses, carriers, generators, global_constraints, lines, links, loads, shapes, storage_units, stores
INFO:__main__:Removing buses with carrier ['H2', 'battery']
INFO:__main__:Removing links with carrier ['H2 electrolysis', 'H2 fuel cell', 'battery charger', 'battery discharger']
INFO:__main__:Removing generators with carrier ['CCGT', 'OCGT', 'biomass', 'coal', 'geothermal', 'lignite', 'oil']
INFO:__main__:Removing stores with carrier ['H2', 'battery']
INFO:__main__:Added connection cost of 8834-39750 Eur/MW/a to offwind-dc
INFO:__main__:Added connection cost of 3950-25835 Eur/MW/a to offwind-ac
INFO:__main__:Added connection cost of 5182-85173 Eur/MW/a to offwind-float
WARNING:pypsa.components:Bus has no attribute location, ignoring this passed value.
WARNING:pypsa.components:Bus has no attribute location, ignoring this passed value.
INFO:__main__:Adding electricity generation
INFO:__main__:Add hydrogen storage
INFO:__main__:Adding hydrogen fuel cell for re-electrification.
INFO:__main__:Add hydrogen underground storage
INFO:__main__:Add options for new hydrogen pipelines.
INFO:__main__:Add land transport
INFO:__main__:fuel_cell share: 0%
INFO:__main__:electric share: 100%
INFO:__main__:ice share: 0%
INFO:__main__:Add heat sector
INFO:__main__:Assumed space heat reduction of 29.00%
INFO:__main__:Add biomass
INFO:__main__:Add industrial demand
INFO:__main__:Add possibility to use industrial waste heat in district heating
INFO:__main__:Add agriculture, forestry and fishing sector.
ERROR:root:Uncaught exception
Traceback (most recent call last):
File "/home/projekt-resilient03/pypsa-eur/scripts/prepare_sector_network.py", line 4057, in <module>
n = set_temporal_aggregation(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/projekt-resilient03/pypsa-eur/scripts/prepare_sector_network.py", line 3662, in set_temporal_aggregation
.astype(int)
^^^^^^^^^^^
File "/home/projekt-resilient03/conda/envs/pypsa-eur/lib/python3.11/site-packages/pandas/core/generic.py", line 6643, in astype
new_data = self._mgr.astype(dtype=dtype, copy=copy, errors=errors)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/projekt-resilient03/conda/envs/pypsa-eur/lib/python3.11/site-packages/pandas/core/internals/managers.py", line 430, in astype
return self.apply(
^^^^^^^^^^^
File "/home/projekt-resilient03/conda/envs/pypsa-eur/lib/python3.11/site-packages/pandas/core/internals/managers.py", line 363, in apply
applied = getattr(b, f)(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/projekt-resilient03/conda/envs/pypsa-eur/lib/python3.11/site-packages/pandas/core/internals/blocks.py", line 758, in astype
new_values = astype_array_safe(values, dtype, copy=copy, errors=errors)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/projekt-resilient03/conda/envs/pypsa-eur/lib/python3.11/site-packages/pandas/core/dtypes/astype.py", line 237, in astype_array_safe
new_values = astype_array(values, dtype, copy=copy)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/projekt-resilient03/conda/envs/pypsa-eur/lib/python3.11/site-packages/pandas/core/dtypes/astype.py", line 182, in astype_array
values = _astype_nansafe(values, dtype, copy=copy)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/projekt-resilient03/conda/envs/pypsa-eur/lib/python3.11/site-packages/pandas/core/dtypes/astype.py", line 101, in _astype_nansafe
return _astype_float_to_int_nansafe(arr, dtype, copy)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/projekt-resilient03/conda/envs/pypsa-eur/lib/python3.11/site-packages/pandas/core/dtypes/astype.py", line 145, in _astype_float_to_int_nansafe
raise IntCastingNaNError(
pandas.errors.IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integer
Related:
#1118
I run into the same error once I use the nH trigger in the {sector_opts} wildcard. Is there any meaningful workaround or hint?
Might be related to the geopandas issue. (Did not try; just occured at the same time). => Ensure that you are using latest version or try geopandas==0.14.4.
Also see #1136
Thanks for mentioning! Unfortunately that didn't do the trick for me...
Some additional information: I looked a bit into the code. If I set e.g. 3h or 3H in the sector_opts
, in the set_temporal_aggregation() it finds that there is no 'sn' in the resolution (which is instead 3h) and then takes a csv file for the snapshot_weightings which in my case is an empty file which then leads to nan values causing the problem here.
I tried out a few quick and dirty workarounds. One, entering a '3sn' to the sector_opts
config seems to work... But I do not know if it leads to any other unwanted changes. E.g. I was wondering if this aggregation part would be missing in this case.
You can try to set the temporal resolution here:
pypsa-eur/config/config.default.yaml
Line 858 in d2e7562
replacing 'false' with 'nH'.