google-research/weatherbench2

ERA5 IFS ensemble mean containing nans?

stratisMarkou opened this issue · 3 comments

stratisMarkou commented

Does the ERA5 IFS ensemble mean contain nans? This might just be incorrect usage on my side, but consider this example:

import numpy as np
import xarray as xr

# Set locations for IFS ensemble and IFS ensemble mean
IFS_ENS = "gs://weatherbench2/datasets/ens/2018-240x121_equiangular_with_poles_conservative.zarr" 
IFS_ENS_MEAN = "gs://weatherbench2/datasets/ens/2018-240x121_equiangular_with_poles_conservative_mean.zarr" 

# Load IFS ENS arrays
ifs_ens_array = xr.open_zarr(IFS_ENS)
ifs_ens_mean_array = xr.open_zarr(IFS_ENS_MEAN)

The IFS ensemble array seems okay, but the IFS ensemble mean array seems to contain nans:

# Running this returns `False`
np.isnan(ifs_ens_array["2m_temperature"][0, :, :, :, :].values).any()

# returns
# False


# but running this
ifs_ens_mean_array["2m_temperature"][0, 0, :, :].values

# returns
# array([[nan, nan, nan, ..., nan, nan, nan],
#        [nan, nan, nan, ..., nan, nan, nan],
#        [nan, nan, nan, ..., nan, nan, nan],
#        ...,
#        [nan, nan, nan, ..., nan, nan, nan],
#        [nan, nan, nan, ..., nan, nan, nan],
#        [nan, nan, nan, ..., nan, nan, nan]], dtype=float32)
chloepilonv commented

I obtain the same results - and "True" when I run np.isnan(ifs_ens_mean_array["2m_temperature"][0, 0, :, :].values).any().

chloepilonv commented

I ran other "mean" files, from other years and different resolution, and this one seems like the only one with "NaN" (but I might be wrong!).

raspstephan commented

Hi,
thanks for pointing this out. I made a mistake computing the 2018 ensemble means. I am in the process of updating the ensemble files (they might not be available for a day or two). After that, the corrected 2018-2022 ensemble files will be online. I will close this issue once the upload is done. Thanks for your patience.

  • 👍1