reduce precision to float32?

[aaronspring]

the benchmark is in float32, the forecasts are in float64.
I did a rough check judithberner/climpred_CESM1_S2S#4 and https://renkulab.io/gitlab/aaron.spring/s2s-ai-competition-bootstrap/-/blob/master/notebooks/verification_RPSS-precision.ipynb.

I think we can massively save computing time and cloud storage if we go to float32.

for v in varlist_obs:
    for d in [dates[0]]:
        for dsl in obs_dataset_labels:
            # only netcdf, no choice
            try:
                if 'input' in dsl:
                    ds = cml.load_dataset(f"s2s-ai-challenge-{dsl}", origin='ecmwf', date=d, parameter=v).to_xarray()
                elif 'bench' in dsl:
                    ds = cml.load_dataset(f"s2s-ai-challenge-{dsl}", parameter=v, weeks='34').to_xarray()
                else:
                    ds = cml.load_dataset(f"s2s-ai-challenge-{dsl}", date=d, parameter=v).to_xarray()
                for v in ds.data_vars:
                    print(dsl,v,ds[v].dtype)
            except Exception as e:
                print(dsl,v, type(e).__name__,e)

training-output-reference tp float64
test-output-reference tp float64
training-input tp FileNotFoundError [Errno 2] No such file or directory: '/opt/conda/lib/python3.8/site-packages/climetlab_s2s_ai_challenge/training_input.yaml'
test-input tp FileNotFoundError [Errno 2] No such file or directory: '/opt/conda/lib/python3.8/site-packages/climetlab_s2s_ai_challenge/test_input.yaml'
test-output-benchmark tp float32
training-output-reference t2m float64
test-output-reference t2m float64
training-input t2m FileNotFoundError [Errno 2] No such file or directory: '/opt/conda/lib/python3.8/site-packages/climetlab_s2s_ai_challenge/training_input.yaml'
test-input t2m FileNotFoundError [Errno 2] No such file or directory: '/opt/conda/lib/python3.8/site-packages/climetlab_s2s_ai_challenge/test_input.yaml'
test-output-benchmark t2m float32


ds = cml.load_dataset(f"s2s-ai-challenge-training-output-reference", date=d, parameter=v).to_xarray()
print(ds[v].nbytes, ds[v].astype('float32').nbytes)
213734400 106867200

Thoughts @floriankrb @b8raoult ?

[aaronspring]

hi @floriankrb, see the gist here: https://gist.github.com/aaronspring/5182caf232a5d103b07322cf0bdfd191 Is this the test/demo you asked for? I would be just about the fore/hindcast-like-observations. ds=ds.astype('float32') is all we need in the script IMO. Am I missing something?