ClimateImpactLab/downscaleCMIP6

HadGEM3-GC31-LL e2e tasmax fails at cleaning step for all ssps

emileten opened this issue · 2 comments

This might be an easy one, as there is only one type of error. Blocks progress on #546.

Workflow : https://argo.cildc6.org/archived-workflows/default/a7a809e4-0856-498d-ac53-12cd9f8b064a

Example of log -- all ssps have the same problem :

INFO:dodola.services:Starting dodola service clean_cmip6 with args=('gs://scratch-170cd6ec/a7a809e4-0856-498d-ac53-12cd9f8b064a/e2e-hadgem3-gc31-ll-tasmax-7ctsw-2787356673/timesliced.zarr', 'gs://scratch-170cd6ec/a7a809e4-0856-498d-ac53-12cd9f8b064a/e2e-hadgem3-gc31-ll-tasmax-7ctsw-1459604069/standardized.zarr', True), kwargs={})
INFO:dodola.repository:Read gs://scratch-170cd6ec/a7a809e4-0856-498d-ac53-12cd9f8b064a/e2e-hadgem3-gc31-ll-tasmax-7ctsw-2787356673/timesliced.zarr
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/xarray/coding/times.py", line 460, in encode_cf_datetime
    raise OutOfBoundsDatetime
pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/bin/dodola", line 33, in <module>
    sys.exit(load_entry_point('dodola', 'console_scripts', 'dodola')())
  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/opt/dodola/dodola/cli.py", line 477, in cleancmip6
    services.clean_cmip6(x, out, drop_leapdays)
  File "/opt/dodola/dodola/services.py", line 34, in service_logger
    func(*args, **kwargs)
  File "/opt/dodola/dodola/services.py", line 601, in clean_cmip6
    storage.write(out, cleaned_ds)
  File "/opt/dodola/dodola/repository.py", line 71, in write
    x.to_zarr(url_or_path, mode="w", compute=True)
  File "/opt/conda/lib/python3.9/site-packages/xarray/core/dataset.py", line 2031, in to_zarr
    return to_zarr(
  File "/opt/conda/lib/python3.9/site-packages/xarray/backends/api.py", line 1414, in to_zarr
    dump_to_store(dataset, zstore, writer, encoding=encoding)
  File "/opt/conda/lib/python3.9/site-packages/xarray/backends/api.py", line 1124, in dump_to_store
    store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)
  File "/opt/conda/lib/python3.9/site-packages/xarray/backends/zarr.py", line 521, in store
    variables_encoded, attributes = self.encode(
  File "/opt/conda/lib/python3.9/site-packages/xarray/backends/common.py", line 201, in encode
    variables = {k: self.encode_variable(v) for k, v in variables.items()}
  File "/opt/conda/lib/python3.9/site-packages/xarray/backends/common.py", line 201, in <dictcomp>
    variables = {k: self.encode_variable(v) for k, v in variables.items()}
  File "/opt/conda/lib/python3.9/site-packages/xarray/backends/zarr.py", line 480, in encode_variable
    variable = encode_zarr_variable(variable)
  File "/opt/conda/lib/python3.9/site-packages/xarray/backends/zarr.py", line 276, in encode_zarr_variable
    var = conventions.encode_cf_variable(var, name=name)
  File "/opt/conda/lib/python3.9/site-packages/xarray/conventions.py", line 269, in encode_cf_variable
    var = coder.encode(var, name=name)
  File "/opt/conda/lib/python3.9/site-packages/xarray/coding/times.py", line 513, in encode
    (data, units, calendar) = encode_cf_datetime(
  File "/opt/conda/lib/python3.9/site-packages/xarray/coding/times.py", line 487, in encode_cf_datetime
    num = _encode_datetime_with_cftime(dates, units, calendar)
  File "/opt/conda/lib/python3.9/site-packages/xarray/coding/times.py", line 426, in _encode_datetime_with_cftime
    return np.array([encode_datetime(d) for d in dates.ravel()]).reshape(dates.shape)
  File "/opt/conda/lib/python3.9/site-packages/xarray/coding/times.py", line 426, in <listcomp>
    return np.array([encode_datetime(d) for d in dates.ravel()]).reshape(dates.shape)
  File "/opt/conda/lib/python3.9/site-packages/xarray/coding/times.py", line 424, in encode_datetime
    return np.nan if d is None else cftime.date2num(d, units, calendar)
  File "src/cftime/_cftime.pyx", line 257, in cftime._cftime.date2num
  File "src/cftime/_cftime.pyx", line 351, in cftime._cftime.to_calendar_specific_datetime
AttributeError: 'numpy.float64' object has no attribute 'year'

I see why that fails and we have a fix -- same situation as in #548.

That particular bug was fixed.