HadGEM3-GC31-LL e2e tasmax fails at cleaning step for all ssps
emileten opened this issue · 2 comments
emileten commented
This might be an easy one, as there is only one type of error. Blocks progress on #546.
Workflow : https://argo.cildc6.org/archived-workflows/default/a7a809e4-0856-498d-ac53-12cd9f8b064a
Example of log -- all ssps have the same problem :
INFO:dodola.services:Starting dodola service clean_cmip6 with args=('gs://scratch-170cd6ec/a7a809e4-0856-498d-ac53-12cd9f8b064a/e2e-hadgem3-gc31-ll-tasmax-7ctsw-2787356673/timesliced.zarr', 'gs://scratch-170cd6ec/a7a809e4-0856-498d-ac53-12cd9f8b064a/e2e-hadgem3-gc31-ll-tasmax-7ctsw-1459604069/standardized.zarr', True), kwargs={})
INFO:dodola.repository:Read gs://scratch-170cd6ec/a7a809e4-0856-498d-ac53-12cd9f8b064a/e2e-hadgem3-gc31-ll-tasmax-7ctsw-2787356673/timesliced.zarr
Traceback (most recent call last):
File "/opt/conda/lib/python3.9/site-packages/xarray/coding/times.py", line 460, in encode_cf_datetime
raise OutOfBoundsDatetime
pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/bin/dodola", line 33, in <module>
sys.exit(load_entry_point('dodola', 'console_scripts', 'dodola')())
File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
return self.main(*args, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1062, in main
rv = self.invoke(ctx)
File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1668, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 763, in invoke
return __callback(*args, **kwargs)
File "/opt/dodola/dodola/cli.py", line 477, in cleancmip6
services.clean_cmip6(x, out, drop_leapdays)
File "/opt/dodola/dodola/services.py", line 34, in service_logger
func(*args, **kwargs)
File "/opt/dodola/dodola/services.py", line 601, in clean_cmip6
storage.write(out, cleaned_ds)
File "/opt/dodola/dodola/repository.py", line 71, in write
x.to_zarr(url_or_path, mode="w", compute=True)
File "/opt/conda/lib/python3.9/site-packages/xarray/core/dataset.py", line 2031, in to_zarr
return to_zarr(
File "/opt/conda/lib/python3.9/site-packages/xarray/backends/api.py", line 1414, in to_zarr
dump_to_store(dataset, zstore, writer, encoding=encoding)
File "/opt/conda/lib/python3.9/site-packages/xarray/backends/api.py", line 1124, in dump_to_store
store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)
File "/opt/conda/lib/python3.9/site-packages/xarray/backends/zarr.py", line 521, in store
variables_encoded, attributes = self.encode(
File "/opt/conda/lib/python3.9/site-packages/xarray/backends/common.py", line 201, in encode
variables = {k: self.encode_variable(v) for k, v in variables.items()}
File "/opt/conda/lib/python3.9/site-packages/xarray/backends/common.py", line 201, in <dictcomp>
variables = {k: self.encode_variable(v) for k, v in variables.items()}
File "/opt/conda/lib/python3.9/site-packages/xarray/backends/zarr.py", line 480, in encode_variable
variable = encode_zarr_variable(variable)
File "/opt/conda/lib/python3.9/site-packages/xarray/backends/zarr.py", line 276, in encode_zarr_variable
var = conventions.encode_cf_variable(var, name=name)
File "/opt/conda/lib/python3.9/site-packages/xarray/conventions.py", line 269, in encode_cf_variable
var = coder.encode(var, name=name)
File "/opt/conda/lib/python3.9/site-packages/xarray/coding/times.py", line 513, in encode
(data, units, calendar) = encode_cf_datetime(
File "/opt/conda/lib/python3.9/site-packages/xarray/coding/times.py", line 487, in encode_cf_datetime
num = _encode_datetime_with_cftime(dates, units, calendar)
File "/opt/conda/lib/python3.9/site-packages/xarray/coding/times.py", line 426, in _encode_datetime_with_cftime
return np.array([encode_datetime(d) for d in dates.ravel()]).reshape(dates.shape)
File "/opt/conda/lib/python3.9/site-packages/xarray/coding/times.py", line 426, in <listcomp>
return np.array([encode_datetime(d) for d in dates.ravel()]).reshape(dates.shape)
File "/opt/conda/lib/python3.9/site-packages/xarray/coding/times.py", line 424, in encode_datetime
return np.nan if d is None else cftime.date2num(d, units, calendar)
File "src/cftime/_cftime.pyx", line 257, in cftime._cftime.date2num
File "src/cftime/_cftime.pyx", line 351, in cftime._cftime.to_calendar_specific_datetime
AttributeError: 'numpy.float64' object has no attribute 'year'
emileten commented
That particular bug was fixed.