MITgcm/xmitgcm

Conflicting sizes when reading in ASTE diagnostic output

pittwolfe opened this issue · 3 comments

pittwolfe commented

I'm trying to read the budg2d_hflux_set2 (containing vertically integrated horizontal fluxes) MDS diagnostics from a recent run of ASTE, but open_mdsdataset throws the error ValueError: conflicting sizes for dimension 'face': length 6 on 'UVELMASS' and length 13 on 'face'. Is this because xmitgcm thinks that UVELMASS should be a 3D field or something else?

I'm using the 30x30x242 tile layout with blank tiles, so 2D files end up having dimensions 90x4050. The full invocation is

ds = open_mdsdataset(os.path.join(aste_dir, 'mds_diags'), grid_dir=os.path.join(aste_dir, 'mds_grid'), 
                     delta_t=1200, ref_date='2002-1-1 00:00:00', geometry='llc',
                     extra_metadata={'has_faces': True,
                                     'ny': 1350,
                                     'nx': 270,
                                     'ny_facets': [450, 0, 270, 180, 450],
                                     'pad_before_y': [90, 0, 0, 0, 0],
                                     'pad_after_y': [0, 0, 0, 90, 90],
                                     'face_facets': [0, 0, 2, 3, 4, 4],
                                     'facet_orders': ['C', 'C', 'C', 'F', 'F'],
                                     'face_offsets': [0, 1, 0, 0, 0, 1],
                                     'transpose_face': [False, False, False, True, True, True]})

Not sure if the extra metadata is correct. I'm using what I get from xmitgcm.utils.get_extra_metadata(domain='aste', nx=270).

The full backtrace is

ValueError                                Traceback (most recent call last)
<ipython-input-34-79126a79253a> in <module>
      1 ds = open_mdsdataset(os.path.join(aste_dir, 'mds_diags'), grid_dir=os.path.join(aste_dir, 'mds_grid'), 
      2                      delta_t=1200, ref_date='2002-1-1 00:00:00', geometry='llc',
----> 3                     extra_metadata=aste_extra_metadata)

~/miniconda3/envs/py3/lib/python3.7/site-packages/xmitgcm-0.4.1+17.gf5ee774-py3.7.egg/xmitgcm/mds_store.py in open_mdsdataset(data_dir, grid_dir, iters, prefix, read_grid, delta_t, ref_date, calendar, levels, geometry, grid_vars_to_coords, swap_dims, endian, chunks, ignore_unknown_vars, default_dtype, nx, ny, nz, llc_method, extra_metadata)
    217                 datasets = [open_mdsdataset(
    218                         data_dir, iters=iternum, read_grid=False, **kwargs)
--> 219                     for iternum in iters]
    220                 # now add the grid
    221                 if read_grid:

~/miniconda3/envs/py3/lib/python3.7/site-packages/xmitgcm-0.4.1+17.gf5ee774-py3.7.egg/xmitgcm/mds_store.py in <listcomp>(.0)
    217                 datasets = [open_mdsdataset(
    218                         data_dir, iters=iternum, read_grid=False, **kwargs)
--> 219                     for iternum in iters]
    220                 # now add the grid
    221                 if read_grid:

~/miniconda3/envs/py3/lib/python3.7/site-packages/xmitgcm-0.4.1+17.gf5ee774-py3.7.egg/xmitgcm/mds_store.py in open_mdsdataset(data_dir, grid_dir, iters, prefix, read_grid, delta_t, ref_date, calendar, levels, geometry, grid_vars_to_coords, swap_dims, endian, chunks, ignore_unknown_vars, default_dtype, nx, ny, nz, llc_method, extra_metadata)
    254                           nx=nx, ny=ny, nz=nz, llc_method=llc_method,
    255                           levels=levels, extra_metadata=extra_metadata)
--> 256     ds = xr.Dataset.load_store(store)
    257 
    258     if swap_dims:

~/miniconda3/envs/py3/lib/python3.7/site-packages/xarray/core/dataset.py in load_store(cls, store, decoder)
    561         if decoder:
    562             variables, attributes = decoder(variables, attributes)
--> 563         obj = cls(variables, attrs=attributes)
    564         obj._file_obj = store
    565         return obj

~/miniconda3/envs/py3/lib/python3.7/site-packages/xarray/core/dataset.py in __init__(self, data_vars, coords, attrs)
    542 
    543         variables, coord_names, dims, indexes, _ = merge_data_and_coords(
--> 544             data_vars, coords, compat="broadcast_equals"
    545         )
    546 

~/miniconda3/envs/py3/lib/python3.7/site-packages/xarray/core/merge.py in merge_data_and_coords(data, coords, compat, join)
    465     indexes = dict(_extract_indexes_from_coords(coords))
    466     return merge_core(
--> 467         objects, compat, join, explicit_coords=explicit_coords, indexes=indexes
    468     )
    469 

~/miniconda3/envs/py3/lib/python3.7/site-packages/xarray/core/merge.py in merge_core(objects, compat, join, combine_attrs, priority_arg, explicit_coords, indexes, fill_value)
    598     assert_unique_multiindex_level_names(variables)
    599 
--> 600     dims = calculate_dimensions(variables)
    601 
    602     coord_names, noncoord_names = determine_coords(coerced)

~/miniconda3/envs/py3/lib/python3.7/site-packages/xarray/core/dataset.py in calculate_dimensions(variables)
    207                     "conflicting sizes for dimension %r: "
    208                     "length %s on %r and length %s on %r"
--> 209                     % (dim, size, k, dims[dim], last_used[dim])
    210                 )
    211     return dims

ValueError: conflicting sizes for dimension 'face': length 6 on 'UVELMASS' and length 13 on 'face'

Thanks!

pittwolfe commented

Turns out that adding nx = 270 to the call to open_mdsdataset fixes the issue. I figured I didn't need to specify nx since it's already in extra_metadata.

raphaeldussin commented

@pittwolfe I was gonna tell you the exact same thing. It's unfortunately a redundancy in the API that I could not fix. But since we're talking about it, let me point you to some examples using ASTE here: https://github.com/raphaeldussin/MITgcm-recipes/tree/master/examples

pittwolfe commented