Broadcast bug in noise_generation for 1D case
Closed this issue · 4 comments
When passing 1D level1 and noise datasets to scene_generation.noise_generation we recieve the following error:
ValueError Traceback (most recent call last)
Cell In[4], line 33
10 print('RSV noise =',noise_RSV)
11 noise = xr.Dataset(
12 data_vars=dict(
13 RSV=(['along','Antenna'],
(...)
31 ),
32 )
---> 33 level1n = ss.performance.scene_generation.noise_generation(level1, noise)
34 L2_RSV_noise[ii] = ss.retrieval.level2.wind_current_retrieval(level1n, noise, gmf, ambiguity)
35 L2_RSV_noise = xr.concat(L2_RSV_noise.values(), dim='RSV_noise')
File D:\data\SEASTAR\code\seastar_project\seastar\performance\scene_generation.py:208, in noise_generation(truth, noise)
204 level1 = noise.drop_vars(noise.data_vars)
206 rng = np.random.default_rng()
207 level1['Sigma0'] = truth.Sigma0
--> 208 + noise.Sigma0 * rng.standard_normal(size=truth.Sigma0.shape) # Draw samples from a standard Normal distribution (mean=0, stdev=1).
209 level1['RSV'] = truth.RSV
210 + noise.RSV * rng.standard_normal(size=truth.RSV.shape)
212 level1.RSV.attrs['long_name'] = 'Radial Surface Velocity'
File ~\anaconda3\envs\seastar\lib\site-packages\xarray\core_typed_ops.py:212, in DataArrayOpsMixin.mul(self, other)
211 def mul(self, other):
--> 212 return self._binary_op(other, operator.mul)
File ~\anaconda3\envs\seastar\lib\site-packages\xarray\core\dataarray.py:4357, in DataArray._binary_op(self, other, f, reflexive)
4353 other_variable = getattr(other, "variable", other)
4354 other_coords = getattr(other, "coords", None)
4356 variable = (
-> 4357 f(self.variable, other_variable)
4358 if not reflexive
4359 else f(other_variable, self.variable)
4360 )
4361 coords, indexes = self.coords._merge_raw(other_coords, reflexive)
4362 name = self._result_name(other)
File ~\anaconda3\envs\seastar\lib\site-packages\xarray\core_typed_ops.py:402, in VariableOpsMixin.mul(self, other)
401 def mul(self, other):
--> 402 return self._binary_op(other, operator.mul)
File ~\anaconda3\envs\seastar\lib\site-packages\xarray\core\variable.py:2639, in Variable._binary_op(self, other, f, reflexive)
2636 attrs = self._attrs if keep_attrs else None
2637 with np.errstate(all="ignore"):
2638 new_data = (
-> 2639 f(self_data, other_data) if not reflexive else f(other_data, self_data)
2640 )
2641 result = Variable(dims, new_data, attrs=attrs)
2642 return result
ValueError: operands could not be broadcast together with shapes (20,3) (3,20)
This bug also appears in 2D and can be traced to the shape of truth.Sigma0 in scene_generation.
The dataset truth has the correct size (e.g., along, across, Antenna), however the variables (Sigma0 and RSV) have shapes corresponding to Antenna, along, across. This results in a broadcast error when adding the noise on line 208 of the function
This is further traced to line 96 and:
truth['Sigma0'] = seastar.gmfs.nrcs.compute_nrcs(truth, geo, gmf['nrcs'])
The output of seastar.gmfs.nrcs.compute_nrcs() has the order of its dimensions in the output changed relative to its input
In seastar.gmfs.nrcs.compute_nrcs() line 66 there is:
nrcs = nrcs.to_array(dim='Antenna'). As nrcs is a list of DataArrays it would be better to use xr.concat() to preserve the order of dataset dimensions
Closing as bug was due to incorrect use of function in processing chain