Feature Data Normalization
grantbuster opened this issue · 2 comments
The SpatialBatchHandler should (by default, norm=True) return normalized data and have an option (norm=False) to return data in the physical units. Typically we scale to a mean value of 0 and stdev of 1 for each feature. We will need to calculate the mean+stdev of each feature across all DataHandler objects and then save this data in a property so that the GAN has access to it.
We also need to be able to handle data with pre-defined mean/stdev values when we're making predictions using a trained model with pre-determined mean/stdev values. I'm not sure whats the best way to go about this. Maybe have DataHandler accept a mean/stdev value to normalize but then when being used by the SpatialBatchHandler with multiple DataHandler objects we only do the normalization on the SpatialBatchHandler level? I'm not sure how we'll handle data during production data generation, happy to brainstorm.
Still need the ability to normalize on a different mean/stdev for the case of transfer learning
closed in #14