ArnauMiro/pyLowOrder

Adapting pyLOM.NN.Dataset to multiple input formats

Closed this issue · 0 comments

Explanation of the problem

When working with a pyLOM.Dataset, the fields, as explained on the wiki , may have multiple dimensions, each one corresponding to a variable. A problem arises with this format when combinations of these variables do not have an associated field value, which result in having many elements of the filed array with a value of zero.

To address this, a pyLOM.Dataset can be created with just 2 dimensions, where the second one corresponds to the valid combinations of the variables.

This issue is meant to properly adapt the pyLOM.NN.Dataset to this last format mentioned.

Additional enhancements

  • Adding a static method load_from_file to create a pyLOM.NN.Dataset directly from a pyLOM.Dataset.
  • Adding save and load method for pyLOM.NN.MinMaxScaler.
  • Adding method get_splits_by_parameters to split the dataset by parameters. This will be useful to split it by flight conditions.
  • Reduced the memory usage of the datasets. Noticeable for big datasets.
  • Added map method to apply a function, i.e. a scaling function, to a dataset. Changes are made in place.
  • Added filtermethod to filter a dataset by a condition given as a function.
  • Added remove_column method to remove not desired columns from a dataset.