mllam/neural-lam

Variable names and weights as part of static dataset files

Closed this issue · 1 comments

Currently the variables in the dataset are listed in constants.py. This is bad if the code is to be used with other datasets.

Proposition

Create a file variables.json in data/my_dataset/static that describe all variables. This includes:

  • Weather state variables (e.g. u_65)
  • Forcing variables for the full grid
  • Batch-static forcing variables (static during one forecast, but changing throughout the dataset. i.e. open water currently)

All of these should be listed in order with names. For the weather state variables, their weighting (as in parameter_weights.npy currently) should also be listed with them. We can then remove the lines https://github.com/joeloskarsson/neural-lam/blob/89a4c63370201c9ea1a5f04d4cf1e5e75b7cc83e/create_parameter_weights.py#L26-L31 that generate this weighting file. It is better to let this be something that is set manually when preparing a dataset.

Such a variables.json file could then be loaded into a VariableDescription object and used in the models. The variable dimensions https://github.com/joeloskarsson/neural-lam/blob/89a4c63370201c9ea1a5f04d4cf1e5e75b7cc83e/neural_lam/models/ar_model.py#L22-L24 should then be read from this object rather than hard-coded in a model definition.

Superseded by #23