atomistic-machine-learning/schnetpack

Model Testing

Closed this issue · 4 comments

MasterLucas commented

Dear Sir/ Madam,

I have a question about testing a trained model. I know I can specify the number of examples I want to use for testing when creating the dataset using the spk.data.AtomsDataModule. However, suppose I am training a set of different models, for instance, to compare their performance with different training and validation splits. How could I ensure that the same examples are used for testing each model?

Yours faithfully,

Lucas Bandeira

jnsLs commented

Hi Lucas,

in spk.data.AtomsDataModule you can either specify the number of training and validation samples ('num_train' and 'num_val') or provide the path to a split file. The latter contains sample indices for the train, val and test set. The indices correspond to the respective sample indices in the dataset. In your case I would recommend specifying several split files with identical indices in the test set.

Best, Jonas

MasterLucas commented

Hi Jonas,

Thank you very much for your reply. What would be the most appropriate way to do this? Could I simply use np.savez to create the split files?

Yours sincerely,

Lucas Bandeira

jnsLs commented

Hi Lucas,
using np.savez is a valid way to do this :)
Best, Jonas

MasterLucas commented

Nice! Thank you very much, Jonas!