keras-team/keras-tuner

tuner.get_best_models() issues warning Detecting that an object or model or tf.train.Checkpoint is being deleted with unrestored values

Opened this issue · 1 comments

Describe the bug
Calling tuner.get_best_models() at the end of hp tuning issues several warnings due to missing checkpoints.

WARNING:tensorflow:Detecting that an object or model or tf.train.Checkpoint is being deleted with unrestored values. See the following logs for the specific values in question. To silence these warnings, use status.expect_partial(). See https://www.tensorflow.org/api_docs/python/tf/train/Checkpoint#restorefor details about the status object returned by the restore function.
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).layer_with_weights-0.kernel
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).layer_with_weights-0.bias
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).layer_with_weights-1.kernel
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).layer_with_weights-1.bias
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer._variables.1
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer._variables.2
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer._variables.3
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer._variables.4
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer._variables.5
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer._variables.6
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer._variables.7
WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer._variables.8

To Reproduce
The official example in the documentation shows this problem.
It can be reproduced in the related Colab Notebook.

Expected behavior
No warnings should be issued.

Would you like to help us fix it?
I do not know the codebase enough.

I might not be your case, however I have fixed this error by removing my custom-defined metric (in my case f1score) from the model used in build_model(). It seems it was not saved properly/not implemented properly for this specific (tuner) case.