Unable to find conditions in hyperparameter.py after an abrupt end and restart of keras tuner
Closed this issue · 2 comments
When keras tuner is ended with either a restart of the computer or a kill comand, it is unable to reload/restart due to trying to load the trial.json of an incomplete model build.
To Reproduce
Start a keras tuner process, kill it, then restart keras tuner. Should reproduce the error.
Expected behavior
Skip the incomplete model test.
Additional context
If trial = trial_module.Trial.load(fname) in oracle.py had the fname iterate over the finished models in oracle.json instead of filenames, it would only do so with completed model test.
Would you like to help us fix it?
This happened when the folder name for the trial used the hash instead of the id e.g. instead of trial_00000001, trial_85a8388ceb4f3626d8bc6cc985269376 was the folder name of the unfinished trial. In a more recent run, an unfinished trial used the former, so I am unsure if this will be reproducable.
My error. A previous earlier version of keras tuner must have saved in the model folder before upgrading.