mila-iqia/blocks

EarlyStopping does nothing with default triggers

Opened this issue · 0 comments

If no triggers are handed as parameters to the EarlyStopping extension, it won't actually save the best model. From what I can tell, this is because its TrackTheBest subextension is by default called "after_epoch", but Checkpoint with the additional condition of the "best_so_far" log entry being present is called "after_batch". This means that it will e.g. be called after the last batch of the epoch, which is before any "after_epoch" extensions (including TrackTheBest()) are run, meaning that the log entry for this iteration step isn't actually present when Checkpoint is called.
I would propose that the condition added to the Checkpoint subextension should be "after_batch" or "after_epoch" depending on whether the "iterations" or "epochs" argument is not None. As it is now, EarlyStopping also doesn't play nice with DataStreamMonitoring's "after_epoch" default for similar reasons; instead I need to call the Monitoring "after_n_batches" with a batch number equal to the number of batches per epoch, which is really awkward.