jankrepl/deepdow

Set callbacks to be based on validation loss (not training loss)

turmeric-blend opened this issue · 1 comments

This is my current setup for a run, EarlyStoppingCallback and ModelCheckpointCallback seems to be based on training loss. How to switch for callbacks to be based on validation loss?

losses = {'MeanReturn': MeanReturns(),
	  'CumulativeReturn': CumulativeReturn(),
	  'SharpeRatio': SharpeRatio(), 
	  'SortinoRatio': SortinoRatio(),}

run = Run(model,
          losses['SharpeRatio'],
          dataloader_train,
          val_dataloaders={'train': dataloader_train,
                           'valid': dataloader_valid},
          metrics = {'MeanReturn': losses['MeanReturn'], 
                     'CumulativeReturn': losses['CumulativeReturn'], 
          	     'SharpeRatio': losses['SharpeRatio'], 
          	     'SortinoRatio': losses['SortinoRatio']},
          optimizer=optimizer,
          callbacks=[EarlyStoppingCallback(dataloader_name='valid', 
			                   metric_name='loss',
					   patience=patience), 
	             ModelCheckpointCallback(folder_path=saved_model_folder, 
					     dataloader_name='valid', 
					     metric_name='loss'), 
		     TensorBoardCallback(log_dir=tensorboard_path, 
					 log_benchmarks=True)],
	  device=device)

I realised I read tensorboard wrongly