lanpa/tensorboardX

How can I connect previous log file into new one in tensorboard

mohanades opened this issue · 1 comments

Hi,
I trained my model for 50 epochs. Now I want to continue training for other 15 epochs. so I start training with model.load_from_checkpoint("path")
but I don't know how to show continuation of logs with tensorboard
I thought if I write same path of previous file logs, it continue but it didn't show me the previous logs.

I'd be appreciate if you could help me because I need to see the previous training logs along with the new ones.
Part of source code is:

checkpoint_callback=ModelCheckpoint(dirpath="model",filename="newmodel_1",save_last=True,verbose=True,monitor="val_ce_loss",mode="min")
logger=TensorBoardLogger("training-logs",name="test_1")
model1 = model(config)
model_with_previous = model1.load_from_checkpoint(base_path_ckpt)
trainer = pl.Trainer(
     max_steps = max_steps,
     max_epochs=N_EPOCHS,
     gpus=(1 if torch.cuda.is_available() else 0),
     logger=logger,
     callbacks=[checkpoint_callback], 
)
%load_ext tensorboard
%tensorboard --logdir ./lightning_logs
lanpa commented

Hi, you might need to pass the purge_step parameter.
Here is an low level API example:
https://github.com/lanpa/tensorboardX/blob/master/examples/demo_purge.py