ChunML/NLP

How to prevent overfitting the model without validation?

Opened this issue · 0 comments

Hi,
I am trying to use your model on my own dataset, however my main issue is that I am not sure how the model is being validated during training, and for that purpose, how do I recognise at which point the training needs to be terminated? But how to then do early stopping, for example with a portion of training data (raw_data) in your case? Can you show this in code for your model please? Thanks

%%time
print("Model without Attention ")
NUM_EPOCHS = 200
BATCH_SIZE = 32

starttime = time.time()

for e in range(NUM_EPOCHS):
    en_initial_states = encoder.init_states(BATCH_SIZE)

    for batch, (source_seq, target_seq_in, target_seq_out) in enumerate(dataset.take(-1)):
        loss = train_step(source_seq, target_seq_in,
                          target_seq_out, en_initial_states)

        if batch % 100 == 0:
            print('Epoch {} Batch {} Loss {:.4f} Elapsed time {:.2f}s'.format(
                e + 1, batch, loss.numpy(), time.time() - starttime))
            starttime = time.time()
# How to stop training at the right point when model is not overfitting/underfitting.
    try:
        predict()
    except Exception:              
      continue