training auc is equal validation auc?
Opened this issue · 0 comments
yasong commented
in line 140 of the train.py
if (i % TEST_FREQ == 0):
auc, fpr, tpr, thres = get_auc_epoch(gnn, Gs_train, classes_train,
BATCH_SIZE, load_data=valid_epoch) #valid_epoch?
gnn.say("Testing model: training auc = {0} @ {1}".format(
auc, datetime.now()))
auc, fpr, tpr, thres = get_auc_epoch(gnn, Gs_dev, classes_dev,
BATCH_SIZE, load_data=valid_epoch)
gnn.say("Testing model: validation auc = {0} @ {1}".format(
auc, datetime.now()))
partial results:
Initial training auc = 0.8385795480339078 @ 2021-01-14 10:49:18.571989
Initial validation auc = 0.8385795480339078 @ 2021-01-14 10:49:26.453895
EPOCH 1/100, loss = 0.6696747210700994 @ 2021-01-14 10:52:33.539342
Testing model: training auc = 0.9225966099742847 @ 2021-01-14 10:52:41.815191
Testing model: validation auc = 0.9225966099742847 @ 2021-01-14 10:52:49.352049
Model saved in ./saved_model/graphnn-model_best
EPOCH 2/100, loss = 0.6475605093730598 @ 2021-01-14 10:55:58.523499
Testing model: training auc = 0.9278334338892803 @ 2021-01-14 10:56:06.467269
Testing model: validation auc = 0.9278334338892803 @ 2021-01-14 10:56:14.051002
so should I change it as the following?
if (i % TEST_FREQ == 0):
auc, fpr, tpr, thres = get_auc_epoch(gnn, Gs_train, classes_train,
BATCH_SIZE, load_data=None) #change valid_epoch into None
gnn.say("Testing model: training auc = {0} @ {1}".format(
auc, datetime.now()))
auc, fpr, tpr, thres = get_auc_epoch(gnn, Gs_dev, classes_dev,
BATCH_SIZE, load_data=valid_epoch)
gnn.say("Testing model: validation auc = {0} @ {1}".format(
auc, datetime.now()))