The audio dataset urbansound8k test is abnormal

Question

The audio dataset urbansound8k test is abnormal

yichenwang231 opened this issue 9 months ago · 4 comments

Bug

###When I tested UrbanSound8k, I set up 8 epochs and 4 epochs, respectively, and they were the best models stored in the fourth epoch according to the validation set, the losses and cf_mat of the first four epochs are exactly the same,but the results of loading the optimal model at the end of the run are different for the test set .

The logs are as follows：

8epoch；
Semi-supervised-learning-main$ python train.py --c config/usb_audio/supervised/supervised_urbansound8k_100_0.yaml
train.py:185: UserWarning: You have chosen to seed training. This will turn on the CUDNN deterministic setting, which can slow down your training considerably! You may see unexpected behavior when restarting from checkpoints.
warnings.warn('You have chosen to seed training. '
[2023-10-22 14:25:27,612 INFO] Use GPU: None for training
/media/ubuntu20/D/wyc/fuxian/Semi-supervised-learning-main/semilearn/datasets/utils.py:38: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
data, targets = np.array(data), np.array(targets)
[2023-10-22 14:25:29,227 INFO] unlabeled data number: 7079, labeled data number 100
[2023-10-22 14:25:29,227 INFO] Create train and test data loaders
[2023-10-22 14:26:09,317 INFO] [!] data loader keys: dict_keys(['train_lb', 'train_ulb', 'eval', 'test'])
[2023-10-22 14:26:29,943 INFO] Create optimizer and scheduler
[2023-10-22 14:26:29,945 INFO] Number of Trainable Params: 94969994
[2023-10-22 14:26:31,640 INFO] Arguments: Namespace(algorithm='supervised', amp=False, batch_size=8, c='config/usb_audio/supervised/supervised_urbansound8k_100_0.yaml', clip=0.0, clip_grad=0, crop_ratio=0.875, data_dir='./data', dataset='urbansound8k', dist_backend='nccl', dist_url='tcp://127.0.0.1:29980', distributed=False, ema_m=0.0, epoch=8, eval_batch_size=16, gpu=None, imb_algorithm=None, img_size=32, include_lb_to_ulb=True, layer_decay=0.75, lb_dest_len=100, lb_imb_ratio=1, load_path='./saved_models/usb_audio/supervised_urbansound8k_100_0/latest_model.pth', lr=5e-05, max_length=512, max_length_seconds=4.0, momentum=0.9, multiprocessing_distributed=False, net='hubert_base', net_from_name=False, num_classes=10, num_eval_iter=2048, num_labels=100, num_log_iter=256, num_train_iter=8192, num_warmup_iter=5120, num_workers=4, optim='AdamW', overwrite=True, pretrain_path='', rank=0, resume=False, sample_rate=16000, save_dir='./saved_models/usb_audio', save_name='supervised_urbansound8k_100_0', seed=0, train_sampler='RandomSampler', ulb_dest_len=7079, ulb_imb_ratio=1, ulb_loss_ratio=1.0, ulb_num_labels=None, uratio=1, use_aim=False, use_cat=False, use_pretrain=False, use_tensorboard=True, use_wandb=False, weight_decay=2e-05, world_size=1)
[2023-10-22 14:26:31,641 INFO] Resume load path ./saved_models/usb_audio/supervised_urbansound8k_100_0/latest_model.pth does not exist
[2023-10-22 14:26:31,641 INFO] Model training
[2023-10-22 14:26:48,589 INFO] 256 iteration USE_EMA: False, train/sup_loss: 2.2898, train/run_time: 0.0554, lr: 0.0000, train/prefecth_time: 0.0014
[2023-10-22 14:27:05,295 INFO] 512 iteration USE_EMA: False, train/sup_loss: 2.2459, train/run_time: 0.0535, lr: 0.0000, train/prefecth_time: 0.0012
[2023-10-22 14:27:21,938 INFO] 768 iteration USE_EMA: False, train/sup_loss: 2.0217, train/run_time: 0.0554, lr: 0.0000, train/prefecth_time: 0.0011
[2023-10-22 14:27:38,534 INFO] 1024 iteration USE_EMA: False, train/sup_loss: 1.6001, train/run_time: 0.0537, lr: 0.0000, train/prefecth_time: 0.0010
[2023-10-22 14:27:55,515 INFO] 1280 iteration USE_EMA: False, train/sup_loss: 1.1751, train/run_time: 0.0539, lr: 0.0000, train/prefecth_time: 0.0014
[2023-10-22 14:28:12,193 INFO] 1536 iteration USE_EMA: False, train/sup_loss: 0.6881, train/run_time: 0.0555, lr: 0.0000, train/prefecth_time: 0.0011
[2023-10-22 14:28:28,843 INFO] 1792 iteration USE_EMA: False, train/sup_loss: 0.4191, train/run_time: 0.0575, lr: 0.0000, train/prefecth_time: 0.0012
[2023-10-22 14:28:45,480 INFO] validating...
[2023-10-22 14:28:47,310 INFO] confusion matrix:
[[0.19 0. 0.11 0.07 0.06 0.16
0. 0.35 0. 0.06 ]
[0.15625 0.53125 0. 0.125 0.125 0.
0. 0.0625 0. 0. ]
[0. 0.04 0.59 0.09 0.06 0.
0. 0.01 0.12 0.09 ]
[0.02 0.1 0.1 0.41 0.02 0.04
0.13 0.08 0.01 0.09 ]
[0.12 0.05 0. 0. 0.37 0.06
0.11 0.25 0.02 0.02 ]
[0.26966292 0. 0.01123596 0.03370787 0. 0.24719101
0. 0.42696629 0.01123596 0. ]
[0. 0.12903226 0. 0.06451613 0. 0.
0.74193548 0.06451613 0. 0. ]
[0.06097561 0. 0. 0. 0.01219512 0.06097561
0. 0.86585366 0. 0. ]
[0. 0.02439024 0. 0. 0.01219512 0.
0. 0.03658537 0.92682927 0. ]
[0. 0.06 0.18 0.01 0.02 0.
0. 0. 0.12 0.61 ]]
[2023-10-22 14:28:48,248 INFO] model saved: ./saved_models/usb_audio/supervised_urbansound8k_100_0/latest_model.pth
[2023-10-22 14:28:49,165 INFO] model saved: ./saved_models/usb_audio/supervised_urbansound8k_100_0/model_best.pth
[2023-10-22 14:28:49,165 INFO] 2048 iteration, USE_EMA: False, train/sup_loss: 0.3801, train/run_time: 0.0553, eval/loss: 1.9100, eval/top-1-acc: 0.5221, eval/balanced_acc: 0.5483, eval/precision: 0.5193, eval/recall: 0.5483, eval/F1: 0.5103, lr: 0.0000, train/prefecth_time: 0.0010 BEST_EVAL_ACC: 0.5221, at 2048 iters
[2023-10-22 14:29:06,224 INFO] 2304 iteration USE_EMA: False, train/sup_loss: 0.0385, train/run_time: 0.0575, lr: 0.0000, train/prefecth_time: 0.0011
[2023-10-22 14:29:22,958 INFO] 2560 iteration USE_EMA: False, train/sup_loss: 1.6380, train/run_time: 0.0516, lr: 0.0000, train/prefecth_time: 0.0016
[2023-10-22 14:29:39,666 INFO] 2816 iteration USE_EMA: False, train/sup_loss: 0.0323, train/run_time: 0.0556, lr: 0.0000, train/prefecth_time: 0.0012
[2023-10-22 14:29:56,411 INFO] 3072 iteration USE_EMA: False, train/sup_loss: 0.6749, train/run_time: 0.0534, lr: 0.0000, train/prefecth_time: 0.0010
[2023-10-22 14:30:13,461 INFO] 3328 iteration USE_EMA: False, train/sup_loss: 0.0032, train/run_time: 0.0578, lr: 0.0000, train/prefecth_time: 0.0012
[2023-10-22 14:30:30,235 INFO] 3584 iteration USE_EMA: False, train/sup_loss: 0.0210, train/run_time: 0.0556, lr: 0.0000, train/prefecth_time: 0.0011
[2023-10-22 14:30:46,976 INFO] 3840 iteration USE_EMA: False, train/sup_loss: 0.1609, train/run_time: 0.0557, lr: 0.0000, train/prefecth_time: 0.0009
[2023-10-22 14:31:03,653 INFO] validating...
[2023-10-22 14:31:05,480 INFO] confusion matrix:
[[0.19 0. 0.06 0.05 0.11 0.15
0. 0.36 0. 0.08 ]
[0. 0.90625 0. 0. 0.03125 0.03125
0. 0.03125 0. 0. ]
[0. 0.01 0.53 0.14 0.03 0.
0. 0.01 0.12 0.16 ]
[0.04 0.1 0.07 0.41 0.02 0.03
0.17 0.03 0.03 0.1 ]
[0.04 0.06 0. 0. 0.44 0.11
0.06 0.19 0.07 0.03 ]
[0.03370787 0. 0. 0.07865169 0.02247191 0.47191011
0. 0.38202247 0.01123596 0. ]
[0. 0.06451613 0. 0.06451613 0. 0.
0.74193548 0.12903226 0. 0. ]
[0.08536585 0. 0. 0. 0. 0.15853659
0. 0.75609756 0. 0. ]
[0. 0.03658537 0. 0. 0. 0.01219512
0. 0. 0.95121951 0. ]
[0. 0.07 0.12 0. 0. 0.
0. 0. 0.09 0.72 ]]
[2023-10-22 14:31:12,474 INFO] model saved: ./saved_models/usb_audio/supervised_urbansound8k_100_0/latest_model.pth
[2023-10-22 14:31:19,461 INFO] model saved: ./saved_models/usb_audio/supervised_urbansound8k_100_0/model_best.pth
[2023-10-22 14:31:19,462 INFO] 4096 iteration, USE_EMA: False, train/sup_loss: 0.0226, train/run_time: 0.0535, eval/loss: 3.0230, eval/top-1-acc: 0.5674, eval/balanced_acc: 0.6117, eval/precision: 0.5731, eval/recall: 0.6117, eval/F1: 0.5634, lr: 0.0000, train/prefecth_time: 0.0011 BEST_EVAL_ACC: 0.5674, at 4096 iters
[2023-10-22 14:31:36,450 INFO] 4352 iteration USE_EMA: False, train/sup_loss: 0.0025, train/run_time: 0.0576, lr: 0.0000, train/prefecth_time: 0.0011
[2023-10-22 14:31:53,131 INFO] 4608 iteration USE_EMA: False, train/sup_loss: 0.0046, train/run_time: 0.0557, lr: 0.0000, train/prefecth_time: 0.0011
[2023-10-22 14:32:09,836 INFO] 4864 iteration USE_EMA: False, train/sup_loss: 0.0062, train/run_time: 0.0558, lr: 0.0000, train/prefecth_time: 0.0012
[2023-10-22 14:32:26,536 INFO] 5120 iteration USE_EMA: False, train/sup_loss: 0.0009, train/run_time: 0.0559, lr: 0.0001, train/prefecth_time: 0.0014
[2023-10-22 14:32:43,623 INFO] 5376 iteration USE_EMA: False, train/sup_loss: 0.0009, train/run_time: 0.0557, lr: 0.0000, train/prefecth_time: 0.0011
[2023-10-22 14:33:00,300 INFO] 5632 iteration USE_EMA: False, train/sup_loss: 0.0010, train/run_time: 0.0535, lr: 0.0000, train/prefecth_time: 0.0012
[2023-10-22 14:33:17,000 INFO] 5888 iteration USE_EMA: False, train/sup_loss: 0.0055, train/run_time: 0.0557, lr: 0.0000, train/prefecth_time: 0.0012
[2023-10-22 14:33:33,726 INFO] validating...
[2023-10-22 14:33:35,554 INFO] confusion matrix:
[[0.43 0.25 0.09 0.01 0.05 0.05
0. 0. 0.01 0.11 ]
[0. 0.875 0. 0. 0.03125 0.03125
0.03125 0.03125 0. 0. ]
[0. 0.04 0.54 0.08 0. 0.
0. 0. 0.19 0.15 ]
[0.05 0.17 0.08 0.38 0.02 0.02
0.15 0.04 0.03 0.06 ]
[0.13 0.12 0.02 0. 0.24 0.06
0.06 0.27 0.03 0.07 ]
[0.04494382 0.3258427 0.01123596 0.12359551 0. 0.47191011
0. 0. 0.02247191 0. ]
[0. 0.06451613 0. 0.03225806 0. 0.
0.74193548 0.16129032 0. 0. ]
[0.06097561 0. 0. 0. 0. 0.14634146
0. 0.79268293 0. 0. ]
[0. 0.02439024 0. 0. 0. 0.
0. 0.02439024 0.95121951 0. ]
[0. 0.15 0.08 0. 0. 0.
0. 0. 0.18 0.59 ]]
[2023-10-22 14:33:42,682 INFO] model saved: ./saved_models/usb_audio/supervised_urbansound8k_100_0/latest_model.pth
[2023-10-22 14:33:42,683 INFO] 6144 iteration, USE_EMA: False, train/sup_loss: 0.2796, train/run_time: 0.0555, eval/loss: 3.8913, eval/top-1-acc: 0.5564, eval/balanced_acc: 0.6013, eval/precision: 0.5861, eval/recall: 0.6013, eval/F1: 0.5471, lr: 0.0000, train/prefecth_time: 0.0012 BEST_EVAL_ACC: 0.5674, at 4096 iters
[2023-10-22 14:33:59,804 INFO] 6400 iteration USE_EMA: False, train/sup_loss: 0.0002, train/run_time: 0.0557, lr: 0.0000, train/prefecth_time: 0.0012
[2023-10-22 14:34:16,741 INFO] 6656 iteration USE_EMA: False, train/sup_loss: 0.0004, train/run_time: 0.0557, lr: 0.0000, train/prefecth_time: 0.0011
[2023-10-22 14:34:33,427 INFO] 6912 iteration USE_EMA: False, train/sup_loss: 2.9305, train/run_time: 0.0495, lr: 0.0000, train/prefecth_time: 0.0012
[2023-10-22 14:34:50,156 INFO] 7168 iteration USE_EMA: False, train/sup_loss: 0.0003, train/run_time: 0.0535, lr: 0.0000, train/prefecth_time: 0.0010
[2023-10-22 14:35:07,255 INFO] 7424 iteration USE_EMA: False, train/sup_loss: 0.0005, train/run_time: 0.0558, lr: 0.0000, train/prefecth_time: 0.0010
[2023-10-22 14:35:23,970 INFO] 7680 iteration USE_EMA: False, train/sup_loss: 0.0018, train/run_time: 0.0536, lr: 0.0000, train/prefecth_time: 0.0014
[2023-10-22 14:35:40,720 INFO] 7936 iteration USE_EMA: False, train/sup_loss: 0.0002, train/run_time: 0.0577, lr: 0.0000, train/prefecth_time: 0.0014
[2023-10-22 14:35:57,467 INFO] validating...
[2023-10-22 14:35:59,296 INFO] confusion matrix:
[[0.19 0.01 0.04 0.01 0.13 0.26
0. 0.02 0. 0.34 ]
[0.03125 0.875 0.03125 0. 0.03125 0.03125
0. 0. 0. 0. ]
[0. 0.01 0.6 0.08 0.02 0.
0. 0.02 0.14 0.13 ]
[0.06 0.12 0.1 0.36 0.07 0.05
0.12 0.03 0.02 0.07 ]
[0.04 0.02 0. 0. 0.49 0.04
0.04 0.29 0.01 0.07 ]
[0.02247191 0.20224719 0. 0.14606742 0. 0.48314607
0. 0.12359551 0.02247191 0. ]
[0. 0.09677419 0. 0.06451613 0. 0.
0.70967742 0.12903226 0. 0. ]
[0.07317073 0. 0. 0. 0.01219512 0.09756098
0. 0.81707317 0. 0. ]
[0. 0.02439024 0.01219512 0. 0. 0.01219512
0. 0. 0.95121951 0. ]
[0. 0.09 0.18 0. 0.01 0.
0. 0. 0.15 0.57 ]]
[2023-10-22 14:36:06,133 INFO] model saved: ./saved_models/usb_audio/supervised_urbansound8k_100_0/latest_model.pth
[2023-10-22 14:36:06,134 INFO] 8192 iteration, USE_EMA: False, train/sup_loss: 0.0001, train/run_time: 0.0575, eval/loss: 3.6727, eval/top-1-acc: 0.5625, eval/balanced_acc: 0.6046, eval/precision: 0.5584, eval/recall: 0.6046, eval/F1: 0.5546, lr: 0.0000, train/prefecth_time: 0.0010 BEST_EVAL_ACC: 0.5674, at 4096 iters
[2023-10-22 14:36:13,459 INFO] model saved: ./saved_models/usb_audio/supervised_urbansound8k_100_0/latest_model.pth
[2023-10-22 14:36:14,138 INFO] Model loaded
[2023-10-22 14:36:16,045 INFO] confusion matrix:
[[0.71 0. 0.07 0.02 0.03 0.12
0. 0.01 0. 0.04 ]
[0.03030303 0.48484848 0.15151515 0. 0.03030303 0.
0. 0. 0.09090909 0.21212121]
[0. 0.01 0.54 0.1 0.05 0.
0. 0. 0.14 0.16 ]
[0.01 0.14 0.07 0.44 0.15 0.
0.05 0.03 0.04 0.07 ]
[0.11 0.21 0.02 0.07 0.37 0.1
0.01 0.01 0.04 0.06 ]
[0.05376344 0.01075269 0. 0.59139785 0.05376344 0.02150538
0. 0.24731183 0.02150538 0. ]
[0. 0.03125 0. 0.03125 0. 0.
0.90625 0.03125 0. 0. ]
[0. 0.01041667 0. 0. 0. 0.10416667
0. 0.875 0. 0.01041667]
[0.02409639 0.04819277 0.27710843 0. 0.02409639 0.
0. 0.02409639 0.59036145 0.01204819]
[0. 0.02 0.14 0. 0.05 0.
0. 0. 0.13 0.66 ]]
[2023-10-22 14:36:16,046 INFO] Model result - eval/best_acc : 0.5674019607843137
[2023-10-22 14:36:16,046 INFO] Model result - eval/best_it : 4095
[2023-10-22 14:36:16,046 INFO] Model result - test/best_acc : 0.5400238948626045
[2023-10-22 14:36:16,046 WARNING] GPU 0 training is FINISHED

4epoch；
Semi-supervised-learning-main$ python train.py --c config/usb_audio/supervised/supervised_urbansound8k_100_0.yaml
train.py:185: UserWarning: You have chosen to seed training. This will turn on the CUDNN deterministic setting, which can slow down your training considerably! You may see unexpected behavior when restarting from checkpoints.
warnings.warn('You have chosen to seed training. '
[2023-10-22 14:36:31,817 INFO] Use GPU: None for training
/media/ubuntu20/D/wyc/fuxian/Semi-supervised-learning-main/semilearn/datasets/utils.py:38: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
data, targets = np.array(data), np.array(targets)
[2023-10-22 14:36:33,448 INFO] unlabeled data number: 7079, labeled data number 100
[2023-10-22 14:36:33,448 INFO] Create train and test data loaders
[2023-10-22 14:37:13,542 INFO] [!] data loader keys: dict_keys(['train_lb', 'train_ulb', 'eval', 'test'])
[2023-10-22 14:37:34,191 INFO] Create optimizer and scheduler
[2023-10-22 14:37:34,193 INFO] Number of Trainable Params: 94969994
[2023-10-22 14:37:35,883 INFO] Arguments: Namespace(algorithm='supervised', amp=False, batch_size=8, c='config/usb_audio/supervised/supervised_urbansound8k_100_0.yaml', clip=0.0, clip_grad=0, crop_ratio=0.875, data_dir='./data', dataset='urbansound8k', dist_backend='nccl', dist_url='tcp://127.0.0.1:22474', distributed=False, ema_m=0.0, epoch=4, eval_batch_size=16, gpu=None, imb_algorithm=None, img_size=32, include_lb_to_ulb=True, layer_decay=0.75, lb_dest_len=100, lb_imb_ratio=1, load_path='./saved_models/usb_audio/supervised_urbansound8k_100_0/latest_model.pth', lr=5e-05, max_length=512, max_length_seconds=4.0, momentum=0.9, multiprocessing_distributed=False, net='hubert_base', net_from_name=False, num_classes=10, num_eval_iter=2048, num_labels=100, num_log_iter=256, num_train_iter=4096, num_warmup_iter=5120, num_workers=4, optim='AdamW', overwrite=True, pretrain_path='', rank=0, resume=False, sample_rate=16000, save_dir='./saved_models/usb_audio', save_name='supervised_urbansound8k_100_0', seed=0, train_sampler='RandomSampler', ulb_dest_len=7079, ulb_imb_ratio=1, ulb_loss_ratio=1.0, ulb_num_labels=None, uratio=1, use_aim=False, use_cat=False, use_pretrain=False, use_tensorboard=True, use_wandb=False, weight_decay=2e-05, world_size=1)
[2023-10-22 14:37:35,883 INFO] Resume load path ./saved_models/usb_audio/supervised_urbansound8k_100_0/latest_model.pth does not exist
[2023-10-22 14:37:35,883 INFO] Model training
[2023-10-22 14:37:52,823 INFO] 256 iteration USE_EMA: False, train/sup_loss: 2.2898, train/run_time: 0.0554, lr: 0.0000, train/prefecth_time: 0.0010
[2023-10-22 14:38:09,517 INFO] 512 iteration USE_EMA: False, train/sup_loss: 2.2459, train/run_time: 0.0534, lr: 0.0000, train/prefecth_time: 0.0010
[2023-10-22 14:38:26,149 INFO] 768 iteration USE_EMA: False, train/sup_loss: 2.0217, train/run_time: 0.0554, lr: 0.0000, train/prefecth_time: 0.0011
[2023-10-22 14:38:42,741 INFO] 1024 iteration USE_EMA: False, train/sup_loss: 1.6001, train/run_time: 0.0532, lr: 0.0000, train/prefecth_time: 0.0010
[2023-10-22 14:38:59,719 INFO] 1280 iteration USE_EMA: False, train/sup_loss: 1.1751, train/run_time: 0.0536, lr: 0.0000, train/prefecth_time: 0.0008
[2023-10-22 14:39:16,388 INFO] 1536 iteration USE_EMA: False, train/sup_loss: 0.6881, train/run_time: 0.0555, lr: 0.0000, train/prefecth_time: 0.0012
[2023-10-22 14:39:33,051 INFO] 1792 iteration USE_EMA: False, train/sup_loss: 0.4191, train/run_time: 0.0574, lr: 0.0000, train/prefecth_time: 0.0012
[2023-10-22 14:39:49,708 INFO] validating...
[2023-10-22 14:39:51,540 INFO] confusion matrix:
[[0.19 0. 0.11 0.07 0.06 0.16
0. 0.35 0. 0.06 ]
[0.15625 0.53125 0. 0.125 0.125 0.
0. 0.0625 0. 0. ]
[0. 0.04 0.59 0.09 0.06 0.
0. 0.01 0.12 0.09 ]
[0.02 0.1 0.1 0.41 0.02 0.04
0.13 0.08 0.01 0.09 ]
[0.12 0.05 0. 0. 0.37 0.06
0.11 0.25 0.02 0.02 ]
[0.26966292 0. 0.01123596 0.03370787 0. 0.24719101
0. 0.42696629 0.01123596 0. ]
[0. 0.12903226 0. 0.06451613 0. 0.
0.74193548 0.06451613 0. 0. ]
[0.06097561 0. 0. 0. 0.01219512 0.06097561
0. 0.86585366 0. 0. ]
[0. 0.02439024 0. 0. 0.01219512 0.
0. 0.03658537 0.92682927 0. ]
[0. 0.06 0.18 0.01 0.02 0.
0. 0. 0.12 0.61 ]]
[2023-10-22 14:39:52,458 INFO] model saved: ./saved_models/usb_audio/supervised_urbansound8k_100_0/latest_model.pth
[2023-10-22 14:39:53,347 INFO] model saved: ./saved_models/usb_audio/supervised_urbansound8k_100_0/model_best.pth
[2023-10-22 14:39:53,348 INFO] 2048 iteration, USE_EMA: False, train/sup_loss: 0.3801, train/run_time: 0.0552, eval/loss: 1.9100, eval/top-1-acc: 0.5221, eval/balanced_acc: 0.5483, eval/precision: 0.5193, eval/recall: 0.5483, eval/F1: 0.5103, lr: 0.0000, train/prefecth_time: 0.0010 BEST_EVAL_ACC: 0.5221, at 2048 iters
[2023-10-22 14:40:10,441 INFO] 2304 iteration USE_EMA: False, train/sup_loss: 0.0385, train/run_time: 0.0578, lr: 0.0000, train/prefecth_time: 0.0012
[2023-10-22 14:40:27,202 INFO] 2560 iteration USE_EMA: False, train/sup_loss: 1.6380, train/run_time: 0.0515, lr: 0.0000, train/prefecth_time: 0.0012
[2023-10-22 14:40:43,915 INFO] 2816 iteration USE_EMA: False, train/sup_loss: 0.0323, train/run_time: 0.0556, lr: 0.0000, train/prefecth_time: 0.0011
[2023-10-22 14:41:00,652 INFO] 3072 iteration USE_EMA: False, train/sup_loss: 0.6749, train/run_time: 0.0534, lr: 0.0000, train/prefecth_time: 0.0010
[2023-10-22 14:41:17,699 INFO] 3328 iteration USE_EMA: False, train/sup_loss: 0.0032, train/run_time: 0.0577, lr: 0.0000, train/prefecth_time: 0.0009
[2023-10-22 14:41:34,457 INFO] 3584 iteration USE_EMA: False, train/sup_loss: 0.0210, train/run_time: 0.0556, lr: 0.0000, train/prefecth_time: 0.0013
[2023-10-22 14:41:51,176 INFO] 3840 iteration USE_EMA: False, train/sup_loss: 0.1609, train/run_time: 0.0556, lr: 0.0000, train/prefecth_time: 0.0012
[2023-10-22 14:42:07,815 INFO] validating...
[2023-10-22 14:42:09,646 INFO] confusion matrix:
[[0.19 0. 0.06 0.05 0.11 0.15
0. 0.36 0. 0.08 ]
[0. 0.90625 0. 0. 0.03125 0.03125
0. 0.03125 0. 0. ]
[0. 0.01 0.53 0.14 0.03 0.
0. 0.01 0.12 0.16 ]
[0.04 0.1 0.07 0.41 0.02 0.03
0.17 0.03 0.03 0.1 ]
[0.04 0.06 0. 0. 0.44 0.11
0.06 0.19 0.07 0.03 ]
[0.03370787 0. 0. 0.07865169 0.02247191 0.47191011
0. 0.38202247 0.01123596 0. ]
[0. 0.06451613 0. 0.06451613 0. 0.
0.74193548 0.12903226 0. 0. ]
[0.08536585 0. 0. 0. 0. 0.15853659
0. 0.75609756 0. 0. ]
[0. 0.03658537 0. 0. 0. 0.01219512
0. 0. 0.95121951 0. ]
[0. 0.07 0.12 0. 0. 0.
0. 0. 0.09 0.72 ]]
[2023-10-22 14:42:16,593 INFO] model saved: ./saved_models/usb_audio/supervised_urbansound8k_100_0/latest_model.pth
[2023-10-22 14:42:23,699 INFO] model saved: ./saved_models/usb_audio/supervised_urbansound8k_100_0/model_best.pth
[2023-10-22 14:42:23,700 INFO] 4096 iteration, USE_EMA: False, train/sup_loss: 0.0226, train/run_time: 0.0534, eval/loss: 3.0230, eval/top-1-acc: 0.5674, eval/balanced_acc: 0.6117, eval/precision: 0.5731, eval/recall: 0.6117, eval/F1: 0.5634, lr: 0.0000, train/prefecth_time: 0.0010 BEST_EVAL_ACC: 0.5674, at 4096 iters
[2023-10-22 14:42:30,586 INFO] model saved: ./saved_models/usb_audio/supervised_urbansound8k_100_0/latest_model.pth
[2023-10-22 14:42:31,242 INFO] Model loaded
[2023-10-22 14:42:33,139 INFO] confusion matrix:
[[0.7 0. 0.02 0. 0.05 0.1
0. 0. 0.01 0.12 ]
[0. 0.45454545 0.24242424 0. 0.03030303 0.
0.03030303 0. 0. 0.24242424]
[0. 0. 0.42 0.14 0.03 0.
0. 0. 0.12 0.29 ]
[0.01 0.14 0.07 0.5 0.05 0.
0.11 0.03 0.03 0.06 ]
[0.03 0.17 0.01 0.07 0.37 0.18
0.04 0.06 0.02 0.05 ]
[0.07526882 0. 0. 0.5483871 0.07526882 0.07526882
0.01075269 0.21505376 0. 0. ]
[0. 0.03125 0. 0. 0. 0.
0.9375 0.03125 0. 0. ]
[0. 0.01041667 0. 0. 0.01041667 0.23958333
0.02083333 0.71875 0. 0. ]
[0.01204819 0.03614458 0.3253012 0. 0.02409639 0.
0.01204819 0.02409639 0.55421687 0.01204819]
[0. 0.01 0.1 0. 0.07 0.
0. 0.01 0.1 0.71 ]]
[2023-10-22 14:42:33,140 INFO] Model result - eval/best_acc : 0.5674019607843137
[2023-10-22 14:42:33,141 INFO] Model result - eval/best_it : 4095
[2023-10-22 14:42:33,141 INFO] Model result - test/best_acc : 0.5221027479091995
[2023-10-22 14:42:33,141 WARNING] GPU 0 training is FINISHED

Answer 1 · 2023-10-23T21:16:45.000Z

This might be normal for fully supervised training on 100 labels of urbansound, where the model is learning nothing.

Answer 2 · 2023-10-24T07:43:14.000Z

I compared the classifier weights of the model when epoch was set to 8.The classifier weights I tested on the validation set at the end of the fourth epoch were different from the classifier weights tested on the test set when the model loaded into the fourth epoch after training was completed.

Answer 3 · 2023-11-24T01:47:44.000Z

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

Answer 4 · 2023-11-29T01:48:24.000Z

This issue was closed because it has been stalled for 5 days with no activity.