Can't reproduce the result of PEMS03_96_96 task
Opened this issue · 6 comments
Hi dear authors, thank you for your contribution in this article. However, now I have trouble in reproducing the result of PEMS03_96_96 task using the script provided in scripts/multivariate_forecasting/PEMS/iTransformer_03.sh.
I didn't change any code after downloading the repo and used the data you provide.
Here are my logs:
Args in experiment:
Namespace(activation='gelu', batch_size=32, c_out=358, channel_independence=False, checkpoints='./checkpoints/', class_strategy='projection', d_ff=512, d_layers=1, d_model=512, data='PEMS', data_path='PEMS03.npz', dec_in=358, des='Exp', devices='0,1,2,3', distil=True, do_predict=False, dropout=0.1, e_layers=4, efficient_training=False, embed='timeF', enc_in=358, exp_name='MTSF', factor=1, features='M', freq='h', gpu=0, inverse=False, is_training=1, itr=1, label_len=48, learning_rate=0.001, loss='MSE', lradj='type1', model='iTransformer', model_id='PEMS03_96_96', moving_avg=25, n_heads=8, num_workers=10, output_attention=False, partial_start_index=0, patience=3, pred_len=96, root_path='./dataset/PEMS/', seq_len=96, target='OT', target_data_path='electricity.csv', target_root_path='./data/electricity/', train_epochs=10, use_amp=False, use_gpu=True, use_multi_gpu=False, use_norm=True)
Use GPU: cuda:0
start training : PEMS03_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0>>>>>>>>>>>>>>>>>>>>>>>>>>
train 15533
val 5051
test 5051
iters: 100, epoch: 1 | loss: 0.6254231
speed: 0.0592s/iter; left time: 281.3028s
iters: 200, epoch: 1 | loss: 0.5153803
speed: 0.0428s/iter; left time: 198.9109s
iters: 300, epoch: 1 | loss: 0.5594893
speed: 0.0432s/iter; left time: 196.7037s
iters: 400, epoch: 1 | loss: 0.5180222
speed: 0.0436s/iter; left time: 193.9329s
Epoch: 1 cost time: 22.676926612854004
Epoch: 1, Steps: 485 | Train Loss: 0.6499067 Vali Loss: 2.3864691 Test Loss: 2.3295264
Validation loss decreased (inf --> 2.386469). Saving model ...
Updating learning rate to 0.001
iters: 100, epoch: 2 | loss: 2.3072736
speed: 0.5208s/iter; left time: 2221.7887s
iters: 200, epoch: 2 | loss: 1.6124172
speed: 0.0418s/iter; left time: 174.2366s
iters: 300, epoch: 2 | loss: 1.4063895
speed: 0.0422s/iter; left time: 171.6863s
iters: 400, epoch: 2 | loss: 1.6972579
speed: 0.0424s/iter; left time: 168.3158s
Epoch: 2 cost time: 21.366225242614746
Epoch: 2, Steps: 485 | Train Loss: 1.7878543 Vali Loss: 2.2568407 Test Loss: 2.1668501
Validation loss decreased (2.386469 --> 2.256841). Saving model ...
Updating learning rate to 0.0005
iters: 100, epoch: 3 | loss: 1.9239104
speed: 0.5242s/iter; left time: 1981.9011s
iters: 200, epoch: 3 | loss: 1.8586614
speed: 0.0418s/iter; left time: 153.7583s
iters: 300, epoch: 3 | loss: 2.1840055
speed: 0.0424s/iter; left time: 151.8389s
iters: 400, epoch: 3 | loss: 2.3375337
speed: 0.0426s/iter; left time: 148.3466s
Epoch: 3 cost time: 21.4084894657135
Epoch: 3, Steps: 485 | Train Loss: 1.9079526 Vali Loss: 2.0450311 Test Loss: 1.9737376
Validation loss decreased (2.256841 --> 2.045031). Saving model ...
Updating learning rate to 0.00025
iters: 100, epoch: 4 | loss: 1.9168335
speed: 0.5273s/iter; left time: 1737.9915s
iters: 200, epoch: 4 | loss: 1.7213866
speed: 0.0422s/iter; left time: 134.7857s
iters: 300, epoch: 4 | loss: 2.4566705
speed: 0.0425s/iter; left time: 131.6806s
iters: 400, epoch: 4 | loss: 1.9949968
speed: 0.0428s/iter; left time: 128.2878s
Epoch: 4 cost time: 21.496201038360596
Epoch: 4, Steps: 485 | Train Loss: 2.0230194 Vali Loss: 2.0335276 Test Loss: 1.9665655
Validation loss decreased (2.045031 --> 2.033528). Saving model ...
Updating learning rate to 0.000125
iters: 100, epoch: 5 | loss: 1.7979079
speed: 0.5207s/iter; left time: 1463.5948s
iters: 200, epoch: 5 | loss: 2.1647246
speed: 0.0422s/iter; left time: 114.4971s
iters: 300, epoch: 5 | loss: 2.3066046
speed: 0.0425s/iter; left time: 110.9998s
iters: 400, epoch: 5 | loss: 1.9180266
speed: 0.0428s/iter; left time: 107.3851s
Epoch: 5 cost time: 21.60309934616089
Epoch: 5, Steps: 485 | Train Loss: 2.0156756 Vali Loss: 2.0310166 Test Loss: 1.9661391
Validation loss decreased (2.033528 --> 2.031017). Saving model ...
Updating learning rate to 6.25e-05
iters: 100, epoch: 6 | loss: 1.8521897
speed: 0.5214s/iter; left time: 1212.6685s
iters: 200, epoch: 6 | loss: 1.7516365
speed: 0.0421s/iter; left time: 93.6896s
iters: 300, epoch: 6 | loss: 1.8441006
speed: 0.0424s/iter; left time: 90.1091s
iters: 400, epoch: 6 | loss: 1.7079446
speed: 0.0426s/iter; left time: 86.2513s
Epoch: 6 cost time: 21.40552520751953
Epoch: 6, Steps: 485 | Train Loss: 1.8906130 Vali Loss: 2.3260319 Test Loss: 2.2649562
EarlyStopping counter: 1 out of 3
Updating learning rate to 3.125e-05
iters: 100, epoch: 7 | loss: 1.6522958
speed: 0.5173s/iter; left time: 952.3502s
iters: 200, epoch: 7 | loss: 1.4734029
speed: 0.0420s/iter; left time: 73.1873s
iters: 300, epoch: 7 | loss: 1.4864014
speed: 0.0424s/iter; left time: 69.5053s
iters: 400, epoch: 7 | loss: 1.1650115
speed: 0.0426s/iter; left time: 65.6658s
Epoch: 7 cost time: 21.409071683883667
Epoch: 7, Steps: 485 | Train Loss: 1.4858343 Vali Loss: 2.6214731 Test Loss: 2.5515866
EarlyStopping counter: 2 out of 3
Updating learning rate to 1.5625e-05
iters: 100, epoch: 8 | loss: 1.3040706
speed: 0.5174s/iter; left time: 701.6062s
iters: 200, epoch: 8 | loss: 1.2672142
speed: 0.0420s/iter; left time: 52.7530s
iters: 300, epoch: 8 | loss: 1.3993770
speed: 0.0423s/iter; left time: 48.9095s
iters: 400, epoch: 8 | loss: 1.7092217
speed: 0.0426s/iter; left time: 44.9640s
Epoch: 8 cost time: 21.37936282157898
Epoch: 8, Steps: 485 | Train Loss: 1.4200343 Vali Loss: 2.6516719 Test Loss: 2.5801837
EarlyStopping counter: 3 out of 3
Early stopping
testing : PEMS03_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
test 5051
test shape: (5051, 1, 96, 358) (5051, 1, 96, 358)
test shape: (5051, 96, 358) (5051, 96, 358)
mse:1.9661425352096558, mae:1.2062638998031616
I tried not to use norm and found that the scale of the mse became normal, but still couldn't achieve the performance described in the paper.
Args in experiment:
Namespace(activation='gelu', batch_size=32, c_out=358, channel_independence=False, checkpoints='./checkpoints/', class_strategy='projection', d_ff=512, d_layers=1, d_model=512, data='PEMS', data_path='PEMS03.npz', dec_in=358, des='Exp', devices='0,1,2,3', distil=True, do_predict=False, dropout=0.1, e_layers=4, efficient_training=False, embed='timeF', enc_in=358, exp_name='MTSF', factor=1, features='M', freq='h', gpu=0, inverse=False, is_training=1, itr=1, label_len=48, learning_rate=0.001, loss='MSE', lradj='type1', model='iTransformer', model_id='PEMS03_96_96', moving_avg=25, n_heads=8, num_workers=10, output_attention=False, partial_start_index=0, patience=3, pred_len=96, root_path='./dataset/PEMS/', seq_len=96, target='OT', target_data_path='electricity.csv', target_root_path='./data/electricity/', train_epochs=10, use_amp=False, use_gpu=True, use_multi_gpu=False, use_norm=0)
Use GPU: cuda:0
start training : PEMS03_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0>>>>>>>>>>>>>>>>>>>>>>>>>>
train 15533
val 5051
test 5051
iters: 100, epoch: 1 | loss: 0.2624913
speed: 0.0681s/iter; left time: 323.3598s
iters: 200, epoch: 1 | loss: 0.2046000
speed: 0.0414s/iter; left time: 192.7160s
iters: 300, epoch: 1 | loss: 0.1952162
speed: 0.0416s/iter; left time: 189.4265s
iters: 400, epoch: 1 | loss: 0.1915123
speed: 0.0417s/iter; left time: 185.6636s
Epoch: 1 cost time: 22.924086570739746
Epoch: 1, Steps: 485 | Train Loss: 0.2236028 Vali Loss: 0.1956492 Test Loss: 0.2641603
Validation loss decreased (inf --> 0.195649). Saving model ...
Updating learning rate to 0.001
iters: 100, epoch: 2 | loss: 0.1867465
speed: 0.4899s/iter; left time: 2090.0281s
iters: 200, epoch: 2 | loss: 0.1924414
speed: 0.0424s/iter; left time: 176.4389s
iters: 300, epoch: 2 | loss: 0.1788835
speed: 0.0427s/iter; left time: 173.5791s
iters: 400, epoch: 2 | loss: 0.1569002
speed: 0.0430s/iter; left time: 170.4055s
Epoch: 2 cost time: 21.624907970428467
Epoch: 2, Steps: 485 | Train Loss: 0.1681240 Vali Loss: 0.1649887 Test Loss: 0.2362694
Validation loss decreased (0.195649 --> 0.164989). Saving model ...
Updating learning rate to 0.0005
iters: 100, epoch: 3 | loss: 0.1209449
speed: 0.4986s/iter; left time: 1885.1409s
iters: 200, epoch: 3 | loss: 0.1395759
speed: 0.0429s/iter; left time: 157.9302s
iters: 300, epoch: 3 | loss: 0.1247872
speed: 0.0433s/iter; left time: 154.9850s
iters: 400, epoch: 3 | loss: 0.1239995
speed: 0.0436s/iter; left time: 151.5990s
Epoch: 3 cost time: 21.9407217502594
Epoch: 3, Steps: 485 | Train Loss: 0.1301747 Vali Loss: 0.1335264 Test Loss: 0.2049689
Validation loss decreased (0.164989 --> 0.133526). Saving model ...
Updating learning rate to 0.00025
iters: 100, epoch: 4 | loss: 0.1209110
speed: 0.5015s/iter; left time: 1652.8661s
iters: 200, epoch: 4 | loss: 0.1202964
speed: 0.0432s/iter; left time: 138.1157s
iters: 300, epoch: 4 | loss: 0.1029550
speed: 0.0435s/iter; left time: 134.5850s
iters: 400, epoch: 4 | loss: 0.1273251
speed: 0.0438s/iter; left time: 131.2271s
Epoch: 4 cost time: 22.013235807418823
Epoch: 4, Steps: 485 | Train Loss: 0.1163763 Vali Loss: 0.1264918 Test Loss: 0.1891304
Validation loss decreased (0.133526 --> 0.126492). Saving model ...
Updating learning rate to 0.000125
iters: 100, epoch: 5 | loss: 0.1112179
speed: 0.5003s/iter; left time: 1406.2395s
iters: 200, epoch: 5 | loss: 0.1191604
speed: 0.0433s/iter; left time: 117.5131s
iters: 300, epoch: 5 | loss: 0.1151067
speed: 0.0439s/iter; left time: 114.6108s
iters: 400, epoch: 5 | loss: 0.1100017
speed: 0.0441s/iter; left time: 110.7392s
Epoch: 5 cost time: 22.304459810256958
Epoch: 5, Steps: 485 | Train Loss: 0.1107414 Vali Loss: 0.1210305 Test Loss: 0.1781913
Validation loss decreased (0.126492 --> 0.121031). Saving model ...
Updating learning rate to 6.25e-05
iters: 100, epoch: 6 | loss: 0.1029362
speed: 0.5167s/iter; left time: 1201.8610s
iters: 200, epoch: 6 | loss: 0.1482204
speed: 0.0434s/iter; left time: 96.5248s
iters: 300, epoch: 6 | loss: 0.1002703
speed: 0.0437s/iter; left time: 92.8868s
iters: 400, epoch: 6 | loss: 0.1189812
speed: 0.0440s/iter; left time: 89.1635s
Epoch: 6 cost time: 22.138785123825073
Epoch: 6, Steps: 485 | Train Loss: 0.1079675 Vali Loss: 0.1200538 Test Loss: 0.1769875
Validation loss decreased (0.121031 --> 0.120054). Saving model ...
Updating learning rate to 3.125e-05
iters: 100, epoch: 7 | loss: 0.0990048
speed: 0.5107s/iter; left time: 940.2505s
iters: 200, epoch: 7 | loss: 0.1185950
speed: 0.0435s/iter; left time: 75.7064s
iters: 300, epoch: 7 | loss: 0.0920268
speed: 0.0438s/iter; left time: 71.8683s
iters: 400, epoch: 7 | loss: 0.1048138
speed: 0.0441s/iter; left time: 67.9027s
Epoch: 7 cost time: 22.343345403671265
Epoch: 7, Steps: 485 | Train Loss: 0.1064951 Vali Loss: 0.1190044 Test Loss: 0.1757530
Validation loss decreased (0.120054 --> 0.119004). Saving model ...
Updating learning rate to 1.5625e-05
iters: 100, epoch: 8 | loss: 0.1264642
speed: 0.5120s/iter; left time: 694.2418s
iters: 200, epoch: 8 | loss: 0.1057447
speed: 0.0435s/iter; left time: 54.5842s
iters: 300, epoch: 8 | loss: 0.1032038
speed: 0.0439s/iter; left time: 50.7019s
iters: 400, epoch: 8 | loss: 0.1228641
speed: 0.0441s/iter; left time: 46.6026s
Epoch: 8 cost time: 22.21909260749817
Epoch: 8, Steps: 485 | Train Loss: 0.1056254 Vali Loss: 0.1186761 Test Loss: 0.1778394
Validation loss decreased (0.119004 --> 0.118676). Saving model ...
Updating learning rate to 7.8125e-06
iters: 100, epoch: 9 | loss: 0.1092731
speed: 0.5034s/iter; left time: 438.5034s
iters: 200, epoch: 9 | loss: 0.1012566
speed: 0.0435s/iter; left time: 33.5258s
iters: 300, epoch: 9 | loss: 0.1146228
speed: 0.0438s/iter; left time: 29.3889s
iters: 400, epoch: 9 | loss: 0.1040003
speed: 0.0456s/iter; left time: 26.0287s
Epoch: 9 cost time: 22.3890643119812
Epoch: 9, Steps: 485 | Train Loss: 0.1051820 Vali Loss: 0.1180485 Test Loss: 0.1759509
Validation loss decreased (0.118676 --> 0.118049). Saving model ...
Updating learning rate to 3.90625e-06
iters: 100, epoch: 10 | loss: 0.1146770
speed: 0.5025s/iter; left time: 193.9549s
iters: 200, epoch: 10 | loss: 0.1029931
speed: 0.0435s/iter; left time: 12.4420s
iters: 300, epoch: 10 | loss: 0.0811080
speed: 0.0441s/iter; left time: 8.1950s
iters: 400, epoch: 10 | loss: 0.0947617
speed: 0.0442s/iter; left time: 3.7979s
Epoch: 10 cost time: 22.295016527175903
Epoch: 10, Steps: 485 | Train Loss: 0.1048957 Vali Loss: 0.1181574 Test Loss: 0.1758959
EarlyStopping counter: 1 out of 3
Updating learning rate to 1.953125e-06
testing : PEMS03_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
test 5051
test shape: (5051, 1, 96, 358) (5051, 1, 96, 358)
test shape: (5051, 96, 358) (5051, 96, 358)
mse:0.1759510487318039, mae:0.28603628277778625
你别的试了吗
I'm also experiencing this issue. In PEMS03, the {12, 24}-step prediction results are consistent with the paper. However, the predicted {48,96}-step results are far from the authors' report. And then I change --use_norm hyperparmeters to 0, just as as @Secilia-Cxy did, the performance reported by the paper can still not be reached.
results produced by the scripts/multivariate_forecasting/PEMS/iTransformer_03.sh. (--use_norm use default value 1 in this script
[close to the results in paper] PEMS03_96_12_iTransformer_PEMS_M_ft96_sl48_ll12_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0
mse:0.06867164373397827, mae:0.17403899133205414
[close to the results in paper]
PEMS03_96_24_iTransformer_PEMS_M_ft96_sl48_ll24_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0
mse:0.09742313623428345, mae:0.20868924260139465
[worse than the results in paper]
PEMS03_96_48_iTransformer_PEMS_M_ft96_sl48_ll48_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0
mse:0.16373442113399506, mae:0.2756078243255615
[far wrose than the results in the paper]
PEMS03_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0
mse:1.262604832649231, mae:0.8765576481819153
set --use_norm as 0
[close to the results in paper]
PEMS03_96_48_iTransformer_PEMS_M_ft96_sl48_ll48_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0
mse:0.13765227794647217, mae:0.24710021913051605,rmse:0.37101519107818604,mape:2.035256862640381
[close to the results in paper]
PEMS03_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0
mse:0.17429415881633759, mae:0.28364065289497375
What's more, I find that, results for PEMS07 dataset can't be reproduced either when prediction horizon are 48 and 96.
results produced by scripts/multivariate_forecasting/PEMS/iTransformer_07.sh (--use_norm is 0 in this script)
PEMS07_96_48_iTransformer_PEMS_M_ft96_sl48_ll48_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0
mse:0.23394542932510376, mae:0.3273244798183441,rmse:0.48367905616760254,mape:3.564363479614258
PEMS07_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0
mse:1.0463123321533203, mae:0.8682820796966553,rmse:1.0228941440582275,mape:2.4747323989868164
change --use_norm to 1
PEMS07_96_48_iTransformer_PEMS_M_ft96_sl48_ll48_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0
mse:1.5068509578704834, mae:1.0126152038574219,rmse:1.2275385856628418,mape:5.98640775680542
PEMS07_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0
mse:0.8029258847236633, mae:0.6813836693763733,rmse:0.8960613012313843,mape:4.556532859802246
我也遇到了这个问题。在PEMS03中，{12,24}步预测结果与论文一致。然而，预测的{48,96}步结果与作者的报告相去甚远。然后我把 --use_norm hyperparmeters 改为 0，就像一样，论文报告的性能仍然无法达到。
scripts/multivariate_forecasting/PEMS/iTransformer_03.sh. （--use_norm 生成的结果在此脚本中使用默认值 1
[接近论文结果]PEMS03_96_12_iTransformer_PEMS_M_ft96_sl48_ll12_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.06867164373397827，mae：0.17403899133205414
[接近论文结果] PEMS03_96_24_iTransformer_PEMS_M_ft96_sl48_ll24_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.09742313623428345， mae：0.20868924260139465
[比纸上的结果还差] PEMS03_96_48_iTransformer_PEMS_M_ft96_sl48_ll48_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.16373442113399506， mae：0.2756078243255615
[比论文中的结果要好得多] PEMS03_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：1.262604832649231，MAE：0.8765576481819153
将 --use_norm 设置为 0
[接近论文结果] PEMS03_96_48_iTransformer_PEMS_M_ft96_sl48_ll48_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.13765227794647217， mae：0.24710021913051605，rmse：0.37101519107818604，mape：2.035256862640381
[接近论文结果] PEMS03_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.17429415881633759， mae：0.28364065289497375
并且我发现，当预测范围为 48 和 96 时，PEMS07 数据集的结果也无法重现。
scripts/multivariate_forecasting/PEMS/iTransformer_07.sh 生成的结果（此脚本中 --use_norm 为 0）
PEMS07_96_48_iTransformer_PEMS_M_ft96_sl48_ll48_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.23394542932510376， mae：0.3273244798183441，rmse：0.48367905616760254，mape：3.564363479614258
PEMS07_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：1.0463123321533203， mae：0.8682820796966553，rmse：1.0228941440582275，mape：2.4747323989868164
将 --use_norm 更改为 1
PEMS07_96_48_iTransformer_PEMS_M_ft96_sl48_ll48_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：1.5068509578704834，mae：1.0126152038574219，rmse：1.2275385856628418，mape：5.98640775680542
PEMS07_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.8029258847236633， mae：0.6813836693763733，rmse：0.8960613012313843，mape：4.556532859802246
[比论文中的结果要好得多]
PEMS03_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0
MSE：1.262604832649231，MAE：0.8765576481819153 兄弟你看错了吧
我也遇到了这个问题。在PEMS03中，{12,24}步预测结果与论文一致。然而，预测的{48,96}步结果与作者的报告相去甚远。然后我把 --use_norm hyperparmeters 改为 0，但论文报告的性能仍然无法达到，跟一楼的结论相近。
scripts/multivariate_forecasting/PEMS/iTransformer_03.sh. （--use_norm 生成的结果在此脚本中使用默认值 1
[接近论文结果]PEMS03_96_12_iTransformer_PEMS_M_ft96_sl48_ll12_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.06867164373397827，mae：0.17403899133205414
[接近论文结果] PEMS03_96_24_iTransformer_PEMS_M_ft96_sl48_ll24_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.09742313623428345， mae：0.20868924260139465
[比论文的结果稍差] PEMS03_96_48_iTransformer_PEMS_M_ft96_sl48_ll48_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.16373442113399506， mae：0.2756078243255615
[比论文中的结果要差得多] PEMS03_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：1.262604832649231，MAE：0.8765576481819153将 --use_norm 设置为 0
[接近论文结果] PEMS03_96_48_iTransformer_PEMS_M_ft96_sl48_ll48_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.13765227794647217， mae：0.24710021913051605，rmse：0.37101519107818604，mape：2.035256862640381
[接近论文结果] PEMS03_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.17429415881633759， mae：0.28364065289497375并且我发现，当预测范围为 48 和 96 时，PEMS07 数据集的结果也无法重现。
scripts/multivariate_forecasting/PEMS/iTransformer_07.sh 生成的结果（此脚本中 --use_norm 为 0）
PEMS07_96_48_iTransformer_PEMS_M_ft96_sl48_ll48_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.23394542932510376， mae：0.3273244798183441，rmse：0.48367905616760254，mape：3.564363479614258
PEMS07_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：1.0463123321533203， mae：0.8682820796966553，rmse：1.0228941440582275，mape：2.4747323989868164将 --use_norm 更改为 1
PEMS07_96_48_iTransformer_PEMS_M_ft96_sl48_ll48_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：1.5068509578704834，mae：1.0126152038574219，rmse：1.2275385856628418，mape：5.98640775680542
PEMS07_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.8029258847236633， mae：0.6813836693763733，rmse：0.8960613012313843，mape：4.556532859802246
应该没弄错，检查了一遍。不过我试着降低了sh文件中的学习率，跑出来的PEMS07数据集上的表现已经跟论文中的接近了。
我也遇到了这个问题。在PEMS03中，{12,24}步预测结果与论文一致。然而，预测的{48,96}步结果与作者的报告相去甚远。然后我把 --use_norm hyperparmeters 改为 0，但论文报告的性能仍然无法达到，跟一楼的结论相近。
scripts/multivariate_forecasting/PEMS/iTransformer_03.sh. （--use_norm 生成的结果在此脚本中使用默认值 1
[接近论文结果]PEMS03_96_12_iTransformer_PEMS_M_ft96_sl48_ll12_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.06867164373397827，mae：0.17403899133205414
[接近论文结果] PEMS03_96_24_iTransformer_PEMS_M_ft96_sl48_ll24_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.09742313623428345， mae：0.20868924260139465
[比论文的结果稍差] PEMS03_96_48_iTransformer_PEMS_M_ft96_sl48_ll48_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.16373442113399506， mae：0.2756078243255615
[比论文中的结果要差得多] PEMS03_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：1.262604832649231，MAE：0.8765576481819153将 --use_norm 设置为 0
[接近论文结果] PEMS03_96_48_iTransformer_PEMS_M_ft96_sl48_ll48_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.13765227794647217， mae：0.24710021913051605，rmse：0.37101519107818604，mape：2.035256862640381
[接近论文结果] PEMS03_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.17429415881633759， mae：0.28364065289497375并且我发现，当预测范围为 48 和 96 时，PEMS07 数据集的结果也无法重现。
scripts/multivariate_forecasting/PEMS/iTransformer_07.sh 生成的结果（此脚本中 --use_norm 为 0）
PEMS07_96_48_iTransformer_PEMS_M_ft96_sl48_ll48_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.23394542932510376， mae：0.3273244798183441，rmse：0.48367905616760254，mape：3.564363479614258
PEMS07_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：1.0463123321533203， mae：0.8682820796966553，rmse：1.0228941440582275，mape：2.4747323989868164将 --use_norm 更改为 1
PEMS07_96_48_iTransformer_PEMS_M_ft96_sl48_ll48_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：1.5068509578704834，mae：1.0126152038574219，rmse：1.2275385856628418，mape：5.98640775680542
PEMS07_96_96_iTransformer_PEMS_M_ft96_sl48_ll96_pl512_dm8_nh4_el1_dl512_df1_fctimeF_ebTrue_dtExp_projection_0 MSE：0.8029258847236633， mae：0.6813836693763733，rmse：0.8960613012313843，mape：4.556532859802246应该没弄错，检查了一遍。不过我试着降低了sh文件中的学习率，跑出来的PEMS07数据集上的表现已经跟论文中的接近了。
你好，我降低学习率得到的结果还是和论文原文结果有差距，你这边能提供一下完整的PEMS数据集的脚本吗？