detailed performance for model

[LongxingTan]

Dear @shaido987 ,

Thanks for sharing such nice solution.

I noticed you use 2 models here, DLinear and GraphWavenet. Both of them sounds interesting. Do you mind to share each model's performance respectively, I just want to know which model performs better.

[shaido987]

Hello @LongxingTan,

Thanks and congratulations for taking 3rd place in the competition (or 4th? the about section and readme differ).

Unfortunately, we don't have the exact online scores of the two models as we did a number of smaller improvements to both and mainly submitted different merged models in the 3rd phase. For only DLinear (we denote the model with our adjustments as MDLinear) we got a score of -45.64451 (compared to the final fused model score of -45.18225) in the third phase but this is before some improvements. For our graph/TCN-based model (XTGN) we don't have an exact score in phase 3 but noted that its standalone scores in the 2nd phase were a bit worse than MDLinear overall.

We have some offline evaluation scores, but as you can see low scores here do not directly correspond to low scores in the online evaluation (as the fused model give the best online result):

[LongxingTan]

@shaido987
Congratulations to you and your team as well.

Good to know about this. I once noticed DLinear and Graph-TCN during the competition, but there are so many potential models for this task, and it's difficult for me to reproduce it in Tensorflow, so I didn't try them. It's nice to see they perform so well.

Thanks for your fast response, and such nice solution.
Longxing

[shaido987]

Yes, we had some other models as well that we were experimenting with but didn't make it until the end, including fusions with more than two models. The final combination of these two approaches seems to have complemented each other well in the final evaluation.