A professionally curated list of awesome resources (paper, code, data, etc.) on Transformers in Time Series, which is first work to comprehensively and systematically summarize the recent advances of Transformers for modeling time series data to the best of our knowledge.
We will continue to update this list with newest resources. If you found any missed resources (paper/code) or errors, please feel free to open an issue or make a pull request.
Transformers in Time Series: A Survey
Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan and Liang Sun.
@article{wen2022tstransformers,
title={Transformers in Time Series: A Survey},
author={Wen, Qingsong and Zhou, Tian and Zhang, Chaoli and Chen, Weiqi and Ma, Ziqing and Yan, Junchi and Sun, Liang},
journal={arXiv preprint arXiv:2202.07125},
year={2022}
}
- Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting, in NeurIPS 2019. [paper] [code]
- Informer: Beyond efficient transformer for long sequence time-series forecasting, in AAAI 2021. [paper] [official code] [dataset]
- Adversarial sparse transformer for time series forecasting, in NeurIPS 2020. [paper] [code]
- Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting, in NeurIPS 2021. [paper] [official code]
- Probabilistic Transformer For Time Series Analysis, in NeurIPS 2021. [paper]
- Temporal fusion transformers for interpretable multi-horizon time series forecasting, in International Journal of Forecasting 2021. [paper] [code]
- SSDNet: State Space Decomposition Neural Network for Time Series Forecasting, in ICDM 2021, [paper]
- From Known to Unknown: Knowledge-guided Transformer for Time-Series Sales Forecasting in Alibaba, in arXiv 2021. [paper]
- Pyraformer: Low-Complexity Pyramidal Attention for Long-Range Time Series Modeling and Forecasting, in ICLR 2022. [paper]
- FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting, in arXiv 2022. [paper]
- Spatio-temporal graph transformer networks for pedestrian trajectory prediction, in ECCV 2020. [paper] [official code]
- Spatial-temporal transformer networks for traffic flow forecasting, in arXiv 2020. [paper] [official code]
- Traffic transformer: Capturing the continuity and periodicity of time series for traffic forecasting, in Transactions in GIS 2022. [paper]
- Self-attentive Hawkes process, in ICML 2020. [paper] [official code]
- Transformer Hawkes process, in ICML 2020. [paper] [official code]
- Transformer Embeddings of Irregularly Spaced Events and Their Participants, in ICLR 2022. [paper] [official code]
- Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy, in ICLR 2022. [paper] [code]
- TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data, in VLDB 2022. [paper] [official code]
- Learning graph structures with transformer for multivariate time series anomaly detection in IoT, in IEEE Internet of Things Journal 2021. [paper] [official code]
- Spacecraft Anomaly Detection via Transformer Reconstruction Error, in ICASSE 2019. [paper]
- Unsupervised Anomaly Detection in Multivariate Time Series through Transformer-based Variational Autoencoder, in CCDC 2021. [paper]
- Variational Transformer-based anomaly detection approach for multivariate time series, in Measurement 2022. [paper]
- A transformer-based framework for multivariate time series representation learning, in KDD 2021. [paper] [official code]
- Voice2series: Reprogramming acoustic models for time series classification, in ICML 2021. [paper] [official code]
- Gated Transformer Networks for Multivariate Time Series Classification, in arXiv 2021. [paper] [official code]
- Self-attention for raw optical satellite time series classification, in ISPRS Journal of Photogrammetry and Remote Sensing 2020. [paper] [official code]
- Self-supervised pretraining of transformers for satellite image time series classification, in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 2020. [paper]
- Time series data augmentation for deep learning: a survey, in IJCAI 2021. [paper]
- Neural temporal point processes: a review, in IJCAI 2021. [paper]
- Time-series forecasting with deep learning: a survey, in Philosophical Transactions of the Royal Society A 2021. [paper]
- Deep learning for time series forecasting: a survey, in Big Data 2021. [paper]
- Neural forecasting: Introduction and literature overview, in arXiv 2020. [paper]
- Deep learning for anomaly detection in time-series data: review, analysis, and guidelines, in Access 2021. [paper]
- A review on outlier/anomaly detection in time series data, in ACM Computing Surveys 2021. [paper]
- A unifying review of deep and shallow anomaly detection, in Proceedings of the IEEE 2021. [paper]
- Deep learning for time series classification: a review, in Data Mining and Knowledge Discovery 2019. [paper]
- A survey on visual transformer, in IEEE TPAMI 2022. [paper]
- Transformers in vision: A survey, in ACM Computing Surveys 2021. [paper]
- Video Transformers: A Survey, in arXiv 2022. [paper]
- Efficient transformers: A survey, in arXiv 2020. [paper]
- A Survey of Transformers, in arXiv 2021. [paper]
- A Survey of Vision-Language Pre-Trained Models, in arXiv 2022. [paper]
- Survey: Transformer based video-language pre-training, in AI Open 2022. [paper]
- Pre-trained models for natural language processing: A survey, in Science China Technological Sciences 2020. [paper]
- Pre-trained models: Past, present and future, in AI Open 2021. [paper]
- An attentive survey of attention models, in ACM TIST 2021. [paper]
- Attention in natural language processing, in IEEE TNNLS 2020. [paper]
- Transformer for Graphs: An Overview from Architecture Perspective, in arXiv 2022. [paper]
- Transformers in Medical Imaging: A Survey, in arXiv 2022. [paper]