This document shows a list of bibliographical references on DeepLearning and Time Series, organized by type and year. I add some additional notes on each reference.
Table of contents
- Deef Belief Network with Restricted Boltzmann Machine
- Long short-term memory
- Auto-Encoders
- Deef Belief Network with Restricted Boltzmann Machine - Auto-Encoders
- Long Short-Term Memory - Deef Belief Network with Restricted Boltzmann Machine - AutoEncoders Long Short-Term Memory
- Others
- Reviews
-
Summary: The paper proposes deep neural network (DNN)-based load forecasting models and apply them to a demand side empirical load database. DNNs are trained in two different ways: a pre-training restricted Boltzmann machine and using ReLu without pre-training.
Notes:
- Model 1 train -> greedy layer-wise manner
- Model 1 Fine-tuning connection weights -> Back-propagation
- Model 2 train -> ReLu
- Model Sizes -> trial and error
-
Summary: In this paper a Deep Belief Network (DBN) including two restricted Boltzmann machines (RBMs) was used to model load demand series.
-
Summary: This paper presents a hybrid prediction method using DBNs (deep Belief Network) and ARIMA. (without access to full paper)
-
Summary: This paper introduces a reinforcement learning method named stochastic gradient ascent (SGA) to the DBN with RBMs instead conventional BackPropagation to predict a benchmark named CATS data.
-
Summary: The paper proposes a deep learning approach, which hybridizes a deep belief networks (DBNs) and a nonlinear kernel-based parallel evolutionary SVM (ESVM), to predict evolution states of complex systems in a classification manner.
Notes:
- Top layer -> SVM
- Fine-tuning connection weights -> Back-propagation
-
Summary: In this paper, a deep learning method, the Deep Belief Network (DBN) model, is proposed for short-term traffic speed information prediction.
Notes:
- Model train -> greedy layer-wise manner
- Fine-tuning connection weights -> Back-propagation
- Model Sizes -> several ccombinations
-
Summary: This paper proposes an ensemble of deep learning belief networks (DBN) for regression and time series forecasting on electricity load demand datasets. Another contribution is to aggregate the outputs from various DBNs by a support vector regression (SVR) model.
Notes:
- Top layer -> support vector regression (SVR)
-
Summary: This papers proposes a method for time series prediction using deep belief nets (DBN) (with 3-layer of RBMs to capture the feature of input space of time series data).
Notes:
- Mode Train -> greedy layer-wise manner
- Fine-tuning connection weights -> Back-propagation
- Mode sizes and learning rates -> PSO
-
Summary: This paper pses a traffic forecast model based on long short-term memory (LSTM) network, that considers temporal-spatial correlation in traffic system via a two-dimensional network which is composed of many memory units.
-
Summary: The paper proposes a model incorporating a sequence-to-sequence model that consists two LSTMs, one encoder and one decoder. The encoder LSTM accepts input time series, extracts information and based on which the decoder LSTM constructs fixed length sequences that can be regarded as discriminatory features. The paper also introduces the attention mechanism.
-
Summary: This paper proposes an application of deep learning models, Paragraph Vector, and Long Short-Term Memory (LSTM), to financial time series forecasting.
-
Summary: This paper explores a deep learning model, the LSTM neural network model, for travel time prediction. By employing the travel time data provided by Highways England dataset, the paper construct 66 series prediction LSTM neural networks.
-
Summary: This paper presents an energy load forecasting methodology based on Deep Neural Networks (Long Short Term Memory (LSTM) algorithms). The presented work investigates two LSTM based architectures: 1) standard LSTM and 2) LSTM-based Sequence to Sequence (S2S) architecture. Both methods were implemented on a benchmark data set of electricity consumption data from one residential customer.
Notes:
- Model train -> Backpropagation
-
Summary: This paper explores the application of Long Short-Term Memory Networks (LSTMs) in short-term traffic flow prediction.
-
Summary: This paper compares conventional machine learning methods with modern neural network architectures to better forecast analgesic responses. The paper applies the LSTM to predict what the next measured pain score will be after administration of an analgesic drug, and compared the results with simpler techniques.
-
Summary: This paper uses a stacked long short-term memory model to learn and predict the patterns of traffic conditions (that are collected from online open web based map services).
Notes:
- Model sizes and learning rates -> several ccombinations
-
Summary: This paper proposed a novel spatiotemporal deep learning (STDL)-based air quality prediction method that inherently considers spatial and temporal correlations. A stacked autoencoder (SAE) model is used to extract inherent air quality features.
Notes:
- Model Train -> greedy layer-wise manner
- Top layer -> logistic regression
- Fine-tuning connection weights -> Back-propagation
- Model sizes -> several ccombinations
-
Summary: The paper introduces an architecture based on Deep Learning for the prediction of the accumulated daily precipitation for the next day. More specifically, it includes an autoencoder for reducing and capturing non-linear relationships between attributes, and a multilayer perceptron for the prediction task.
Notes:
- Top layer -> multilayer perceptron
- Model sizes and learning rates -> several combinations
-
Summary: This paper proposes a stacked autoencoder Levenberg–Marquardt model to improve forecasting accuracy. It is applied to real-world data collected from the M6 freeway in the U.K.
Notes:
- Fine-tuning connection weights -> Levenberg-Marquadt
-
Summary: This paper compares a deep learning network (Stacked Denoising Auto-Encoders (SDAE)) against a standard neural network for predicting air temperature from historical pressure, humidity, and temperature data gathered from meteorological sensors in Northwestern Nevada. In addition, predicting air temperature from historical air temperature data alone can be improved by employing related weather variables like barometric pressure, humidity and wind speed data in the training process.
Notes:
- Top layer -> feed-forward neural network
-
Summary: This paper presents a study of deep learning techniques (Stacked Denoising Auto-Encoders (SDAEs)) applied to time-series forecasting in a real indoor temperature forecasting task.
-
Summary: This work aims to investigate the use of some of deep learning architectures (deep belief networks and aunto-encoders) in predicting the hourly average speed of winds in the Northeastern region of Brazil.
Notes:
- Model Train -> greedy layer-wise manner
- Fine-tuning connection weights -> Levenberg-Marquadt
- Model sizes -> several combinations
Long Short-Term Memory - Deef Belief Network with Restricted Boltzmann Machine - AutoEncoders Long Short-Term Memory
-
Summary: This paper introduces different Deep Learning and Artificial Neural Network algorithms, such as Deep Belief Networks, AutoEncoder, and LSTM in the field of renewable energy power forecasting of 21 solar power plants.
-
Summary: This paper proposes a convolutional neural network (CNN) framework for time series classification. Two groups of experiments are conducted on simulated data sets and eight groups of experiments are conducted on real-world data sets from different application domains.
Type: Convolutional neural network
-
Summary: This paper exploits the applicability of and compares the performance of the Feed-forward Deep Neural Network (FF-DNN) and Recurrent Deep Neural Network (R-DNN) models on the basis of accuracy and computational performance in the context of time-wise short term forecast of electricity load. The herein proposed method is evaluated over real datasets gathered in a period of 4 years and provides forecasts on the basis of days and weeks ahead.
Type: Recurrent neural network
-
Summary: The paper presents a deep convolutional factor analyser (DCFA) for multivariate time series modeling. The network is constructed in a way that bottom layer nodes are independent. Through a process of up-sampling and convolution, higher layer nodes gain more temporal dependency.
Type: Convolutional neural network
-
Summary: The present study uses three years' worth of point-of-sale (POS) data from a retail store to construct a sales prediction model that, given the sales of a particular day, predicts the changes in sales on the following day.
Type: Not specified
-
Summary: This work reports on employing the deep learning artificial intelligence techniques to predict the energy consumption and power generation together with the weather forecasting numerical simulation. An optimization tool platform using Boltzmann machine algorithm for NMIP problem is also proposed for better computing scalable decentralized renewable energy system.
Type: a novel optimization tool platform using Boltzmann machine algorithm for NMIP
-
Summary: This study investigates deep learning techniques for weather forecasting. In particular, this study will compare prediction performance of Recurrence Neural Network (RNN), Conditional Restricted Boltzmann Machine (CRBM), and Convolutional Network (CN) models. Those models are tested using weather dataset which are collected from a number of weather stations.
Type: Recurrent neural network, convolutional neural network
-
Summary: The aim of this paper is to present deep neural network architectures and algorithms and explore their use in time series prediction. Shallow and deep neural networks coupled with two input variable selection algorithms are compared on a ultra-short-term wind prediction task.
Type: MultiLayer Perceptron.
-
Summary: This paper explores the feature learning techniques to improve the performance of traditional feature-based approaches. Specifically, the paper proposes a deep learning framework for multivariate time series classification in two groups of experiments on real-world data sets from different application domains.
Type: Multi-Channels Deep Convolution Neural Networks
-
Dmitry Vengertsev (2014). Deep Learning Architecture for Univariate Time Series Forecasting
Summary: This paper overviews the particular challenges present in applying Conditional Restricted Boltzmann Machines (CRBM) to univariate time-series forecasting and provides a comparison to common algorithms used for time-series prediction. As a benchmark dataset for testing and comparison of forecasting algorithms, the paper selected M3 competition dataset.
Type: Conditional Restricted Boltzmann Machines (CRBM)
-
John Cristian Borges Gamboa 2017. Deep Learning for Time-Series Analysis
Summary: This paper presents review of the main Deep Learning techniques, and some applications on Time-Series analysis are summaried.
-
Summary: This paper gives a review of the recent developments in deep learning and unsupervised feature learning for time-series problems.