Xylambda/attention

Minor time series forecasting experiments using attention mechanisms.

Description

This repository contains some simple experiments to test the goodness of simple attention mechanisms + Time2Vec on LSTM networks.

Find more information in this Quantdare post.

Run the experiments

Install the requirements.txt file and then run the main.py file:

python main.py

Hyperparameters and Feature Engineering

Beware of the number of parameters: it must be approximately the same for all models to ensure the comparison is fair. The hyperparameter configuration is only set to ensure all models have approximately the same number of parameters.

Regarding the features, I haven't done anything since I just wanted to test the models' goodness, not caring about achieving optimal performance.

CREDITS

I did not implement the Time2Vec Module but I used the one create by garyzccisme in his repo.