The repository contains the experimental studies for "Evolutionary automated machine learning approach for time series classification" paper.
The experimental was conducted on the datasets taken from UEA & UCR Time Series Classification Repository.
-
For binary classification – BirdChicken, Chinatown, Computers, Coffee, DistalPhalanxOutlineCorrect Earthquakes, ECG200, FordA, GunPointAgeSpan, GunPointMaleVersusFemale, GunPointOldVersusYoung, Ham, Herring, ItalyPowerDemand, Lightning2, MiddlePhalanxOutlineCorrect, MoteStrain, PhalangesOutlinesCorrect, PowerCons, ProximalPhalanxOutlineCorrect, ShapeletSim, SonyAIBORobotSurface1, SonyAIBORobotSurface2, Strawberry, ToeSegmentation2, TwoLegECG, Wafer, WormsTwoC1ass, Yoga
-
For multiclass classification – ACSF1, Adiac, ArrowHead, Beef, Car, ChlorineConcentration, CricketX, CricketY, CricketZ, Crop, DistalPhalanxTW, DistalPhalanxOutlineAgeGroup, ECG5000, ElectricDevices, EOGVerticalSignal, EthanolLevel, FaceFour, Haptics, InlineSkate, LargeKitchenAppliances, Lightning7, Mallat, Meat, MiddlePhalanxOutlineAgeGroup, MiddlePhalanxTW, OliveOil, Phoneme, Plane, ProximalPhalanxOutlineAgeGroup, ProximalPhalanxTW, RefrigerationDevices, Rock, ScreenType, SwedishLeaf, SyntheticControl, Trace, UMD
FEDOT.Industrial framework is available in main repository.
To parse the results of the experiments, please use provided script results_parser.py
.
Dataset | SOTA result | SOTA algorithm | Baseline model | FEDOT result | Feature generation algorithm |
---|---|---|---|---|---|
ACSF1 | 0.901 | OS-CNN | 0.733 | 0.849 | WindowQuantiIe |
Adiac | 0.851 | OS-CNN | 0.011 | 0.776 | Ensemble: Quantile, Topological |
ArrowHead | 0.876 | cBOSS | 0.635 | 0.803 | WindowSpectraI |
Beef | 0.768 | OS-CNN | 0.572 | 0.828 | Quantile |
Car | 0.914 | ROCKET | 0.668 | 0.933 | Ensemble: WindowSpectral, WindowQuantile, Topological |
ChlorineConcentration | 0.850 | OS-CNN | 0.522 | 0.720 | WindowQuantiIe |
CricketX | 0.855 | InceptionTime | 0.469 | 0.707 | WindowQuantile |
Crickety | 0.863 | OS-CNN | 0.452 | 0.653 | WindowQuantile |
CricketZ | 0.859 | OS-CNN | 0.515 | 0.729 | Ensemble: WindowSpectraI, WindowQuantiIe, Topological |
Crop | 0.791 | InceptionTime | 0.647 | 0.798 | Ensemble:WindowSpectraI, WindowQuantiIe |
DistalPha1anxTW | 0.863 | OS-CNN | 0.590 | 0.660 | WindowQuantile |
DistalPhalanxOutIineAgeGroup | 0.808 | TS-CHIEF | 0.779 | 0.749 | Ensemble: Quantile, Spectral, WindowQuantiIe |
ECG5000 | 0.945 | OS-CNN | 0.008 | 0.933 | recurrence |
ElectricDevices | 0.868 | ROCKET | 0.649 | 0.725 | Ensemble: Quantile, WindowQuantile |
EOGVerticalSignaI | 0.811 | InceptionTime | 0.373 | 0.475 | Ensemble: Quantile, WindowQuantiIe |
EthanolLeveI | 0.875 | InceptionTime | 0.343 | 0.792 | Ensemble: Quantile, ECM |
FaceFour | 0.999 | TS-CHIEF | 0.579 | 0.831 | Ensemble: Topological, Quantile |
Haptics | 0.521 | STC | 0.350 | 0.440 | Ensemble: Topological, Quantile |
InlineSkate | 0.668 | WEASEL | 0.341 | 0.378 | Ensemble: Topological, WindowQuantile, Wavelet |
LargeKitchenAppliances | 0.954 | ResNet | 0.766 | 0.816 | Ensemble: Quantile, Topological |
Lightning7 | 0.818 | InceptionTime | 0.497 | 0.779 | Quantile |
Mallat | 0.976 | TS-CHIEF | 0.739 | 0.884 | Ensemble: Wavelet, WindowQuantiIe |
Meat | 0.994 | ResNet | 0.770 | 0.836 | Ensemble: Topological, Spectral |
MiddlePhaIanxOutIineAgeGroup | 0.653 | ROCKET | 0.521 | 0.606 | Ensemble: Spectral, Topological, Quantile |
MiddlePhalanxTW | 0.543 | OS-CNN | 0.442 | 0.497 | Ensemble: Spectral, WindowQuantile |
OliveOi1 | 0.897 | TS-CHIEF | 0.640 | 0.810 | Ensemble: Topological, Spectral |
Phoneme | 0.331 | OS-CNN | 0.028 | 0.244 | Topological |
Plane | 1.0 | OS-CNN | 1.0 | 1.0 | WindowQuantile |
ProximalPhalanxOutlineAgeGroup | 0.840 | OS-CNN | 0.831 | 0.847 | Ensemble: Wavelet, WindowQuantile |
ProximalPha1anxTW | 0.793 | OS-CNN | 0.725 | 0.844 | Ensemble: Wavelet, WindowQuantiIe |
RefrigerationDevices | 0.790 | HIVE-COTE v1.0 | 0.498 | 0.545 | WindowQuantile |
Rock | 0.848 | STC | 0.493 | 0.880 | Ensemble: Topological, WindowQuantiIe |
ScreenOpe | 0.755 | ResNet | 0.420 | 0.484 | Ensemble: Topological, WindowQuantiIe, Wavelet |
SwedishLeaf | 0.971 | OS-CNN | 0.810 | 0.904 | Ensemble: Topological, Spectral |
SyntheticControI | 0.999 | TS-CHIEF | 0.912 | 0.999 | Ensemble: Quantile, Spectral |
Trace | 1.0 | OS-CNN | 1.0 | 1.0 | Spectral |
UMD | 0.993 | OS-CNN | 0.892 | 1.0 | Ensemble: WindowQuantile, WindowSpectral |
Average values | 0.839 | 0.574 | 0.750 |
Dataset | SOTA result | SOTA algorithm | Baseline model | FEDOT result | Feature generation algorithm |
---|---|---|---|---|---|
BirdChicken | 0.999 | TS-CHIEF | 0.800 | 1.0 | Statistical |
Chinatown | 0.993 | ROCKET | 0.896 | 0.995 | WindowQuantile |
Computers | 0.927 | InceptionTime | 0.744 | 0.766 | Quantile |
Coffee | 1.0 | OS-CNN | 0.933 | 1.0 | WindowSpectral |
DistalPhalanxOutlineCorrect | 0.914 | ROCKET | 0.769 | 0.771 | WindowQuanti1e |
Earthquakes | 0.693 | ProximityForest | 0.509 | 0.740 | Quantile |
ECG200 | 0.957 | InceptionTime | 0.820 | 0.894 | Ensemble: Quantile, WindowQuantile |
FordA | 0.994 | WEASEL | 0.705 | 0.970 | Spectral |
GunPointAgeSpan | 0.999 | TS-CHIEF | 0.968 | 0.971 | Spectral |
GunPointMaleVersusFemale | 1.0 | OS-CNN | 0.997 | 1.0 | Spectral |
GunPointOldVersusYoung | 1.0 | OS-CNN | 1.0 | 1.0 | Spectral |
Ham | 0.706 | OS-CNN | 0.600 | 0.724 | WindowQuanti1e |
Herring | 0.686 | STC | 0.653 | 0.626 | Topological |
ItalyPowerDemand | 0.992 | TS-CHIEF | 0.723 | 0.993 | Ensemble: WindowSpectral, ECM |
Lightning2 | 0.928 | InceptionTime | 0.629 | 0.689 | WindowSpectral |
MiddlePhalanxOutlineCorrect | 0.928 | ROCKET | 0.708 | 0.803 | Window Quantile |
MoteStrain | 0.984 | HIVE-COTE v1.0 | 0.804 | 0.834 | Spectral |
PhalangesOutlinesCorrect | 0.929 | InceptionTime | 0.711 | 0.818 | Window Quantile |
PowerCons | 1.0 | TSF | 0.950 | 1.0 | Window Spectral |
ProximalPhalanxOutlineCorrect | 0.946 | InceptionTime | 0.709 | 0.848 | Window Quantile |
ShapeletSim | 1.0 | HIVE-COTE v1.0 | 0.489 | 1.0 | Topological |
SonyAIBORobotSurface1 | 0.998 | ResNet | 0.849 | 0.892 | Window Quantile |
SonyAIBORobotSurface2 | 0.997 | ResNet | 0.770 | 0.824 | Window Quantile |
Strawberry | 0.997 | ROCKET | 0.905 | 0.924 | Spectral |
ToeSegmentation2 | 0.995 | HIVE-COTE v1.0 | 0.622 | 0.869 | Spectral |
TwoLeadECG | 1.0 | ResNet | 0.846 | 0.919 | Quantile |
Wafer | 1.0 | TS-CHIEF | 0.944 | 1.0 | Quantile |
WormsTwoC1ass | 0.904 | BOSS | 0.652 | 0.715 | Topological |
Yoga | 0.975 | S-BOSS | 0.730 | 0.797 | WindowQuantile |
Average values | 0.946 | 0.774 | 0.873 |
Here will be provided a list of citations for the project as soon as articles will be published.
So far you can use citation for this repository:
.. code-block:: bibtex
@online{fedot_industrial,
author = {Revin, Ilya and Potemkin, Vadim and Balabanov, Nikita and Nikitin, Nikolay},
title = {FEDOT.Industrial - Framework for automated time series analysis},
year = 2022,
url = {https://github.com/ITMO-NSS-team/Fedot.Industrial},
urldate = {2022-05-05}
}