RobustETF

Robust Enhanced Trend Filtering with Unknown Noise

This repository contains the implementation details of our paper: [Signal Processing] "Robust Enhanced Trend Filtering with Unknown Noise" by Zhibin Zhao.

About

One important step in time series analysis is the extraction of an underlying trend. However, the true trend is often submerged by complex background noise, espcially non-Gaussian noise or outliers. Accurate trend extraction against outliers from a raw signal is a challenging task. To address this challenge, this paper extends $l_1$ trend filtering to a robust enhanced trend filtering called RobustETF by combining mix of Gaussian (MoG) and non-convex sparsity-inducing functions. We first model the noise as a MoG distribution to allow RobustETF to be robust in the presence of any type of non-Gaussian noise or outliers. After that, to handle the biased estimation of the $l_1$ norm, we use the Gibbs distribution embedding smoothed and non-convex sparsity-inducing functions to faithfully preserve the amplitude of the trend. Furthermore, we design an extended EM algorithm to solve the resulting non-convex optimization problem. Finally, we show the results of experiments on both real-world and synthetic data to compare the performance of the proposed algorithm against other state-of-the-art methods.

Dependencies

Matlab R2016b

Pakages

This repository is organized as:

Figures contains the generated figures of the algorithm.
util contains the extra functions of the test.
Plot contains the Plot functions of the test.
Performance_Data contains the the simulation parameters of the algorithm.
Data contains the data generated by NSF I/UCR Center on Intelligent Maintenance Systems (IMS) and Yahoo’s Anomaly Detection Dataset.
Robust_Trend_Filtering contains the main funtions of the algorithm. In our implementation, Matlab R2016b is used to perform all the experiments.

Main Test Codes:

Plot_SimulationsOutliers.m: Visualition of the extracted results for simulation signals, corresponding to Fig.8 and Fig.9 in the paper.
Plot_Bearing_Data.m: Visualition of the extracted results for bearing run-to-failure signals, corresponding to Fig.11 and Fig.12 in the paper.
Plot_Bearing_Data_with_Outliers.m: Visualition of the extracted results for bearing run-to-failure signals with outliers, corresponding to Fig.13 in the paper.
Plot_Yahoo_Data.m: Visualition of the extracted results for Yahoo signals, corresponding to Fig.15 in the paper.

Implementation:

Flow the steps presented below:

Clone this repository.

git clone https://github.com/ZhaoZhibin/RobustETF.git
open it with matlab

Citation

If you feel our RobustETF is useful for your research, please consider citing our paper:

@article{zhao2020Robust,
  title={Robust Enhanced Trend Filtering with Unknown Noise},
  author={Zhao, Zhibin and Wang, Shibin and Wong, David and Sun, Chuang and Ruqiang Yan and Chen, Xuefeng},
  journal={Signal Processing},
  year={2020},
  publisher={Elsevier}
}

Contact

zhibinzhao1993@gmail.com