arundo/adtk

STLDecomposition Implementation

Opened this issue · 7 comments

Hello,
First of all thank you for open sourcing the package; it is excellently written.

One potential issue I've noticed: the STL Decomposition seems to be implemented using a rolling window method for detrending data, not the Loess method. Here is a code snippet from transformer_1d.py:

def _remove_trend(self, s):
        s_trend = s.rolling(
            window=(self.freq_ if self.freq_ % 2 else self.freq_ + 1),
            center=True,
        ).mean()
        return s - s_trend

Perhaps I'm missing something? The current implementation seems to be what Hyndman calls Classical Decomposition, and not STL.
There is a python library that does STL decompose and it is relatively lightweight. And if what I'm describing is indeed the behavior, can I work on implementing STL with Loess?

Any feedback would be greatly appreciated,
Thank you

@yy910616 Thank you for raising the issue and sorry for the late response (we were on holiday these days in US). I will look into this issue this week and get back to you.

@yy910616 I think you are right. I appreciate that you raised the problem. I'm gonna work on this and hopefully have it in the next release. Will keep you updated when a PR is ready to be reviewed.

@yy910616 I just realize you proposed to work on the implementation. Sorry for missing it. You are definitely welcomed to join the contributors. I opened a WIP PR #18 for this issue.

@tailaiw Awesome! I have pretty limited experience with open source but I'll definitely give it a shot.
Thanks for opening the PR, I've been doing some research on STL and I'll continue the discussion in the PR.

After discussion in #18, we decided to remove the wrong implementation of STL decomposition, and merge two implementations of classic decomposition into the adtk.transformer.ClassicSeasonalDecomposition. We decided to hold on implementation of a new STL decomposition until statsmodels' release with its first implementation of STL algorithm.

@yy910616 i just noticed that statsmodels v0.11 has been released.

Nice. I'm looking into this. Thanks for pointing it out.