Training possible ?

[DirkFries]

Hi @ALL,

is it possible to train Sherlock/EGADS with data from the past to get better results in the future ?
I think I heard somethingh like that in a video, but I can't find any Howto or hint in the menus.

Thanks a lot !

Bye, Dirk

[jigs1993]

Hi @DirkFries

So the training is possible. You can follow the README on EGADS here https://github.com/yahoo/EGADS#configuration

here in Sherlock the same config file is used to specify default training parameters https://github.com/yahoo/sherlock/blob/master/src/main/resources/egads_config.ini (also refer https://github.com/yahoo/sherlock#cli-args-usage to use params from different filepath)

You can use https://github.com/yahoo/sherlock/blob/master/src/main/java/com/yahoo/sherlock/query/EgadsConfig.java class to change params at runtime

[DirkFries]

Hi jigs1993,

thanks a lot for your answer. I already knew the links you have posted, but neither me nor a colleague could find any hints about training or learning. Maybe I should point out a little closer what I am planing to do.

My goal is to collect TimeSeriesData during a time with no real anomalies, give this data to e.g. EGADS/Sherlock so that they can learn what is a normal behavior and find anomalys in the future more precise.

As far as I know at the moment it is only possible to choose between a number of TimeSeriesModels and AnomalyModels and EGADS/Sherlock work in an Unsupervised anomaly detection mode.

If I am wrong I'd be glad if you could correct me.

[jigs1993]

Hi @DirkFries

So currently Sherlock runs anomaly detection job periodically based on the job frequency(i.e. hour/day). Let say the job frequency is hourly, hence each hour anomaly detection job looks back interval-hours=672 (default value) (https://github.com/yahoo/sherlock#interval-hours) hours worth of data and does training on those data-points on the fly(at runtime) and gives you the anomaly result for the current hour.

As far as i understand, your goal seems to train the model on offline data and reuse that model for each anomaly job in the future. is this correct?

[DirkFries]

Hi jigs1993,

thank you very much for your answer and your time.
Yes, your description is absolutely correct.

[jigs1993]

Hi @DirkFries

As of now sherlock does not support offline trained models, although i would be happy to add this as a feature in the future.