The strategy for caculate f1 score
Conearth opened this issue · 2 comments
Hi, this is an amazing job! Thx for your attention.
I have a few questions about the strategy for caculate f1 score. For ASD and SMD, it seems that you trained your model and caculate the result once on a single entity like "omi-x" or "machine-x-x". So how to caculate the final score for the whole dataset?
And is there any possible to train the model on the all entities together? Have you tried?
I'll appreciate it if you can help out.
Yes, we train a model for each single entity in ASD and SMD, and the final score for the whole dataset is the average F1 score for each entity.
We did not train the model on all entities together. One entity represents the monitoring metrics of a server, and we train a model to detect anomalies on that server. Different entities may have different normal patterns. It might be hard for the model to learn such patterns together.
However, you may try to train the model on all the entities together, just using the training data for all entities as the training set. Maybe a mixture of prior distribution is needed to learn the diverse patterns.
Many thx.