This is currently a development project. Hope is that this might become a Masters/PhD research project at some stage. Search for new ideas and concepts continue..
Anomaly detection in the age of deep learning
Background: Anomalies can be broadly identified as patterns in datasets that do not follow a well-defined trend of normal behaviour, and the goal of anomaly detection is to detect all such behaviour if possible. Anomalies can arise out of errors in the data collection process but sometimes, and somewhat more interestingly, can be indicative of a new, previously unknown, underlying process. Anomaly detection is a burgeoning research field and has a diverse range of applications including fraud detection, health care monitoring, fault detection in machines to name a few.
Problem: With the advancements in machine learning techniques and proliferation in computing power in recent years, deep learning has specifically gained widespread attention as a testbed to learn intricate underlying representation of often complex real world datasets.The current project thus aims to compare the performance of deep learning based anomaly detection methods with more traditional approaches for anomaly detection such as k-means, Isolation Forests and SVM. This comparison is particularly important for large scale datasets where performance of traditional models is expected to be less than optimal as it would be difficult for these models to capture complex structure of multidimensional real world datasets.
Action: The project will begin with a brief literature review of the field, with a special focus on identifying relevant data sources and narrowing down the most relevant deep learning models suited for the model. The project will mainly focus on investigating Unsupervised deep anomaly detection methods, if a suitable data source with labelled dataset is identified then Semi-supervised and Supervised deep anomaly detection methods can be investigated as well. Various methods can be explored under deep learning based anomaly detection including Generative models and Deep Belief Networks.