munhouiani/MIDAS

Anomaly Detection on Dynamic (time-evolving) Graphs in Real-time and Streaming manner. Detecting intrusions (DoS and DDoS attacks), frauds, fake rating anomalies.

MIDAS

This repo is forked from Stream-AD/MIDAS. The core is written in C++. This repo added a Python wrapper to it.

Blog post: https://blog.munhou.com/2021/03/27/Network-Anomaly-Detection-with%20MIDAS-MIDAS-R-and-MIDAS-F/

Requirements

• Cmake >= 3.15
• C++11
• C++ standard libraries

Installation

1. Make sure you clone this repo with --recursive
2. Open a terminal
3. cd to the project root MIDAS/
4. Install with pip: pip install .
5. Run following code to test.

from MIDAS import MIDAS, MIDASR, MIDASF


num_row = 2
num_col = 1024
factor = 0.5
threshold = 1e3

midas = MIDAS(num_row=num_row, num_col=num_col)
midas_r = MIDASR(num_row=num_row, num_col=num_col, factor=factor)
midas_f = MIDASF(num_row=num_row, num_col=num_col, threshold=threshold, factor=factor)

example_source = 3
example_destination = 5
example_timestamp = 1


score = midas.add_edge(source=example_source, destination=example_destination, timestamp=example_timestamp)
score_r = midas_r.add_edge(source=example_source, destination=example_destination, timestamp=example_timestamp)
score_f = midas_f.add_edge(source=example_source, destination=example_destination, timestamp=example_timestamp)

# dump model 
midas.dump('midas.json')
midas_r.dump('midas_r.json')
midas_f.dump('midas_f.json')

# load model
midas = MIDAS.load('midas.json')
midas_r = MIDASR.load('midas_r.json')
midas_f = MIDASF.load('midas_f.json')

Citation

If you use this code for your research, please consider citing the original authors' arXiv preprint

@misc{bhatia2020realtime,
    title={Real-Time Streaming Anomaly Detection in Dynamic Graphs},
    author={Siddharth Bhatia and Rui Liu and Bryan Hooi and Minji Yoon and Kijung Shin and Christos Faloutsos},
    year={2020},
    eprint={2009.08452},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

or their AAAI paper

@inproceedings{bhatia2020midas,
    title="MIDAS: Microcluster-Based Detector of Anomalies in Edge Streams",
    author="Siddharth {Bhatia} and Bryan {Hooi} and Minji {Yoon} and Kijung {Shin} and Christos {Faloutsos}",
    booktitle="AAAI 2020 : The Thirty-Fourth AAAI Conference on Artificial Intelligence",
    year="2020"
}