This repository contains the evaluation reproduction material and guideline for the THREATRACE's paper. The complete detection system will be released to the community soon.
This is the guideline for reproducing the evaluation in the paper. THREATRACE has the implementation of the Ubuntu system currently.
-
Clone the complete repository into a directory (named as ROOT, the same below) of your Ubuntu system.
-
Deploy PyG, a geometric deep learning extension library for PyTorch. The installation guideline can be found here: https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html.
-
Other required Python packages:
numpy, pandas, argparse, subprocess, os, sys, time, psutil, random, csv, re
- The version of Python and some packages:
Python 3.6.13
pytorch 1.9.1
torch-cluster 1.5.9
torch-geometric 1.4.3
torch-scatter 2.0.9
torch-sparse 0.6.12
torch-spline-conv 1.2.1
Due to space constraints, we cannot store the parsed datasets in this repository. Therefore, the first step of the evaluation reproduction is to prepare the datasets.
StreamSpot dataset
Download the StreamSport dataset from: https://github.com/sbustreamspot/sbustreamspot-data
You need to download:
all.tar.gz
Copy the dataset to ROOT/threaTrace/graphchi-cpp-master/graph_data/streamspot/:
cp all.tar.gz ROOT/threaTrace/graphchi-cpp-master/graph_data/streamspot/
Run the parsing script:
cd ROOT/threaTrace/scripts
python parse_streamspot.py
Unicorn SC-2 dataset
Download the Unicorn SC-2 dataset from: https://github.com/margoseltzer/shellshock-apt
You need to download:
camflow-benign-*
camflow-attack-*
Copy the dataset to ROOT/threaTrace/graphchi-cpp-master/graph_data/unicornsc/:
cp camflow-benign-* ROOT/threaTrace/graphchi-cpp-master/graph_data/unicornsc/
cp camflow-attack-* ROOT/threaTrace/graphchi-cpp-master/graph_data/unicornsc/
Run the parsing script:
cd ROOT/threaTrace/scripts
python parse_unicornsc.py
DARPA TC dataset
Download the DARPA TC dataset from: https://drive.google.com/drive/folders/1fOCY3ERsEmXmvDekG-LUUSjfWs6TRdp-
You need to download:
cadets/ta1-cadets-e3-official.json.tar.gz
cadets/ta1-cadets-e3-official-2.json.tar.gz
fivedirections/ta1-fivedirections-e3-official-2.json.tar.gz
theia/ta1-theia-e3-official-1r.json.tar.gz
theia/ta1-theia-e3-official-6r.json.tar.gz
trace/ta1-trace-e3-official-1.json.tar.gz
Copy the dataset to ROOT/threaTrace/graphchi-cpp-master/graph_data/darpatc/:
cp ta1-* ROOT/threaTrace/graphchi-cpp-master/graph_data/darpatc/
Run the parsing script:
cd ROOT/threaTrace/scripts
python parse_darpatc.py
We provide example models in ROOT/threaTrace/example_models/streamspot for convenience. You can also choose to train the models.
Use example models
cd ROOT/threaTrace/scripts
python setup.py
cp ROOT/threaTrace/example_models/streamspot/* ROOT/threaTrace/models/
cp ROOT/threaTrace/example_models/streamspot/models_list.txt ./
cp ROOT/threaTrace/example_models/streamspot/run_* ./
Or train models
cd ROOT/threaTrace/scripts
python setup.py
python train_streamspot.py
The training procedure may take some time.
Test
Once the models are prepared, you can start testing THREATRACE in StreamSpot dataset.
cd ROOT/threaTrace/scripts
chmod 777 run_benign.sh
chmod 777 run_attack.sh
./run_benign.sh
./run_attack.sh
When the testing procedure finishes, use the following command to evaluate the detection performance in StreamSpot dataset.
python evaluate_streamspot.py
We provide example models in ROOT/threaTrace/example_models/unicornsc for convenience. You can also choose to train the models.
Use example models
cd ROOT/threaTrace/scripts
python setup.py
cp ROOT/threaTrace/example_models/unicornsc/* ROOT/threaTrace/models/
cp ROOT/threaTrace/example_models/unicornsc/models_list.txt ./
cp ROOT/threaTrace/example_models/unicornsc/run_* ./
Or train models
cd ROOT/threaTrace/scripts
python setup.py
python train_unicornsc.py
The training procedure may take some time.
Test
Once the models are prepared, you can start testing THREATRACE in Unicorn SC-2 dataset.
cd ROOT/threaTrace/scripts
chmod 777 run_benign.sh
chmod 777 run_attack.sh
./run_benign.sh
./run_attack.sh
When the testing procedure finishes, use the following command to evaluate the detection performance in the Unicorn SC-2 dataset.
python evaluate_unicornsc.py
We provide example models in ROOT/threaTrace/example_models/darpatc for convenience. You can also choose to train the models.
Use example models. Choose the scene X: cadets/trace/theia/fivedirections
cd ROOT/threaTrace/scripts
python setup.py
cp ROOT/threaTrace/example_models/darpatc/X/* ROOT/threaTrace/models/
Or train models. Choose the scene X: cadets/trace/theia/fivedirections
cd ROOT/threaTrace/scripts
python setup.py
python train_darpatc.py --scene X
The training procedure may take some time.
Test
Once the models are prepared, you can start testing THREATRACE in DARPA TC dataset. Set X as the same as the value in the training phase.
python test_darpatc.py --scene X
When the testing procedure finishes, use the following command to evaluate the detection performance in DARPA TC dataset.
python evaluate_darpatc.py
Thanks for your interest in THREATRACE! Contact us (wangsu17@mails.tsinghua.edu.cn) if there are any questions or discussions!
This project is licensed under the MIT License - see the LICENSE file for details