CREMEv2: A toolchain of automatic dataset collection for machine learning in intrusion detection based on MITRE ATT&CK
- This tool is an extended part of 1st version CREME: A toolchain of automatic dataset collection for machine learning in intrusion detection.
- In this part we try to:
- improve the stages from original 3 stages into N stages (follow MITRE ATT&CK)
- improve the labeling --> the data will be labelled using MITRE technique
This tool need to be run at the Virtualbox environment. You need install the Virtualbox first. In pricipal, we need at least 10 VMs to be launched to run this tool. The VMs are:
- Controller Machine
- Data Logger Server
- Attacker Server
- Non Vulnerable Client-1
- Non Vulnerable Client-2
- Vulnerable Client
- Malicious Client
- Benign Server
- Target Server
- Router
- 6 Cores of CPU
- At least 32 GB of RAM
- At least 200GB of storage spaces
You need to prepare follow Setup tutorial:
- adapters of each VM
- 10 VMs we provide
- Controller Machine (more than 8GB of RAM)
IP
: 192.168.56.111hostname
: controller-machinepassword
: qsefthukAdapter 1
: Host-Only adapter
- Data Logger Server
IP
: 192.168.56.121hostname
: data-logger-machinepassword
: qsefthukAdapter 1
: Host-Only adapter
- Vulnerable Client
IP
: 192.168.56.151hostname
: vulnerable-machinepassword
: qsefthukAdapter 1
: Host-Only adapter
- Non Vulnerable Client 1
IP
: 192.168.56.141hostname
: non-vulnerable-machine-1password
: qsefthukAdapter 1
: Host-Only adapter
- Non Vulnerable Client 2
IP
: 192.168.56.142hostname
: non-vulnerable-machine-2password
: qsefthukAdapter 1
: Host-Only adapter
- Attacker Server
IP
: 192.168.56.131hostname
: attacker-serverpassword
: qsefthukAdapter 1
: Host-Only adapter
- Malicious Client
IP
: 192.168.56.161hostname
: malicious-clientpassword
: qsefthukAdapter 1
: Host-Only adapter
- Target Server
IP
: 192.168.56.181hostname
: metasploitable3-ub1404password
: qsefthukAdapter 1
: Host-Only adapter
- Benign Server
IP
: 192.168.56.171hostname
: metasploitable3-ub1404password
: qsefthukAdapter 1
: Host-Only adapter
- Router
Adapter 1
: Host-Only adapterAdapter 2
: NAT
- Import 10 provided VMs into VirtualBox:
Import from VMs_Links and check the informations are all correct (VMs_Information).
- Check network adapter of each VM we provided (follow VMs_Information):
Right click on the VM 🡪 Setting 🡪 Network 🡪 Adapter 🡪 chooseHost-Only Ethernet Adapter
- Set Host-Only Ethernet Adapter on your host OS: Open network adapter settings on your host OS 🡪 Right click VirtualBox Host-Only Network adapter 🡪 content 🡪 IPv4 content, then type in the following informations
- IP Address:
192.168.56.1
- Netmask: 255.255.255.0/24
- IP Address:
- Check VMs we provided: Startup VMs 🡪 Settings 🡪
- Network 🡪 Choose Ethernet wired botton 🡪 IPv4 🡪 Manual
- IP Address: follow VMs_Information
- Netmask: 255.255.255.0/24
- Gateway:
192.168.56.2
- DNS: 8.8.8.8, 8.8.4.4 (turn off Automatic botton)
- About 🡪 Software Updates 🡪 Updates 🡪 Automatically check for updates 🡪 Never
- Network 🡪 Choose Ethernet wired botton 🡪 IPv4 🡪 Manual
- Clone and set the Repository on Controller machine: Open terminal and then type in the following commands\
git clone https://github.com/masjohncook/CREMEv2.git
sudo chown -R controller-machine:controller-machine CREMEv2/
sudo chmod -R 777 CREMEv2
cd CREMEv2
chmod +x setup.sh setup_tools.sh run_creme.sh
source ./setup_tools.sh
cd CREMEv2
./setup.sh
- Turn on all your machines (10 Machines)
- Login to your
Controller
cd CREMEv2/
🡪./run_creme.sh
- Access the controll interface using your Host OS Browser
http://192.168.56.111:8000
- You should use a
local network
in your testbed, not a public network. Because in the scanning phase of the attack, we assume we don't know the vulnerable clients, so we will scan in the network (with subnet mask 24) then try to find the vulnerable clients (similar to real attacks). You may get into some trouble if using the public network. - If you would like to rerun several times, you can
take a snapshot
before running, then back to that snapshot to rerun again. The reason is that we already finished configuring some services, so if we reconfigure them again, it may have some behaviors different from the first time - If you try to run, but the error messages showed in the Dash Board indicate that you can't connect to any VM, just check if the VMs_Information are all correct, then try to type in the cmd
systemctl restart ssh
on the VM you can't connect to. - If you want to check the Tmux messages when CREMEv2 is running, open a terminal and type in
tail -f CREMEv2/celery.log
, then you can check the last 10 messages in Tmux.