There are an accompanying set of blog posts detailing the development process and underpinnings of the pipeline. Feel free to check them out if you're so inclined, but they're in no way required reading to use the tool.
Check out recon-pipeline's readthedocs entry for some more in depth information than what this README provides.
Automatic installation tested on kali 2019.4 and Ubuntu 18.04
There are two primary phases for installation:
- prior to cmd2 being installed
- everything else
First, the manual steps to get cmd2 installed in a virtual environment are as follows (and shown below), starting with pipenv
apt install pipenv
sudo apt install python3-pip
pip install --user pipenv
echo "PATH=${PATH}:~/.local/bin" >> ~/.bashrc
bash
git clone https://github.com/epi052/recon-pipeline.git
cd recon-pipeline
pipenv install cmd2
Once manual installation of cmd2 is complete, the recon-pipeline
shell provides its own install
command (seen below). A simple install all
will handle all installation steps.
Ubuntu-18.04 Note: You may consider running
sudo -v
prior to running./recon-pipeline.py
.sudo -v
will refresh your creds, and the underlying subprocess calls during installation won't prompt you for your password. It'll work either way though.
Command execution is handled through the recon-pipeline
shell (seen below).
The pipeline expects a file that describes the target's scope to be provided as an argument to the --target-file
option. The target file can consist of domains, ip addresses, and ip ranges, one per line.
tesla.com
tesla.cn
teslamotors.com
...
Some bug bounty scopes have expressly verboten subdomains and/or top-level domains, for that there is the --exempt-list
option. The exempt list follows the same rules as the target file.
shop.eu.teslamotors.com
energysupport.tesla.com
feedback.tesla.com
...
The backbone of this pipeline is spotify's luigi batch process management framework. Luigi uses the concept of a scheduler in order to manage task execution. Two types of scheduler are available, a local scheduler and a central scheduler. The local scheduler is useful for development and debugging while the central scheduler provides the following two benefits:
- Make sure two instances of the same task are not running simultaneously
- Provide visualization of everything that’s going on
While in the recon-pipeline
shell, running install luigi-service
will copy the luigid.service
file provided in the
repo to its appropriate systemd location and start/enable the service. The result is that the central scheduler is up
and running easily.
The other option is to add --local-scheduler
to your scan
command from within the recon-pipeline
shell.
- @aringo for his help on the precursor to this tool
- @kernelsndrs for identifying a few bugs after initial launch