X Files
A scrapy project for scraping declassified documents in the CIA.
Getting Started
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.
Prerequisites
- python3.6+
- pip
- python3-venv
Installing
- Unix Systems using make
-
create virtual enviroment and install the dependencies.
make
-
activate enviroment
source .venv_py/bin/activate
- Manual mode
- Create virtual enviroment
python3 -m venv .venv_py - Activate
source .venv_py/bin/activate - Opgrade pip (optional)
python3 -m pip install --upgrade pip - Install dependencies
pip install -r requirements.txt requirements-dev.txt
Built With
- scrapy - The framework for extracting data
License
This project is licensed under the MIT License.