This guide will walk you through setting up and running the Princeton Scraping program using a Python virtual environment (venv) and installing all necessary requirements.
Ensure you have Python 3.6 or newer installed on your system. You can check your Python version by running:
python3 --version
-
Clone the Repository: First, clone the repository to your local machine.
-
Create a Virtual Environment: Navigate to the root directory of the cloned repository and create a virtual environment named
venv
:python3 -m venv venv
-
Activate the Virtual Environment:
-
On Windows, run:
.\venv\Scripts\Activate.ps1
-
On macOS and Linux, run:
source venv/bin/activate
You should now see
(venv)
at the beginning of your command line prompt, indicating that the virtual environment is activated. -
-
Install Requirements: Install the required Python packages using
pip
:pip install -r requirements.txt
Before running the program, ensure you have a logs
folder in the root directory of the repository for the log files to be stored. If it doesn't exist, create it:
mkdir logs
To run the program, execute:
python3 main.py
This will start the scraping process, and progress will be logged to the logs
directory.
Once you're done, you can deactivate the virtual environment by running:
deactivate
This will return you to your global Python environment.