mayankt18/glugle

One of the projects used in the second edition of Ten Days of Code by GNU/Linux Users' Group, NIT Durgapur, It is a search engine with functionalities of crawling the web pages and storing them. It makes use of MongoDB to store the data found in the pages. It also has an web interface to search the crawled pages.

Quick Links

• How to run the project

  Documentation

  • Introduction
  • Setup (Linux / Windows)
  • Mongo DB setup
  • Crawler
  • Flask Guide
  • Web application
  • Query Processing
  • Ranking the searches

Search engine made using Python and flask

Project Setup

1. Clone the git in a folder
2. Make a virtual environment in the folder

For Windows

pip install virtualenv
cd project_folder
virtualenv env
.\env\Scripts\activate

For Linux

pip install virtualenv #for version 2 and below 
pip3 install virtualenv #for version 3
cd project file
virtualenv env
source ./env/bin/activate

3. Install requirements

pip3 install -r requirements.txt

4. Start the crawler

python ./crawler/spider.py #for version 2 and below
python3 ./crawler/spider.py #for version 3

5. Start the application

python ./app/app.py #for version 2 and below
python3 ./app/app.py #for version 3