/PyScrapperServer

This script is a Python Scrapper controlled via a Web Interface which uses Bottle; BeautifulSoup 4 is used for scrapping EBooks off Websites which host them for free.

Primary LanguagePythonGNU General Public License v3.0GPL-3.0

PyScrapperServer

This script is a Python Scrapper controlled vai a Web Interface which uses Bottle; BeautifulSoup 4 is used for scrapping EBooks off Websites which host them for free.

Preface This project was entirely created on a Friday when I called in sick from College. This project provided me an introduction to Web Servers, Web Frameworks, Scrapping in Python and the Bottle Framework.

Introduction The code is run on a Raspberry Pi connected to the local WiFi connection, preferably using a Static IP. The user accesses the Web Interfaces hosted on the Pi and then pastes the link of the EBook that he wishes to download. The Ebook is then scrapped off the Link provided by the user. It is further converted to PDF format for easy reading. tl;dr  Web Server which Scrapes the Web.

How to Install?

  1. Clone this Repo
  2. Install Dependencies
    sudo pip install reportlab requests bs4 python-dev install bottle
  3. sudo nano /etc/rc.local
  4. Add sudo python path_of_this_cloned_repo/BottleServer.py to the end of the file before exit 0 , to allow the server to run at boot.
  5. sudo reboot

Done! Now visit the IP Address of the Server example192.168.1.10:8080

Dependencies

  1. txt2pdf
  2. bottle
  3. reportlab
  4. Beautiful Soup
  5. requests

Conclusion This project successfully downloaded several Ebooks from Websites and so was a succcess.