PyScrapperServer
This script is a Python Scrapper controlled vai a Web Interface which uses Bottle; BeautifulSoup 4 is used for scrapping EBooks off Websites which host them for free.
Preface This project was entirely created on a Friday when I called in sick from College. This project provided me an introduction to Web Servers, Web Frameworks, Scrapping in Python and the Bottle Framework.
Introduction The code is run on a Raspberry Pi connected to the local WiFi connection, preferably using a Static IP. The user accesses the Web Interfaces hosted on the Pi and then pastes the link of the EBook that he wishes to download. The Ebook is then scrapped off the Link provided by the user. It is further converted to PDF format for easy reading. tl;dr Web Server which Scrapes the Web.
How to Install?
- Clone this Repo
- Install Dependencies
sudo pip install reportlab requests bs4 python-dev install bottle
sudo nano /etc/rc.local
- Add
sudo python path_of_this_cloned_repo/BottleServer.py
to the end of the file beforeexit 0
, to allow the server to run at boot. sudo reboot
Done!
Now visit the IP Address of the Server example192.168.1.10:8080
Dependencies
Conclusion This project successfully downloaded several Ebooks from Websites and so was a succcess.