The aim of this project is to create a fully automated personalized manga/manhwa newsletter integrated with WhatsApp and emailing services.
STAGE | STATUS |
---|---|
Single page manganelo scraper | ✓ |
Pipeline automated scraper with single page scraper | ✓ |
Multithreaded queue for storing queries | ✗ |
Sentiment analysis of text in comments | ✓ |
Sentiment analysis of memes in comments | ✗ |
Text summarization model | ✓ |
Connection with MySQL DB | ✓ |
Single page newsletter image generation(url sample) | ✓ |
WhatsApp cloup API'S webhook | ✗ |
Fully concurrent final WA client | ✗ |
Flask API between client and DB | ✗ |
Emailing services | ✗ |
Improve Logging | ✗ |
To run this project, you will need to add the following environment variables to your .env file
DB_TYPE
DB_USER
DB_PASS
DB_HOST
DB_NAME
You don't need the environment variables for simple testing as mentioned below.
Clone the project
git clone git@github.com:Kashyapdevesh/Manga-Newsletter.git
Go to the project directory
cd Manga-Newsletter
Prepare and activate virtual environment
virtualenv venv
source venv/bin/activate
Install dependencies
pip install -r requirements.txt
Currently the project is under developement and the instructions will be simulatneously updated.
At present state we can check the functioning of modules listed in milestones.
You can generate single page newsletter sample for the manga of your choice by following the instructions below:
-
Select the manga of your choice from https://m.manganelo.com/www
-
Replace variable url with chosen url in line no. 242 in page_scraper.py
-
Run the script
python3 page_scraper.py
You can view some pre-prepared samples at link
Contributions are always welcome!
If you have any feedback, please reach out to us at devesh.btech.cs19@iiitranchi.ac.in