A Django web app that extracts and parses the HTML of an AliExpress.com product query page. The user receives a CSV file of all product info from the page and can save their data in a PostgreSQL database.
Django | Beautiful Soup (BS4) | PostgreSQL | Selenium | Bootstrap CSS | JQuery | HTML
- All functionality dealing with web scraping, using Beaufitul Soup (for retrieving the page's html) and Selenium (via Google Chrome webdriver), is located in webscrape.py
- Parsing the page's html using Beautiful Soup (BS4) is located in data_parser.py
- Functionality converting the parsed data into a CSV file is located in csv_handling.py
- All PostgreSQL database handling is located in db_handling.py
- Authenticated users have the option to save their data in a PostgreSQL database and can email their CSV file to themselves.
-
Without user authentication, any user can type a product name in the search bar, choose on the slider how many products from the query page they want included in the CSV file, and check the option of having Free Shipping information available in the CSV file.
-
Clicking the "Collect Data" button will web scrape an aliexpress.com query page, using the user's query from the search bar.
-
Once the HTML of the aliexpress.com page is successfully parsed, a CSV file can be downloaded by the user by clicking the "Download CSV" button.
-
With user authentication, the user has the option to email their CSV file to themselves by clicking the "Email CSV" button.
-
The authenticated user can also save thier CSV data in a PostgreSQL database and access their data at a later time, utilizing the "History" section.