This project provides a simple and effective way of extracting data from an auction website, storing it in a secure location, and retrieving it whenever needed. With the use of AWS technologies such as S3, Athena, and AWS CLI, the entire process is streamlined and optimized for maximum efficiency.
Details:
The web scraper is designed to automate the process of gathering data from an auction website. With just one click, the scraper extracts data from the website and compiles it into a CSV file.
To streamline the data storage process, the CSV file is automatically uploaded to an AWS S3 bucket using AWS CLI. This process is designed to make it easy to access the data and also ensure its security.
The S3 bucket is also connected to AWS Athena, which enables the running of queries within AWS to retrieve any necessary data. The use of Athena ensures quick and efficient data retrieval, which is essential for data-driven decision-making.