The project involves data extraction, transformation, and loading (ETL) processes, as well as security features like data encryption. It also follows software design principles like DRY and SOLID.
- Data extraction from Mercado Libre API.
- Data transformation and processing using Python libraries like Pandas.
- Secure storage with data encryption.
- ETL (Extract, Transform, Load) pipeline for handling large datasets.
- Abides by DRY and SOLID software design principles.
- Utilizes the Factory pattern in authentication and user control.
To use the Data Integration with Meli API project, follow these steps:
-
Clone this repository to your local machine.
-
Configure the necessary credentials in the
.env
file. Make sure to provide values forDATABASE_URL
,CLIENT_ID
,CLIENT_SECRET
,CODE
, andREDIRECT_URI
. -
Activate your virtual environment with
venv\Scripts\activate
. -
Run the application with
python main.py
. -
Execute the
Base.metadata.create_all
command to create the necessary databases on the first run. -
The user inputs a search term, and the application accesses the Mercado Libre API to predict the category. Then, it retrieves the filters specific to that category and presents them to the user.
Before running the application, ensure that you have installed the required dependencies by using the following command:
pip install -r requirements.txt
Additionally, set the environment variables in the .env file with the credentials for your Mercado Libre application.
Python Pandas SQLAlchemy Fernet for data encryption
Contributions to this project are welcome. If you'd like to contribute, please follow these steps:
Fork the repository on GitHub.
Clone your forked repository to your local machine.
Create a branch for your changes.
Make your changes and commit them.
Push your changes to your forked repository.
Open a pull request in the main repository and provide a detailed description of your changes.
This project is licensed under the MIT License. Please refer to the LICENSE.md file for more information.
If you have questions or comments about this project, feel free to reach out:
Developer: Carlos Pérez Küper
Email: carlosperezkuper@gmail.com
Special thanks to the open-source developer community and Mercado Libre for providing a robust API that enables this project.