nathadriele
MSc in Information Science | Data Engineer | Python | NLP | MLOps | QA Engineer | Researcher | Bioinformatics
VindiBrazil
Pinned Repositories
airflow-kpi-insertion-pipeline
The script automates the collection and insertion of KPIs related to transaction time and storage usage in a Data Warehouse, using Apache Airflow. It calculates the time elapsed since the last transaction and the percentage of storage usage, recording this data periodically in specific tables.
airflow-tableau-ec2-maintenance
This project automates weekly maintenance for a Tableau server on an EC2 instance using Apache Airflow, ensuring optimal performance and reliability. The DAG performs disk cleanup and sends notifications with the results via AWS SNS.
AWS-DMS-task-restart-and-status-checker
The AWS DMS Task Restart and Status Checker is a Python script designed to restart various AWS Database Migration Service (DMS) tasks and check their status. This script leverages AWS SDK (Boto3) and Mage.ai for DMS task integration and management, ensuring efficient and reliable task handling.
biophenotype-rag
This project implements a RAG (Retrieval-Augmented Generation) application to answer questions about phenotypes using biological and genomic data. The pipeline integrates information retrieval with response generation via language models (LLM), facilitating accurate analysis of phenotypic data.
llm-zoomcamp
Project LLMs
med-neo4j-graphq
mlops-zoomcamp
Project MLOps
mlops-zoomcamp-project-paris-price-house
MLOps Paris Housing Price Prediction.
ner-hcm
vercel-app-mlops-zoomcamp-project-paris-price-house
[Vercel-app] MLOps Paris Housing Price Prediction.
nathadriele's Repositories
nathadriele/mlops-zoomcamp-project-paris-price-house
MLOps Paris Housing Price Prediction.
nathadriele/airflow-kpi-insertion-pipeline
The script automates the collection and insertion of KPIs related to transaction time and storage usage in a Data Warehouse, using Apache Airflow. It calculates the time elapsed since the last transaction and the percentage of storage usage, recording this data periodically in specific tables.
nathadriele/airflow-tableau-ec2-maintenance
This project automates weekly maintenance for a Tableau server on an EC2 instance using Apache Airflow, ensuring optimal performance and reliability. The DAG performs disk cleanup and sends notifications with the results via AWS SNS.
nathadriele/AWS-DMS-task-restart-and-status-checker
The AWS DMS Task Restart and Status Checker is a Python script designed to restart various AWS Database Migration Service (DMS) tasks and check their status. This script leverages AWS SDK (Boto3) and Mage.ai for DMS task integration and management, ensuring efficient and reliable task handling.
nathadriele/biophenotype-rag
This project implements a RAG (Retrieval-Augmented Generation) application to answer questions about phenotypes using biological and genomic data. The pipeline integrates information retrieval with response generation via language models (LLM), facilitating accurate analysis of phenotypic data.
nathadriele/cnpj-data-pipeline
The CNPJ Data ETL Pipeline is designed to automate the download, processing, and storage of public CNPJ data from the Brazilian Federal Revenue. The pipeline is built with Mage.ai and AWS S3 to ensure efficient data management and scalability.
nathadriele/datamart-tables-data-type-validation
This project is a Data Engineering solution implemented to validate the data types of columns in PostgreSQL tables in a DataMart. It aims to validate whether the data stored in tables conforms to the expected data types, improving data integrity and reliability.
nathadriele/datasets
nathadriele/llm-zoomcamp
Project LLMs
nathadriele/med-neo4j-graphq
nathadriele/mlops-zoomcamp
Project MLOps
nathadriele/nathadriele
nathadriele/ner-hcm
nathadriele/NypymeAndroid
nathadriele/README.md
nathadriele/vercel-app-mlops-zoomcamp-project-paris-price-house
[Vercel-app] MLOps Paris Housing Price Prediction.
nathadriele/DMS-CDC-task-status-validator
Automated script developed in Mage.ai to monitor and validate the status of AWS DMS CDC tasks, ensuring data integrity and synchronization. Sends notifications for any detected validation issues.
nathadriele/DMS-missing-or-duplicate-data-validation-script
Contains a Python script designed to validate data replication tasks in AWS Database Migration Service (DMS). The script checks for potential issues such as missing or duplicate data in the tables being replicated.
nathadriele/machine-learning-zoomcamp
Project ML
nathadriele/redshift-to-s3-unload-dag
This Airflow DAG automates the process of extracting data from an Amazon Redshift database and unloading it to Amazon S3 in Parquet format. It runs daily, exporting data from the previous day based on a specified query.
nathadriele/S3-folder-cleanup
This script automates cleaning up a specific folder in an S3 bucket, deleting all objects within it. It uses secure AWS credentials and is built on the Mage.ai platform. Additionally, the roadmap includes approach handling and logging for greater robustness and monitoring.
nathadriele/star-wars-data-visualization
O objetivo é explorar diferentes tipos de visualizações de dados aplicadas ao universo Star Wars, utilizando técnicas para extrair insights dos personagens, espécies e suas relações.