glue-crawler

There are 3 repositories under glue-crawler topic.

  • SourabhSinghRana/real-time_crypto_data_pipeline_using_kafka

    I am using confluent Kafka cluster to produce and consume scraped data. In this project, I've created a real-time data pipeline that utilizes Kafka to scrape, process, and load data onto S3 in JSON format. With a producer-consumer architecture, I ensure that the data is in the right format for loading onto S3 by performing minor transformations

    Language:Python28308

  • DieGit0/windfarm

    Data Engineering project using data streaming produced by python applications, ETL process and availability for ad-hoc SQL queries in the AWS cloud

    Language:Jupyter Notebook1100

  • DieGit0/data_realtime_-_batch_analytics

    Data Streaming and Batch processing using AWS Services

    Language:Python0100