/serverless-msk-workshop

re:invent 2022 workshop for building a Serverless Apache Kafka Data Pipeline

Primary LanguagePythonMIT LicenseMIT

Building a Serverless Apache Kafka Data Pipeline

Serverless services allow us to build applications without having to worry about the underlying infrastructure. This allows developers to avoid provisioning, scaling, and managing resource utilization. In this workshop we will build a serverless data pipeline using Amazon MSK Serverless which enables data to be consumed from many different client applications to accomplish downstream tasks such as dashboards and analytics.

Attendees will will learn how to

  • Start a Serverless MSK Cluster
  • Deploy a Kafka Client Container based Lambda Function
  • ingest data from various sources into a Kafka Client
  • Consume data for downstream tasks such as Quicksight Dashboards

This repo contains the code needed to send data to the Kafka cluster.

Follow the workshop at INSERT_LINK