cloud-dataflow

tosun-si/asgarde

Asgarde allows simplifying error handling with Apache Beam Java, with less code, more concise and expressive code.

mercari/DataflowTemplate

Mercari Dataflow Template

ostelco/ostelco-core

Cloud-native Telco BSS hosted in GCP K8s with standalone Diameter to gRPC gateway. Rule Engine using Neo4j graphs. Analytics Events sent to GCP BigData (Dataflow+BigQuery) via PubSub. It's awesome!

tosun-si/pasgarde

Asgarde allows simplifying error handling with Apache Beam Python, with less code, more concise and expressive code.

hayatoy/dataflow-tutorial

Cloud Dataflow Tutorial for Beginners

datastacktv/apache-beam-batch-processing

Public source code for the Batch Processing with Apache Beam (Python) online course

tosun-si/midgard

Midgard is a wrapper on Beam Kotlin, allowing more concise and expressive code. It removes Beam boilerplate code and proposes more Functional Programming style

GoogleCloudPlatform/dataflow-pubsub-dedup

Niangmohamed/GCP-Professional-Data-Engineer-Learning-Path

I got Google Cloud Certified. I have what it takes to leverage Google Cloud technology. Here my certification: https://www.credential.net/ee1bd2d6-fdb0-4037-8a8d-9afae3d79c86.

dongma/springcloud-dataflow

build streaming apps on spring cloud dataflow platform

tosun-si/dataflow-java-ci-cd

Project showing a CI CD pipeline for Dataflow Java with Flex Template and Cloud Build

viveknaskar/cloud-dataflow-with-memorystore

Cloud Dataflow pipeline that reads the file from Cloud Storage and processes and outputs in the memory store.

goatcheesesaladwithpeanutoildressing/hashtagsbattle

IIM-DEVOPS - (demo) Real-time Twitter's hashtags analytics... fully managed by Google Cloud

zdenulo/upload-data-datastore-dataflow

goatcheesesaladwithpeanutoildressing/meetuplytics

Realtime Meetup "Répondez, s'il vous plaît" (RSVPs) analytics built upon Apache-Beam - Streaming Processing

viveknaskar/google-dataflow-redis-example

Cloud dataflow pipeline code that processes data from a cloud storage bucket, transforms it and stores in Google's highly scalable, reduced latency in-memory database, memorystore which is an implementation of Redis.

esakik/data-engineering-essentials

Samples related to data engineering, e.g. spark, embulk, airflow, etc.

imrenagi/beam-bank-trx-analytics

Experiment in using windowing and trigger for more accurate streaming analytics with Apache Beam

maximoleinyk/pubsub-filter

GKE Replacement for PubSub-to-PubSub Cloud Dataflows in GCP

rafaelgbo/bmf-data-pipeline

Efficient Python data pipeline leveraging Apache Beam and Google Cloud Dataflow to update a Bucket with data concerning daily prices of instruments extracted from BMF website, serving as input for other data pipelines. The code generates a dataflow template, which is then scheduled to run periodically using Cloud Scheduler + Cloud Functions.

saiteja09/CloudDataFlow-ETL-OnPremise-JDBC

ETL OnPremises Data to Google BigQuery using Google Cloud Data Flow via JDBC driver

sanketrs/implementation-of-modern-data-engineering-architecture-with-fabric_analytics

Building a next-generation hybrid data pipeline architecture that combines the power of Microsoft Fabric, Azure Cloud, and Power BI. This pipeline is engineered to tackle the challenges of real-time data ingestion, multi-layered processing, and analytics, delivering business-critical insights.

seahrh/fraud-detection-dataflow

Working example of a real-time inference pipeline on GCP Cloud Dataflow

viveknaskar/cloud-dataflow-template-poc

Creating Cloud Dataflow template using Java for counting a number of words from a document.

mahesh-c-pathak/gcs_pipeline

Building a fully automated data Pipeline with Google Cloud Services