Azure Functions Event Hubs to Cosmos DB

Overview

This repository provides examples of Azure Functions which process events from Event Hubs and store them in Cosmos DB.

Each implementation provides 2 functions:

BatchEventsHubToCosmosDB

BatchEventsHubToCosmosDB processes events in batches and stores them in Cosmos DB in one go. It consumes events from the Consumer Group identified by EVENT_HUB_BATCH_CONSUMER_GROUP and stores processed events in the Cosmos DB container identified by COSMOSDB_BATCH_EVENTS_CONTAINER.

IndividualEventsHubToCosmosDB

IndividualEventsHubToCosmosDB processes events individually, 1 execution per event. It consumes events from the Consumer Group identified by EVENT_HUB_INDIVIDUAL_CONSUMER_GROUP and stores processed events in the Cosmos DB container identified by COSMOSDB_INDIVIDUAL_EVENTS_CONTAINER.

Configuration

App setting	Description
EVENT_HUB_CONNECTION_STRING	Event Hubs connection string
EVENT_HUB_NAME	Name of the hub
EVENT_HUB_BATCH_CONSUMER_GROUP	Name of the consumer group for batch processing
EVENT_HUB_INDIVIDUAL_CONSUMER_GROUP	Name of the consumer group for individual event processing
COSMOSDB_CONNECTION_STRING	Cosmos DB connection string
COSMOSDB_DATABASE	Cosmos DB database name
COSMOSDB_BATCH_EVENTS_CONTAINER	Name of the container for batch-processed events
COSMOSDB_INDIVIDUAL_EVENTS_CONTAINER	Name of the container for individually-processed events

Deployment

You can use Azure Functions Core tools to deploy functions to Azure.

# Replace $FUNCTION_APP_NAME with the name of the Function App
func azure functionapp publish $FUNCTION_APP_NAME

Testing

You can use the Data generator or Events Hub to generate new events from the Yellow Taxi data set and test the functions in this project.