This repository provides examples of Azure Functions which process events from Event Hubs and store them in Cosmos DB.
Each implementation provides 2 functions:
BatchEventsHubToCosmosDB
processes events in batches and stores them in Cosmos DB in one go.
It consumes events from the Consumer Group identified by EVENT_HUB_BATCH_CONSUMER_GROUP
and stores processed events in the Cosmos DB container identified by COSMOSDB_BATCH_EVENTS_CONTAINER
.
IndividualEventsHubToCosmosDB
processes events individually, 1 execution per event.
It consumes events from the Consumer Group identified by EVENT_HUB_INDIVIDUAL_CONSUMER_GROUP
and stores processed events in the Cosmos DB container identified by COSMOSDB_INDIVIDUAL_EVENTS_CONTAINER
.
App setting | Description |
---|---|
EVENT_HUB_CONNECTION_STRING | Event Hubs connection string |
EVENT_HUB_NAME | Name of the hub |
EVENT_HUB_BATCH_CONSUMER_GROUP | Name of the consumer group for batch processing |
EVENT_HUB_INDIVIDUAL_CONSUMER_GROUP | Name of the consumer group for individual event processing |
COSMOSDB_CONNECTION_STRING | Cosmos DB connection string |
COSMOSDB_DATABASE | Cosmos DB database name |
COSMOSDB_BATCH_EVENTS_CONTAINER | Name of the container for batch-processed events |
COSMOSDB_INDIVIDUAL_EVENTS_CONTAINER | Name of the container for individually-processed events |
You can use Azure Functions Core tools to deploy functions to Azure.
# Replace $FUNCTION_APP_NAME with the name of the Function App
func azure functionapp publish $FUNCTION_APP_NAME
You can use the Data generator or Events Hub to generate new events from the Yellow Taxi
data set and test the functions in this project.