Azure/azure-cosmosdb-spark

Missing data when reading data from CosmosDB changefeed

Opened this issue · 0 comments

Hi,

We are using azure-cosmosdb-spark to read data from cosmosdb changefeed.

Today, we got nothing from cosmosdb changefeed eventhough there are data inserted into cosmosdb.

Here is our configuration
`readConfig = {
"Endpoint": cosmos_host,
"Masterkey": cosmos_key,
"Database": "TicketMaster",
"Collection": "Tickets",
"ReadChangeFeed": "true",
"ChangeFeedQueryName": "Tickets-ChangeFeed",
"ChangeFeedStartFromTheBeginning": "false",
"InferStreamSchema": "false",
"ChangeFeedCheckpointLocation": checkpoint_cosmosdb_dir
}

changeFeed = (spark.readStream.format("com.microsoft.azure.cosmosdb.spark.streaming.CosmosDBSourceProvider").options(**readConfig).load())`

Is there anyway, we can reset the checkpoint value to reload missing data from previous checkpoint?

Thanks and regards,
Di Truong