Azure/spark-cdm-connector

Feature to override the file location for an entity's partition in model.json

Opened this issue · 1 comments

Hi,

We have a CDM endpoint which has two data paths for same entity. One file gets updated in NRT basis (<5mins SLA) whose directory structure is "containername/entityname/filename.csv" and other (<1hr SLA) whose directory structure is "containername/entityname/Snapshot/filename.csv". The model.json is pointing to the latter. But we would like to consume data from the former path due to NRT SLA requirements for our data ingestion. Is there a feature that is available in the library to override the model.json file location for an entity?

Thanks.

HI,

Apologies for the delay response. The connector doesn't have a feature to override file locations for a model.json file.