Data API Builder and Semantic Kernel with Aspire

This Repository contains the source code for the Data API Builder and Semantic Kernel, a solution that allows users to create APIs for their data sources and to query them using a semantic kernel. The tool is built using .NET Aspire.

Requirements

To start the project, you need to have the following tools installed:

.NET Aspire
Azure Developer CLI
Container Engine (e.g. Docker)

To configure and test Data API Builder, you need to install the dab cli.

Configuring variables

To get started with Data API Builder, refer to the official documentation. I am going to refer to Data API Builder as dab from now on.

To run the project locally, you need to configure few variables in the appsettings:

the database connection string in the appsettings.Development.json file of DataAPIBuilder.AI.AppHost project:

"ConnectionStrings": {
    "sqldb": "<CONNECTION_STRING>"
}

The Endpoint and API Key for Azure OpenAI in the appsettings.Development.json file of DataAPIBuilder.AI.Web project:

"OpenAI": {
    "Endpoint": "https://<AOAI_NAME>.openai.azure.com/",
    "ApiKey": "<API_KEY>"
}

Notes on Deployment

The solution can be deployed using the Azure Developer CLI - which I will from know on call azd. To learn how azd and .NET Aspire work together, refer to the official documentation.

NOTE: The database won't be deployed by azd, as it is only referenced as a connection string in the AppHost project.

After running the azd init command, we cannot proceed to run azd up because there are a few to change in the Azure Container Apps templates. Therefore, we need to run the following command:

azd infra synth

NOTE: To run azd infra synth we need to enable the feature since it is still in alpha by running azd config set alpha.infraSynth on.

When running azd infra synth, the folders containing the biceps and the yaml templates for the Azure Container Apps will be created in two infra folders. From now on, every time we run azd up, the infra folders will be used for the deployment and the changes made in the AppHost won't be considered until we run azd infra synth again.

After running the azd infra synth command, we can follow these steps:

address the issue with the sqldb connection string parameter
azd up to provision the infrastructure and deploy the code
retrieve the name of the Volume Mount created by azd
address the issue with the configuration file
azd up to apply the changes to the dab's Azure Container App configuration

Avoiding the issue with sqldb connection string parameter

Since we not declaring the Database with the AppHost, the template for the dab's Azure Container App won't show the correct value for sqldb connection string. To fix this, we need to add the following variable in .azure/distributed-dab-sk/.env:

AZURE_SQLDB = "<CONNECTION_STRING>"

After adding the variable, we can update the value for the sqldb connection string in the dab template file with the following value:

    secrets:
      - name: connectionstrings--sqldb
        value: '{{ .Env.AZURE_SQLDB }}'

This will allow the Azure Container App to have the correct value for the sqldb connection string.

Configuring Data API Builder to find the configuration file

When the Data API Builder container starts, it will look for a configuration file, usually name dab-config.json.

In the AppHost project, Data API Builder is configured with a BindMount as follows:

var dabService = builder.AddContainer("dab", "mcr.microsoft.com/azure-databases/data-api-builder")
    .WithHttpEndpoint(targetPort: 5000, name: "http")
    .WithBindMount(@"./dab-config.json", "/App/dab-bm0/dab-config.json", true)
    .WithArgs("--ConfigFileName", "./dab-bm0/dab-config.json")
    .WithReference(sql)
    .WithOtlpExporter()
    .PublishAsContainer();

From release 1.9.4, azd cli supports the binding mount for Aspire. Unfortunatlly, the binding mount is not working as expected, and the configuration file is not being found by the Data API Builder container.

We need to change the configuration for the volume bind in the dab template file AND get rid of the azd.operations.yaml file that azd cli creates. This file should be in charge of uploading the file to the shared volume on Azure, but it is not able to find the file nor it is able to upload it.

The following changes need to be made in the dab template file:

volumeMounts:
  - volumeName: <VOLUME_MOUNT_NAME>
    mountPath: /App/<FILE_SHARE_NAME>
args:
  - --ConfigFileName=./<FILE_SHARE_NAME>/dab-config.json

After running azd up, you will of course need to upload the configuration file to the shared volume.