microsoft/kernel-memory

'Object reference not set to an instance of an object' when search result from AzureAISearch

Closed this issue · 7 comments

Context / Scenario

build memory as below

_memory = new KernelMemoryBuilder()
    .WithOpenAITextGeneration(new OpenAIConfig
    {
        APIKey = apiKey,
        TextModel = "gpt-4o",
    }, null, new HttpClient(handler))
   .WithOpenAITextEmbeddingGeneration(new OpenAIConfig
   {
       APIKey = apiKey,
       EmbeddingModel = "text-embedding-ada-002"
   }, null, false, new HttpClient(handler))
   .WithAzureAISearchMemoryDb(new AzureAISearchConfig
   {
       APIKey = JsonHelper.ReadJsonKey("AI", "Test", "AzureAISearchKey"),
       Auth = AzureAISearchConfig.AuthTypes.APIKey,
       Endpoint = JsonHelper.ReadJsonKey("AI", "Test", "AzureAISearchEndPoint")
   })
    .Build<MemoryServerless>();

invoke SearchResult

relevant = await _memory.SearchAsync(parm.Question, limit: 3, minRelevance: 0.5, index: "GovernmentNews");

What happened?

earlier version is working well, updated to latest version just now, then encounter exception:

'Object reference not set to an instance of an object.'

StackTrace as below

at OpenAI.Embeddings.EmbeddingCollection..ctor(IReadOnlyList1 data, String model, InternalCreateEmbeddingResponseObject object, EmbeddingTokenUsage usage, IDictionary2 serializedAdditionalRawData)
at OpenAI.Embeddings.EmbeddingCollection.DeserializeEmbeddingCollection(JsonElement element, ModelReaderWriterOptions options)
at OpenAI.Embeddings.EmbeddingCollection.FromResponse(PipelineResponse response)
at OpenAI.Embeddings.EmbeddingClient.d__6.MoveNext()
at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.d__65`1.MoveNext()
at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.d__68.MoveNext()
at Microsoft.SemanticKernel.AI.Embeddings.TextEmbeddingGenerationExtensions.d__0.MoveNext()
at Microsoft.KernelMemory.MemoryDb.AzureAISearch.AzureAISearchMemory.d__9.MoveNext()
at Microsoft.KernelMemory.MemoryDb.AzureAISearch.AzureAISearchMemory.d__9.System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult(Int16 token)
at Microsoft.KernelMemory.Search.SearchClient.d__7.MoveNext()
at Microsoft.KernelMemory.Search.SearchClient.d__7.MoveNext()
at MyProject.NewsDetailService.d__13.MoveNext() in C:\MyProject\Service\NewsDetailService.cs:line 426

Importance

a fix would make my life easier

Platform, Language, Versions

.NET8
Kernel Memory version:0.73.240906.1
Semantic Kernel version:1.18.2

Relevant log output

Please see 'What happened'
dluc commented

Without injecting a custom HTTP client, the code works fine, right?

What does the handler do?

Without injecting a custom HTTP client, the code works fine, right?

What does the handler do?

the handler is to change the endpoint instead of the OpenAI official endpoint.
same code works when I downgrade the kernel memory to the previous version.

2024-09-11_135632
the code works when I use version as above.
The handle is to change the endpoint to another gateway server which will send request to Open AI official endpoint.

code of handler

public class OpenAIHttpClientHandler : HttpClientHandler
{
    private readonly string _uri;

    public OpenAIHttpClientHandler(string uri) => _uri = uri.TrimEnd('/');

    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        UriBuilder uriBuilder;
        if (request.RequestUri?.LocalPath == "/v1/chat/completions")
        {
            uriBuilder = new UriBuilder(_uri + "/v1/chat/completions");
            request.RequestUri = uriBuilder.Uri;
        }
        else if (request.RequestUri?.LocalPath == "/v1/embeddings")
        {
            uriBuilder = new UriBuilder(_uri + "/v1/embeddings");
            request.RequestUri = uriBuilder.Uri;
        }

        return await base.SendAsync(request, cancellationToken);
    }
}

Hi @dluc would you have a look?

RAG_Test.zip
Please refer to this.

base64 issue from endpoint.