[Question] Running Kernel Memory as a service in Asp.Net

[Robban1980]

Context / Scenario

I am looking at running Kernel Memory as a service in Asp.Net API.

Question

I took a look at the Service.AspNetCore code and saw that IKernelMemory is registered as a singleton. With an API i was wondering why we should register IKernelMemory as a singleton instead of as a scoped service?

[dluc]

The recommended deployment uses asynchronous queues to execute tasks in the background, to avoid blocking user requests and to deal with long running jobs.

For example, when uploading a file it might take seconds/minutes to process, depending on the pipeline in use, depending on external services bandwidth, and in case of transient errors requiring the service to retry.

To avoid blocking TCP/HTTP requests while documents are processed, the ingestion is designed to run as a task outside the scope of a HTTP request. Using a singleton allows KM components to work. It's also perfectly safe to use KM as a singleton.

On the other hand, if for some reason you cannot use singletons, then I would recommend configuring KM in Serverless mode. Everything will run synchronously, blocking HTTP requests while ingestion is running, and once a request is complete the KM instance can be disposed.