microsoft/winget-cli-restsource

Filter only searches request the entire database to then do filtering in azure function memory

jamespik opened this issue · 1 comments

Brief description of your issue

        // Create feed options for inclusion search: -1 so we can get all matches in inclusion, then filter down.
        FeedOptions feedOptions = new FeedOptions
        {
            ResponseContinuationTokenLimitInKb = CosmosConnectionConstants.ResponseContinuationTokenLimitInKb,
            EnableCrossPartitionQuery = true,
            MaxItemCount = AllElements,
            RequestContinuation = null,
        };

        if (manifestSearchRequest.FetchAllManifests || (manifestSearchRequest.Inclusions == null && manifestSearchRequest.Query == null))
        {
            IQueryable<CosmosPackageManifest> query = this.cosmosDatabase.GetIQueryable<CosmosPackageManifest>(feedOptions);
            IDocumentQuery<CosmosPackageManifest> documentQuery = query.AsDocumentQuery();
            ApiDataPage<CosmosPackageManifest> apiDataDocument = await this.cosmosDatabase.GetByDocumentQuery<CosmosPackageManifest>(documentQuery);
            manifests.AddRange(apiDataDocument.Items);
        }

Filter only searches will result in attempting to load the entire cosmos db database into azure function memory. Winget client uses filter only search for a variety of user command line input.

This implementation should be revised to not attempt to load the entire cosmos db database into memory.

Steps to reproduce

Expected behavior

Actual behavior

Environment

Revisions to the filter composition logic largely addressed this issue a few months ago.