Wrong total calculation in search if the index is out of sync and contains more elements than the actual catalog

We have sometimes the behaviour that our search index (azure search) is a "littlebit" out of sync and contains more elements than exists in the current catalogs. This leads to a wrong total calculation for the search result and to some weired pagination issues related to that wrong total count calculation.

The total is used from the result from azure search:

vc-module-catalog/src/VirtoCommerce.CatalogModule.Data/Search/Indexing/CatalogIndexedSearchService.cs

Line 45 in 49623eb

result.TotalCount = response.TotalCount;

but afterwards the ConvertDocuments method will reduce the result set to the products that are really existent in the catalogs:

vc-module-catalog/src/VirtoCommerce.CatalogModule.Data/Search/Indexing/CatalogIndexedSearchService.cs

Line 89 in 49623eb

ReduceSearchResults(itemsMap.Values.Where(v => v != null), criteria);

So we have e.g. a search query with the criteria as follows:

ProductIndexedSearchCriteria
                {
                    CatalogId = catalogId,
                    Outline = categoryId,
                    PriceRange = new NumericRange
                    {
                        Lower = minPrice,
                        Upper = maxPrice,
                        IncludeLower = true,
                        IncludeUpper = true
                    },
                    Sort = "title-ascending",
                    SearchPhrase = "my search string",
                    Skip = 0,
                    Take = 100,
                    WithHidden = false
                });

and then we got back a total count e.g. of 40 and a count of the Items list of 20 because 20 products are existing in the index but are deleted out of the catalog.

Version info:

Browser version: doesn't matter
Platform version: v3.80.0
Module version: 3.68.0

Expected behavior

The total count should fit to the "real" total count and should maybe be reduced if the result set is lower than the total and the take is bigger than the total.

Task https://virtocommerce.atlassian.net/browse/VP-7805 has been created

I added it to next sprint, ~due date Nov 11, 2021

@corneliusmunz Fix is ready. Release: https://github.com/VirtoCommerce/vc-module-catalog/releases/tag/3.74.0

@dvvkgd I will check the fix and will close the issue afterwards

@OlegoO , @dvvkgd can you please add the v3.74.0 version of the catalog module to the https://github.com/VirtoCommerce/vc-modules/commits/master/modules_v3.json Otherwise i could only update it in downloading the zip file and could not use the update feature in the UI.

Hi, @corneliusmunz! Modules updated. Can you check?

@Vectorfield4 Perfect :-) Many thanks. I will update it now and will check the fixed issue

I have checked the fixed version and it seems the issue is fixed. Many thanks!