milvus-io/milvus

[Bug]: [Nightly] Search failed reporting metric type not match with no metric type specified

Opened this issue · 8 comments

NicoYuan1986 commented

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version: eb1602a
- Deployment mode(standalone or cluster):standalone
- MQ type(rocksmq, pulsar or kafka):    rocksmq
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Search failed reporting metric type not match with no metric type specified.

[pytest : test] status = error_code: UnexpectedError
[pytest : test] reason: "fail to search on QueryNode 2: worker(2) query failed: metric type not match: inv...o search on QueryNode 2: worker(2) query failed: metric type not match: invalid parameter[expected=IP][actual=COSINE]"
[pytest : test] 
[pytest : test] 
[pytest : test]     def check_status(status: Status):
[pytest : test]         if status.code != 0 or status.error_code != 0:
[pytest : test] >           raise MilvusException(status.code, status.reason, status.error_code)
[pytest : test] E           pymilvus.exceptions.MilvusException: <MilvusException: (code=65535, message=fail to search on QueryNode 2: worker(2) query failed: metric type not match: invalid parameter[expected=IP][actual=COSINE])>
[pytest : test]

Expected Behavior

pass

Steps To Reproduce

No response

Milvus Log

  1. link: https://jenkins.milvus.io:18080/blue/organizations/jenkins/Milvus%20Nightly%20CI(new)/detail/2.4/27/pipeline/154/
  2. log: artifacts-milvus-standalone-ms-24-27-py-n-27-e2e-logs.tar.gz
  3. failed time: [pytest : test] [gw5] [ 59%] FAILED testcases/test_search.py::TestCollectionSearch::test_search_expression_all_data_type[200-True]

Anything else?

I try locally.
Here is index params:

[2024-09-20 14:58:03 - INFO - ci_test]: {'index_type': 'FLAT', 'params': {}, 'metric_type': 'COSINE'} (client_base.py:308)
[2024-09-20 14:58:03 - DEBUG - ci_test]: (api_request)  : [Collection.create_index] args: ['multiple_vector_FLOAT16_VECTOR', {'index_type': 'FLAT', 'params': {}, 'metric_type': 'COSINE'}, 1200], kwargs: {'index_name': ''} (api_request.py:62)
[2024-09-20 14:58:04 - DEBUG - ci_test]: (api_response) : Status(code=0, message=)  (api_request.py:37)
[2024-09-20 14:58:04 - INFO - ci_test]: {'index_type': 'FLAT', 'params': {}, 'metric_type': 'COSINE'} (client_base.py:308)
[2024-09-20 14:58:04 - DEBUG - ci_test]: (api_request)  : [Collection.create_index] args: ['multiple_vector_BFLOAT16_VECTOR', {'index_type': 'FLAT', 'params': {}, 'metric_type': 'COSINE'}, 1200], kwargs: {'index_name': ''} (api_request.py:62)
[2024-09-20 14:58:05 - DEBUG - ci_test]: (api_response) : Status(code=0, message=)  (api_request.py:37)
[2024-09-20 14:58:05 - INFO - ci_test]: {'index_type': 'SPARSE_INVERTED_INDEX', 'metric_type': 'IP', 'params': {'drop_ratio_build': 0.2}} (client_base.py:305)
[2024-09-20 14:58:05 - DEBUG - ci_test]: (api_request)  : [Collection.create_index] args: ['multiple_vector_SPARSE_FLOAT_VECTOR', {'index_type': 'SPARSE_INVERTED_INDEX', 'metric_type': 'IP', 'params': {'drop_ratio_build': 0.2}}, 1200], kwargs: {'index_name': ''} (api_request.py:62)
[2024-09-20 14:58:07 - DEBUG - ci_test]: (api_response) : Status(code=0, message=)  (api_request.py:37)

Here is search params:

[2024-09-20 14:58:09 - INFO - ci_test]: {'params': {'nprobe': 32}} (test_search.py:3370)
[2024-09-20 14:58:09 - DEBUG - ci_test]: (api_request)  : [Collection.search] args: [[{47: 0.838443097692992, 37: 0.7926239812446345, 9: 0.7535619979872448, 29: 0.8517887050573193, 28: 0.37161304324529243, 35: 0.47459472085869303, 20: 0.07028837966307355, 10: 0.5288637257480088, 45: 0.7861001162843653, 1: 0.6851123371640158, 8: 0.7676660036141528, 61: 0.8645696442762907, 13: 0.5151......, kwargs: {'_async': False} (api_request.py:62)
[2024-09-20 14:58:10 - ERROR - pymilvus.decorators]: RPC error: [search], <MilvusException: (code=65535, message=fail to search on QueryNode 1: worker(1) query failed: metric type not match: invalid parameter[expected=IP][actual=COSINE])>, <Time:{'RPC start': '2024-09-20 14:58:09.755976', 'RPC error': '2024-09-20 14:58:10.607422'}> (decorators.py:146)

It should use default index metric type rather than report error.

yanliang567 commented

I believe this is a test issue as we set IP metric for sparse vector, while trying to search it by cosine. @binbinlv I thin this was somehow caused by init_collection_general(), please help to take a look
/assign @binbinlv
/unassign

binbinlv commented

I believe this is a test issue as we set IP metric for sparse vector, while trying to search it by cosine. @binbinlv I thin this was somehow caused by init_collection_general(), please help to take a look /assign @binbinlv /unassign

working on it.

binbinlv commented

I think this is not a test case issue.

Milvus should use the metric type defined when creating index automatically if not specifying metric type again in search params.

But here it did not use the correct metric type defined in the index params when not specifying it in search params for sparse vector.

It used "COSINE" while set the "IP" when creating index for sparse vector.

binbinlv commented

This case is good in master-latest image, while it is failed in 2.4 latest (2.4-20240923-b92daa15)

zhengbuqian commented

this is not specific to sparse float vector.

If we have 2 vector indexes with different types of metric, and when searching if we omit the metric type in the search request, the inferred metric type will always be the first index.

search_params = {} # did not specify metric_type

# col1: [vec_ip, vec_cosine]:
col1.search(query, "vec_ip", search_params) # ok
col1.search(query, "vec_cosine", search_params) # not ok: expected cosine, actual ip

working on a fix

zhengbuqian commented

the bug was introduced by #36279: collection metric was used if not provided in search params

if req.Req.MetricType == "" {
	req.Req.MetricType = collection.GetMetricType()
}

/assign @SimFG
/unassign

can you help fix this?

binbinlv commented

/unassign