alibaba/havenask-federation

fed向有两个向量字段的索引写入数据时容易把内存打满导致hang住

Closed this issue · 0 comments

case如下

# 创建索引
PUT multi_vector_test
{
  "settings": {
    "number_of_replicas": 0,
    "number_of_shards": 1,
    "index.engine":"havenask"
  },
  "mappings": {
    "properties": {
      "field1":{
        "type":"vector",
        "dims":"2",
        "similarity": "l2_norm"
      },
      "field2":{
        "type":"vector",
        "dims":"2",
        "similarity": "dot_product"
      }
    }
  }
}

# 写入数据
PUT multi_vector_test/_bulk
{"index":{"_id":1}}
{"field1":[1.1, 1.1], "field2":[0.6, 0.8]}
{"index":{"_id":2}}
{"field1":[2.2, 2.2], "field2":[0.8, 0.6]}
{"index":{"_id":3}}
{"field1":[3.3, 3.3], "field2":[1.0, 0.0]}
{"index":{"_id":4}}
{"field1":[4.4, 4.4], "field2":[0.0, 1.0]}
{"index":{"_id":5}}
{"field1":[5.5, 5.5], "field2":[0.6, 0.8]}
{"index":{"_id":6}}
{"field1":[6.6, 6.6], "field2":[0.8, 0.6]}
{"index":{"_id":7}}
{"field1":[7.7, 7.7], "field2":[1.0, 0.0]}
{"index":{"_id":8}}
{"field1":[8.8, 8.8], "field2":[0.0, 1.0]}
{"index":{"_id":9}}
{"field1":[9.9, 9.9], "field2":[0.6, 0.8]}
{"index":{"_id":10}}
{"field1":[10.01, 10.01], "field2":[1.0, 0.0]}

使用 sar -hr 1可以看到内存明显上涨
image

暂时屏蔽SearchIT.testMultiKnnQuery这个测试。