alexklibisz/elastiknn

'failed to create query' : Expected field [Vector] to have type [elastiknn_dense_float_vector] but had [float]

akash-agr opened this issue · 2 comments

Hi,

Firstly, Thanks a lot for creating this amazing plugin. I went through your video tutorial and blogs. I have used the following code for mapping, Indexing, and Searching.

For Mapping

mapping_field = {
"dynamic" : False,
"properties": {
"Sentence": {
"type": "text",
},
"Vector": {
"type": "elastiknn_dense_float_vector",
"dimension": 384
},
},
"settings": {
"index": {
"number_of_shards": 1,
"elastiknn": True
}
}
}

res = es.indices.create(index=index_name, body=mapping_field, ignore=400)

For Indexing

sentence = "revenue growth was 10%"
model = SentenceTransformer('paraphrase-MiniLM-L6-v2')
embeddings = model.encode([query])
embeddings = np.asarray(embeddings)
embeddings = embeddings[0].tolist()

json_object = {
"Sentence" : sentence,
"Vector": embeddings,
}

es.index(index=index_name, body=json_object)

For Search

query = "growth"
model = SentenceTransformer('paraphrase-MiniLM-L6-v2')
embeddings = model.encode([query])
embeddings = np.asarray(embeddings)
embeddings = embeddings[0].tolist()

exactNN_body = {
"size": 10,
"query": {
"elastiknn_nearest_neighbors": {
"vec": embeddings,
"field": "Vector",
"similarity": "angular",
"model": "exact",
}
},
}
res = es.search(index = index_name, body=exactNN_body)

I am getting the below error, Can you kindly help?

elasticsearch.exceptions.RequestError: RequestError(400, 'search_phase_execution_exception', 'failed to create query: Expected field [Vector] to have type [elastiknn_dense_float_vector] or [elastiknn_sparse_bool_vector] but had [float]')

Hi, thanks for the kind words :)

I think your mapping is slightly incorrect and the create index request might be silently failing.

The "vector" section of the mapping should look like this:

"Vector": {
"type": "elastiknn_dense_float_vector",
"elastiknn": { "dims": 784 }
}

More info here: https://elastiknn.com/api/#elastiknn_dense_float_vector-datatype

I suspect something like this is happening:

  1. The es.indices.create request is failing, but you're not checking the res status.
  2. The es.index request succeeds, but it's inferring an incorrect mapping.
  3. The es.search request tries to fetch the vector as an elastiknn vector, but it was created with the wrong mapping in step 2.

Hi, Thanks a lot. I did a stupid mistake and my mapping object was incorrect. I need to put "properties" inside the "mappings" field. I used below object and everything seems to be working.

mapping_field = {
"mappings": {
"properties": {
"Sentence": {
"type": "text",
},
"Vector": {
"type": "elastiknn_dense_float_vector",
"elastiknn": { "dims": 384 },
},
},
},
"settings": {
"index": {
"number_of_shards": 1,
"elastiknn": True
}
}
}