nmslib/hnswlib

Question: Is index build time unstable?

wu3227834 opened this issue · 2 comments

wu3227834 commented

Hello,

I'm testing the performance of HNSW (HNSWlib) on a 2c8g environment.
I found that there is a difference in the time to construct the index under the same index parameters.

Data Set：Cohere 1M, 768 Dim
Index Parameter：M:30,ef_construction:360、ef_search:100
Construction time(s)：4974.660、4570.640、5673.193、4870.791

Is HNSWlib index build time unstable?

Thank you!

yurymalkov commented

Hi @wu3227834,

It is likely that the node you are running is virtual and is shared across multiple users.
So, the performance that you are measure depends on the programs that other users are running.
If you get a full node, that should go away

wu3227834 commented

Hi @yurymalkov Thank u for ur reply
I encapsulated hnswlib and used docker-compose to limit the 2c8g resource.
I’m not sure if it’s a problem with my program.
Theoretically, with the same index parameters, under the same environment resources and the same training set, the index construction time is stable, right?

Thank you!