GPU Memory issue in using Faiss index

[Amit-GH]

I am able to use flat_retrieve.py for smaller files but not for keys.txt file which has 100M sentences. For that I am trying to use the pre-trained FAISS index. I downloaded 100M_1GPU_16GB which is supposed to fit in 1 16GB GPU. I am using a 32GB memory GPU but still getting OOM errors. Here is the stack trace (deleted some lines to make it short).

Reading FAISS index
 - index: data/100M_1GPU_16GB.faiss.idx
 - found 100000000 sentences of dim 256
 - setting nbprobe to 32
 - transfer index to 1 GPUs 
Traceback (most recent call last):
  File "src/faiss_retrieve.py", line 37, in <module>
    index = IndexLoad(args.index, args.nprobe, args.gpu)
...
RuntimeError: Error in virtual void* faiss::gpu::StandardGpuResourcesImpl::allocMemory(const faiss::gpu::AllocRequest&) at /home/conda/feedstock_root/build_artifacts/faiss-split_1636459943780/work/faiss/gpu/StandardGpuResources.cpp:452: Error: 'err == cudaSuccess' failed: StandardGpuResources: alloc fail type IVFLists dev 0 space Device stream 0x5633af889030 size 606208 bytes (cudaMalloc error out of memory [2])

[Amit-GH]

I am able to use the index by just keeping them in CPU i.e., using --gpu "False".