aws-samples/genai-quickstart-pocs

sentence-transformers not installed

carlosbravoa opened this issue · 5 comments

carlosbravoa commented

I just followed up the quickstart, created the venv, and installed the requirements.txt file
As soon as you launch it and start prompting you get an error with sentence-transformers not installed.

So I installed it with: pip install sentence-transformers but ended with a plethora of dependencies installed that are irrelevant to the case such as CUDA (I don't even have an NVIDIA GPU), safetensors, scikitlearn and huggingface-hub. If the models run on Bedrock and this is just APIs, there is something wrong somewhere.

Thanks!

pip install sentence-transformers
...
...
...
Successfully installed huggingface-hub-0.22.2 joblib-1.3.2 mpmath-1.3.0 networkx-3.2.1 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.19.3 nvidia-nvjitlink-cu12-12.4.99 nvidia-nvtx-cu12-12.1.105 safetensors-0.4.2 scikit-learn-1.4.1.post1 scipy-1.12.0 sentence-transformers-2.6.1 sympy-1.12 threadpoolctl-3.4.0 tokenizers-0.15.2 torch-2.2.2 transformers-4.39.2 triton-2.2.0
carlosbravoa commented

And after that, it asked me to install chromadb.

ryanadoty commented

Hey @carlosbravoa , do you mind sharing what sample you are trying to run when you encountered this issue?

kaiyanchen commented

i tried the amazon-bedrock-chat-poc, it works smoothly on M1 with my miniconda env. you'd better to point out which example you tried. @carlosbravoa

carlosbravoa commented

Sorry for the oversight! It was: Amazon-Bedrock-GenAI-Dynamic-Prompting-Explained-POC
Thanks!

ryanadoty commented

Hey @carlosbravoa can you try pulling down the latest version of the repo, I believe I addressed this issue :)

Just to clarify the reason you need sentence transformers and chromadb is for two similar but different things. Some of the PoCs are using a dynamic prompting technique, leveraging a semantic similarity search based on a database of stored prompts.

When the app is initially started, it uses sentence-transformers and pulls down an embeddings model from hugging face. This model is then used to create embeddings of all your sample prompts, and stores them in a vector database (ChromaDB) in memory.

Each time you make a query, an embedding of your query is created, you do a KNN search against chromaDB, and it returns the three most semantically similar prompts, and injects those into the final prompt along with your question into Bedrock.

If you wanted, you could replace the hugging face embeddings model with an amazon bedrock based embeddings model, but since the one provided is free, we thought it was the best fit.

Hopefully this clarifies things!