redis/redis-vl-python

Expose show_progress_bar option for Hugging Face vectorizer

Closed this issue · 2 comments

Currently, hf.embed() and hf.embed_many() always display the tqdm progress bar while building embeddings, even if it's just for one embedding.

# Embed a sentence
hf = HFTextVectorizer(model="sentence-transformers/all-MiniLM-L6-v2")
test = hf.embed("This is a test sentence.")

Ask: expose show_progress_bar parameter of the underlying HuggingFace model, so one can choose if to display a progress bar.

model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
model.encode(["This is a test sentence."], show_progress_bar=False)

So one can do:

# Embed a sentence
hf = HFTextVectorizer(model="sentence-transformers/all-MiniLM-L6-v2")
test = hf.embed("This is a test sentence.", show_progress_bar=False)

We can accomplish this by exposing optional **kwargs on the .embed() + .embed_many() (and async variants). Each vectorizer would be responsible for parsing and passing through kwargs as needed. @antonum is this something you might be able to contribute if you're looking for a little OSS action ;) ?

Solved thanks to #236