Expose show_progress_bar option for Hugging Face vectorizer
Closed this issue · 2 comments
antonum commented
Currently, hf.embed() and hf.embed_many() always display the tqdm progress bar while building embeddings, even if it's just for one embedding.
# Embed a sentence
hf = HFTextVectorizer(model="sentence-transformers/all-MiniLM-L6-v2")
test = hf.embed("This is a test sentence.")
Ask: expose show_progress_bar parameter of the underlying HuggingFace model, so one can choose if to display a progress bar.
model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
model.encode(["This is a test sentence."], show_progress_bar=False)
So one can do:
# Embed a sentence
hf = HFTextVectorizer(model="sentence-transformers/all-MiniLM-L6-v2")
test = hf.embed("This is a test sentence.", show_progress_bar=False)
tylerhutcherson commented
We can accomplish this by exposing optional **kwargs on the .embed() + .embed_many() (and async variants). Each vectorizer would be responsible for parsing and passing through kwargs as needed. @antonum is this something you might be able to contribute if you're looking for a little OSS action ;) ?
tylerhutcherson commented
Solved thanks to #236