Integrate with rust-bert
Opened this issue · 1 comments
brainless commented
As a user I would like to use local libraries to generate embedding. On their website rust-bert is introduced as:
Rust-native state-of-the-art Natural Language Processing models and pipelines. Port of Hugging Face’s Transformers library, using tch-rs or onnxruntime bindings and pre-processing from rust-tokenizers.
We can use this for classification, summarization or other utilities as well in the future.