Add support for different chunking strategies
dgarnitz opened this issue · 3 comments
dgarnitz commented
Right now vectorflow only supports chunking by the an exact length passed in by the user, independent of how the content is structured. Ideally the user should be able to specify whether they want this strategy, or by paragraph, by sentence, etc.
EteimZ commented
Hi @dgarnitz I will love to work on this issue. I bet a chunking technique similar to langchain's RecursiveCharacterTextSplitter could be of great use to vectorflow.