castorini/hf-spacerini

Segment Open Text like Wikipedia into Passages

Opened this issue · 0 comments

Possible preprocessing feature: Preprocess unstructured text into passages possibly using Pygaggle segmentation
https://github.com/castorini/pygaggle/blob/master/pygaggle/data/segmentation.py