Implement unic-ucd-segment and unic-segment
Closed this issue · 0 comments
behnam commented
References:
- UAX #29: Unicode Text Segmentation http://www.unicode.org/reports/tr29/
Defines Char Properties:
- Grapheme_Cluster_Break
- Word_Break
- Sentence_Break
Needs Char Properties:
- General_Category
- Alphabetic
Related Char Properties that are not needed in algorithm implementation, therefore can be made optional feature in unic-ucd-segment, or implemented in a separate component:
- Grapheme_Base
- Grapheme_Extend
Tests:
Similar crates: