Refactor normalizers
ManyTheFish opened this issue · 0 comments
ManyTheFish commented
Today creating a normalizer is way harder than creating a segmenter, this is mainly due to the char map, a necessary field to manage highlights.
Technical Approach
Refactor the Normalizer
trait to implement a normalize_str
and normalize_char
method that takes a Cow<str>
in parameter and return a Cow<str>
. All the char map creation should be done in a function calling these two methods.