meilisearch/charabia

Refactor normalizers

ManyTheFish opened this issue · 0 comments

Today creating a normalizer is way harder than creating a segmenter, this is mainly due to the char map, a necessary field to manage highlights.

Technical Approach

Refactor the Normalizer trait to implement a normalize_str and normalize_char method that takes a Cow<str> in parameter and return a Cow<str>. All the char map creation should be done in a function calling these two methods.