yooper/php-text-analysis

CharFilter not working?

gvanto opened this issue · 2 comments

My tokenDoc->toArray gives output below after applying CharFilter, I was expecting to not see single-character elements still in there?

array(15) {                                                                                                                             
  [0] =>                                                                                                                                
  string(1) "i"                                                                                                                         
  [1] =>                                                                                                                                
  string(1) "a"                                                                                                                         
  [2] =>                                                                                                                                
  string(7) "plumber"

Done a bit of testing, this seems to work:

class CharFilter implements ITokenTransformation
{ 
    public function transform($word)
    {
        return trim(preg_replace("/ \D /", "", " $word "));
    }
}

Thank you, I will investigate further this weekend.