For punctuation and white space - replace with a character rather than empty string

Question

For punctuation and white space - replace with a character rather than empty string

Closed this issue 4 years ago · 4 comments

If you remove these characters entirely, two words could be joined creating a false positive match. For example me rest would match merest which is a different word. Using a special character as a white space replacement would solve the issue. don't forget to replace any occurrence of 2 white spaces with one as well.

Answer 1 · 2020-07-10T19:41:42.000Z

@goneall Okay I will change that.

Answer 2 · 2020-07-13T13:21:45.000Z

@goneall Done the changes. Please review whenever you are free.

Answer 3 · 2020-07-13T16:02:55.000Z

Almost - there is still a problem if there are multiple whitespace characters. For example, this[5 spaces]is should normalize to this*is rather than this*****is.

You could use a regular expression replacing \s+ with the special character.

Answer 4 · 2020-07-13T16:30:52.000Z

Oh okay didn't see that. Did the change.