Not working with cyrillic strings

[romansidorov]

Hi!

I noticed that the generator returns an empty string if the input data is in cyrillic.

Could you please fix it? Thanks in advance!

[itssamtaylor]

I've personally never dealt with Cyrillic strings but I can definitely work on it.

Are you able to post a couple example strings and their expected outputs just so I have some test cases to use?

Thanks

[romansidorov]

I've personally never dealt with Cyrillic strings but I can definitely work on it.

Are you able to post a couple example strings and their expected outputs just so I have some test cases to use?

Thanks

Sure, thank you much :)

The subject is that with a cyrillic name the generator do not working properly. Lets use name Роман and try to generate a username:

use TaylorNetwork\UsernameGenerator\Generator;

return (new Generator())->generate('Роман'); # empty string will be returned

Now I tricked this way:

use Illuminate\Support\Str;
use TaylorNetwork\UsernameGenerator\Generator;

return (new Generator())->generate(Str::slug('Роман')); # 'roman' returned

[itssamtaylor]

@romansidorov

So I think I've addressed the problem in #40
Can you do do me a favour and clone that branch and test to make sure it is working as intended before I publish it?

You will need to add update the allowed_characters in the config to 'А-Яа-яA-Za-z'

Let me know if its all working and I'll publish the release.

Thanks

[romansidorov]

@romansidorov

So I think I've addressed the problem in #40
Can you do do me a favour and clone that branch and test to make sure it is working as intended before I publish it?

You will need to add update the allowed_characters in the config to 'А-Яа-яA-Za-z'

Let me know if its all working and I'll publish the release.

Thanks

Not working 🤷‍♂️

I've made a quick check – it seems there is a problem with encoding somewhere

[itssamtaylor]

Okay so question from your previous example above, if you run

use TaylorNetwork\UsernameGenerator\Generator;

$config = [
    'allowed_characters' => 'А-Яа-яA-Za-z ',
];

return (new Generator($config))->generate('Роман');  # currently returns 'роман'  

Is that the desired outcome? Or should it return 'roman' ?

[itssamtaylor]

Okay so I added an option to 'convert_to_ascii' in the config.

use TaylorNetwork\UsernameGenerator\Generator;

$config = [
    'allowed_characters' => 'А-Яа-яA-Za-z ',
    'convert_to_ascii' => false,
];

return (new Generator($config))->generate('Роман');  # returns 'роман'  

Or by sticking with just the defaults

use TaylorNetwork\UsernameGenerator\Generator;
return (new Generator())->generate('Роман');  # returns 'roman'  

I've just added pushed that to the #40 PR if you want to test it again

[itssamtaylor]

@romansidorov
After a bunch of testing I think the encoding problem has been fixed.

I believe that with these config options it should work for you now.

$g = new Generator([
    'convert_to_ascii' => false,
    'allowed_characters' => '\p{Cyrillic}\p{Latin}\s ',
]);

Let me know if that solved it, thanks

[itssamtaylor]

I've pushed release 2.6 that I believe addresses this issue. Feel free to open another issue or let me know if you're still experiencing issues.