podigee/device_detector

Encoding::CompatibilityError caused by "é" in User-Agent

Closed this issue · 2 comments

alokhmutov commented

Hi! I've encountered Encoding::CompatibilityError in my project caused by non-standard characters in a custom User-Agent.

Environment: Ruby 3.2.0, DeviceDetector 1.1.0.
Environment: Ruby 3.0.2, DeviceDetector 1.1.0.

User's agent is Mon User-Agent personnalisé.

The error:
Encoding::CompatibilityError (incompatible encoding regexp match (UTF-8 regexp with ASCII-8BIT string))

It seems to be caused by user_agent =~ r[:regex] in device.rb's regex_find

device_detector/lib/device_detector/device.rb

Line 1560 in 609256f

regex_list.find { |r| user_agent =~ r[:regex] }

I see that it already rescues RegexpError. I was wondering if Encoding::CompatibilityError stands to be rescued here as well. I've forked the gem and made the change for my project and it no longer raises an error with this User-Agent, removing the last char instead.

Any thoughts on this?

Trace:

device_detector (1.1.0) lib/device_detector/device.rb in each at line 1560
device_detector (1.1.0) lib/device_detector/device.rb in find at line 1560
device_detector (1.1.0) lib/device_detector/device.rb in regex_find at line 1560
device_detector (1.1.0) lib/device_detector/device.rb in block in matching_regex at line 1535
device_detector (1.1.0) lib/device_detector/parser.rb in block in from_cache at line 92
device_detector (1.1.0) lib/device_detector/memory_cache.rb in get_or_set at line 38
device_detector (1.1.0) lib/device_detector/parser.rb in from_cache at line 92
device_detector (1.1.0) lib/device_detector/device.rb in matching_regex at line 1531
device_detector (1.1.0) lib/device_detector/parser.rb in regex_meta at line 31
device_detector (1.1.0) lib/device_detector/device.rb in type at line 1505
device_detector (1.1.0) lib/device_detector.rb in device_type at line 67
alokhmutov commented

Reproducible with a rails minimal app: https://github.com/alokhmutov/test-project-dd-105.

jrochkind commented

I am also running into Encoding::CompatibilityError, although in my case it may be with actually invalid-for-UTF-8 bytes in user-agent?

I'm not sure if my logging is actually sufficient to capture the exact bytes in the user-agent, they end up looking like 1 ????%2527%2522 in my logs, but I'm not sure that's really the bytes.

My stack trace ends at the same place though:

https://github.com/podigee/device_detector/blob/develop/lib/device_detector/device.rb#L1560

I agree with @alokhmutov's diagnosis -- my guess is maybe ruby 3.2 starts raising a different exception here than previous rubies?