Segfaults in re2_regexp_error after libre2 is upgraded
stanhu opened this issue · 9 comments
stanhu commented
I'm not sure if this is an actual issue with the C extension or whether this just means libre2 needs to be compiled with -fPIC for this not to happen (will test this out). But whenever libre2 is updated (e.g. from v1.1 to v1.2 recently in https://github.com/Homebrew/homebrew-core/blob/40c227a5885561519827edecffd959f6316163d8/Formula/re2.rb#L4), it seems that re2 seg faults if error is called. Running gem pristine re2 fixes the problem, but I'm wondering if this seg fault is something we can avoid. A simple script that reproduces the problem:
require 're2'
regexp = RE2::Regexp.new("{", log_errors: false)
puts regexp.error unless regexp.ok?$ ruby re-test.rb
re-test.rb:4: [BUG] Segmentation fault at 0x0000000000000000
ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-darwin19]
-- Crash Report log information --------------------------------------------
See Crash Report log file under the one of following:
* ~/Library/Logs/DiagnosticReports
* /Library/Logs/DiagnosticReports
for more details.
Don't forget to include the above Crash Report log file in bug reports.
-- Control frame information -----------------------------------------------
c:0003 p:---- s:0012 e:000011 CFUNC :error
c:0002 p:0040 s:0008 E:000f78 EVAL re-test.rb:4 [FINISH]
c:0001 p:0000 s:0003 E:002140 (none) [FINISH]
-- Ruby level backtrace information ----------------------------------------
re-test.rb:4:in `<main>'
re-test.rb:4:in `error'
-- Machine register context ------------------------------------------------
rax: 0x00007fac19584c70 rbx: 0x00007fac098956e0 rcx: 0x0000000107cd12f0
rdx: 0x000000000000000c rdi: 0x0000000000000000 rsi: 0x000000000000000c
rbp: 0x00007ffee8386e10 rsp: 0x00007ffee8386e00 r8: 0x00007fac196d8d28
r9: 0x00007fac20127f90 r10: 0x00007fac1a086600 r11: 0x00007fac19507028
r12: 0x0000000000000000 r13: 0x00007fac098956e0 r14: 0x00007fac195862e0
r15: 0x00007fac09897850 rip: 0x0000000107cd1317 rfl: 0x0000000000010206
-- C level backtrace information -------------------------------------------
/Users/stanhu/.rbenv/versions/2.6.5/lib/libruby.2.6.dylib(rb_vm_bugreport+0x82) [0x107abc572]
/Users/stanhu/.rbenv/versions/2.6.5/lib/libruby.2.6.dylib(rb_bug_context+0x1d6) [0x10790b746]
/Users/stanhu/.rbenv/versions/2.6.5/lib/libruby.2.6.dylib(sigsegv+0x51) [0x107a21261]
/usr/lib/system/libsystem_platform.dylib(_sigtramp+0x1d) [0x7fff6d5235fd]
/Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/re2-1.1.1/lib/re2.bundle(_ZL16re2_regexp_errorm+0x27) [0x107cd1317]
/Users/stanhu/.rbenv/versions/2.6.5/lib/libruby.2.6.dylib(vm_call_cfunc+0x156) [0x107aaede6]
/Users/stanhu/.rbenv/versions/2.6.5/lib/libruby.2.6.dylib(vm_exec_core+0x33da) [0x107a959ba]
/Users/stanhu/.rbenv/versions/2.6.5/lib/libruby.2.6.dylib(rb_vm_exec+0xac4) [0x107aa9834]
/Users/stanhu/.rbenv/versions/2.6.5/lib/libruby.2.6.dylib(ruby_exec_internal+0xe6) [0x1079167c6]
/Users/stanhu/.rbenv/versions/2.6.5/lib/libruby.2.6.dylib(ruby_run_node+0x49) [0x107916639]
/Users/stanhu/.rbenv/versions/2.6.5/bin/ruby(main+0x5d) [0x107878f0d]
-- Other runtime information -----------------------------------------------
* Loaded script: re-test.rb
* Loaded features:
0 enumerator.so
1 thread.rb
2 rational.so
3 complex.so
4 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/x86_64-darwin19/enc/encdb.bundle
5 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/x86_64-darwin19/enc/trans/transdb.bundle
6 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/x86_64-darwin19/rbconfig.rb
7 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/compatibility.rb
8 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/defaults.rb
9 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/deprecate.rb
10 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/errors.rb
11 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/version.rb
12 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/requirement.rb
13 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/platform.rb
14 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/basic_specification.rb
15 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/stub_specification.rb
16 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/delegate.rb
17 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/uri/rfc2396_parser.rb
18 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/uri/rfc3986_parser.rb
19 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/uri/common.rb
20 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/uri/generic.rb
21 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/uri/file.rb
22 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/uri/ftp.rb
23 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/uri/http.rb
24 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/uri/https.rb
25 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/uri/ldap.rb
26 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/uri/ldaps.rb
27 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/uri/mailto.rb
28 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/uri.rb
29 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/specification_policy.rb
30 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/util/list.rb
31 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/x86_64-darwin19/stringio.bundle
32 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/specification.rb
33 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/exceptions.rb
34 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/util.rb
35 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/bundler_version_finder.rb
36 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/dependency.rb
37 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/core_ext/kernel_gem.rb
38 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/monitor.rb
39 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb
40 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/core_ext/kernel_warn.rb
41 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems.rb
42 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0/rubygems/path_support.rb
43 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/did_you_mean-1.3.0/lib/did_you_mean/version.rb
44 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/did_you_mean-1.3.0/lib/did_you_mean/core_ext/name_error.rb
45 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/did_you_mean-1.3.0/lib/did_you_mean/levenshtein.rb
46 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/did_you_mean-1.3.0/lib/did_you_mean/jaro_winkler.rb
47 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checker.rb
48 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/name_error_checkers/class_name_checker.rb
49 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/name_error_checkers/variable_name_checker.rb
50 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/name_error_checkers.rb
51 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/method_name_checker.rb
52 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/key_error_checker.rb
53 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/did_you_mean-1.3.0/lib/did_you_mean/spell_checkers/null_checker.rb
54 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/did_you_mean-1.3.0/lib/did_you_mean/formatters/plain_formatter.rb
55 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/did_you_mean-1.3.0/lib/did_you_mean.rb
56 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/re2-1.1.1/lib/re2.bundle
57 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/re2-1.1.1/lib/re2/scanner.rb
58 /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/re2-1.1.1/lib/re2.rb
[NOTE]
You may have encountered a bug in the Ruby interpreter or extension libraries.
Bug reports are welcome.
For details: https://www.ruby-lang.org/bugreport.html
[IMPORTANT]
Don't forget to include the Crash Report log file under
DiagnosticReports directory in bug reports.
Abort trap: 6
Sometimes it fails:
ruby(46630,0x11203cdc0) malloc: can't allocate region
:*** mach_vm_map(size=140414988976128, flags: 60000100) failed (error code=3)
ruby(46630,0x11203cdc0) malloc: *** set a breakpoint in malloc_error_break to debug
ruby(46630,0x11203cdc0) malloc: can't allocate region
:*** mach_vm_map(size=140414988976128, flags: 60000100) failed (error code=3)
ruby(46630,0x11203cdc0) malloc: *** set a breakpoint in malloc_error_break to debug
Traceback (most recent call last):
re-test.rb: failed to allocate memory (NoMemoryError)
An objdump -d of the bundle file shows:
00000000000032f0 __ZL16re2_regexp_errorm:
32f0: 55 pushq %rbp
32f1: 48 89 e5 movq %rsp, %rbp
32f4: 53 pushq %rbx
32f5: 50 pushq %rax
32f6: 48 89 fb movq %rdi, %rbx
32f9: be 0c 00 00 00 movl $12, %esi
32fe: e8 c9 30 00 00 callq 12489 <re2.cc+0x63cc>
3303: 48 8b 43 20 movq 32(%rbx), %rax
3307: 48 8b 00 movq (%rax), %rax
330a: 83 b8 80 00 00 00 00 cmpl $0, 128(%rax)
3311: 74 1e je 30 <__ZL16re2_regexp_errorm+0x41>
3313: 48 8b 78 78 movq 120(%rax), %rdi
3317: 0f b6 37 movzbl (%rdi), %esi
331a: 40 f6 c6 01 testb $1, %sil
331e: 75 1d jne 29 <__ZL16re2_regexp_errorm+0x4d>
3320: 48 ff c7 incq %rdi
3323: 48 d1 ee shrq %rsi
3326: 48 83 c4 08 addq $8, %rsp
332a: 5b popq %rbx
332b: 5d popq %rbp
332c: e9 49 31 00 00 jmp 12617 <re2.cc+0x647a>
3331: b8 08 00 00 00 movl $8, %eax
3336: 48 83 c4 08 addq $8, %rsp
333a: 5b popq %rbx
333b: 5d popq %rbp
333c: c3 retq
333d: 48 8b 77 08 movq 8(%rdi), %rsi
3341: 48 8b 7f 10 movq 16(%rdi), %rdi
3345: 48 83 c4 08 addq $8, %rsp
3349: 5b popq %rbx
334a: 5d popq %rbp
334b: e9 2a 31 00 00 jmp 12586 <re2.cc+0x647a>
stanhu commented
Compiling with -fPIC did not seem to solve this problem.
mudge commented
Hi @stanhu,
Thanks for reporting this.
I'm having trouble reproducing it. I tried the following on macOS 10.15.4 and Ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c):
- Install re2 20200303 from Homebrew using the following formula:
url "https://github.com/google/re2/archive/2020-03-03.tar.gz"
version "20200303"
sha256 "04ee2aaebaa5038554683329afc494e684c30f82f2a1e47eb62450e59338f84d"
# ...
bottle do
cellar :any
sha256 "abb07cfb9def8d6f25a0d5694f0f23ba44b5b500f2d45d25d27515f62802c01b" => :catalina
sha256 "92c4603fad274003e71699376b20a40c6e57c13f878774a894e28a9ef73295c1" => :mojave
sha256 "58ac31c06c851bc5632c3a0703c0d41e55d8b015229cf03de09ee4a8701d92ef" => :high_sierra
end- Installed version 1.2.0 of the re2 gem with
gem install re2 - Ran your test script in
pry:
~> pry -rre2
[1] pry(main)> regexp = RE2::Regexp.new("{", log_errors: false)
=> #<RE2::Regexp /{/>
[2] pry(main)> puts regexp.error unless regexp.ok?
=> nil- Upgraded re2 to 20200401 with the following formula and
brew upgrade re2:
url "https://github.com/google/re2/archive/2020-04-01.tar.gz"
version "20200401"
sha256 "98794bc5416326817498384a9c43cbb5a406bab8da9f84f83c39ecad43ed5cea"
# ...
bottle do
cellar :any
sha256 "430e7efee518f5235ea75afc039168ce38c5407b525301199ccbc6c698300bb8" => :catalina
sha256 "67eed1f38907d1d7318b5d95f035ca18ff6186d85ee690560b8b1cd387f8afd4" => :mojave
sha256 "f111b7c906ab090d9db56e1f183f94d02e8cf5ce521e4ae4ff8e64d974cb5cbb" => :high_sierra
end- Re-ran your test commands:
~> pry -rre2
[1] pry(main)> regexp = RE2::Regexp.new("{", log_errors: false)
=> #<RE2::Regexp /{/>
[2] pry(main)> puts regexp.error unless regexp.ok?
=> nilLooking at re2.bundle's links, they seem consistent between versions:
~/.gem/ruby/2.6.5/gems/re2-1.2.0/ext/re2> otool -L re2.bundle
re2.bundle:
/usr/local/opt/re2/lib/libre2.0.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 902.1.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)I know there is a breaking change coming in re2 (removing some deprecated APIs) so the SONAME has changed from 6 to 7, even upgrading to HEAD with Homebrew, your commands still pass for me:
[1] pry(main)> regexp = RE2::Regexp.new("{", log_errors: false)
=> #<RE2::Regexp /{/>
[2] pry(main)> puts regexp.error unless regexp.ok?
=> nilThe bigger picture question here is what is reasonable behaviour for the gem when the underlying library is upgraded? Especially if some internal re2 API changes (as RE2::Match did several years ago), I'm not sure what the gem can do if the linked library has shifted under its feet. We could avoid depending on the external library altogether and bundle re2 inside the gem (as Nokogiri does with libxml2) but this seems like a major, breaking change to the behaviour of the library.
Hopefully there's some smaller issue we can fix here.
stanhu commented
You may need to rewind further to https://github.com/Homebrew/homebrew-core/blob/549a5c81c67e401fd3c75339157b3b936ebbbbed/Formula/re2.rb.
The bigger picture question here is what is reasonable behaviour for the gem when the underlying library is upgraded? Especially if some internal re2 API changes (as RE2::Match did several years ago), I'm not sure what the gem can do if the linked library has shifted under its feet. We could avoid depending on the external library altogether and bundle re2 inside the gem (as Nokogiri does with libxml2) but this seems like a major, breaking change to the behaviour of the library.
That's a good question. We have a check for ffi and eventmachine that fails if the require step fails. Could we make re2 just fail fast on require 're2' if the underlying library changes? That would at least make the problem easier to diagnose.
mudge commented
Ah, that did the trick: compiling the gem against re2 2020-01-01 and then upgrading to 2020-04-01 causes the segfault when I call RE2::Regexp#error.
Looking at the diff between the two versions I see that there was a breaking ABI change and Google changed the soname from 0 to 6. (I see the latest release has changed again from 6 to 7).
However, it looks like Homebrew explicitly changes the soname of the library to 0 regardless of ABI version. Removing the lines from the re2 Formula that change the dylib ID and add symlinks stops Homebrew masquerading libre2.6 and libre2.7 as libre2.0 and causes the gem to fail fast as soon as you require it:
# Against re 2020-01-01 without the dylib ID changing or symlinks
~> gem install re2
Fetching re2-1.2.0.gem
Building native extensions. This could take a while...
Successfully installed re2-1.2.0
1 gem installed
~/.gem/ruby/2.6.5/gems/re2-1.2.0/ext/re2> otool -L re2.bundle
re2.bundle:
/usr/local/opt/re2/lib/libre2.0.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 902.1.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
# Upgrading to 2020-04-01 without the dylib ID changing or symlinks
~> brew upgrade re2
~/.gem/ruby/2.6.5/gems/re2-1.2.0/ext/re2> otool -L re2.bundle
re2.bundle:
/usr/local/opt/re2/lib/libre2.0.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 902.1.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
# Now libre2.0.dylib is entirely missing
~> ls -la /usr/local/opt/re2/lib
total 2168
drwxr-xr-x 7 mudge admin 224 2 May 10:47 .
drwxr-xr-x 9 mudge admin 288 2 May 10:47 ..
-r--r--r-- 1 mudge admin 348556 2 May 10:47 libre2.6.0.0.dylib
lrwxr-xr-x 1 mudge admin 18 2 May 10:47 libre2.6.dylib -> libre2.6.0.0.dylib
-r--r--r-- 1 mudge admin 755048 2 May 10:47 libre2.a
lrwxr-xr-x 1 mudge admin 18 2 May 10:47 libre2.dylib -> libre2.6.0.0.dylib
drwxr-xr-x 3 mudge admin 96 2 May 10:47 pkgconfig
# Try requiring re2 linked against the now-missing re2 version
~/.gem/ruby/2.6.5/gems/re2-1.2.0/ext/re2> pry -rre2
Traceback (most recent call last):
9: from /Users/mudge/.gem/ruby/2.7.1/bin/pry:23:in `<main>'
8: from /Users/mudge/.gem/ruby/2.7.1/bin/pry:23:in `load'
7: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/bin/pry:12:in `<top (required)>'
6: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/cli.rb:89:in `parse_options'
5: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/pry_class.rb:146:in `final_session_setup'
4: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/pry_class.rb:104:in `load_requires'
3: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/pry_class.rb:104:in `each'
2: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/pry_class.rb:105:in `block in load_requires'
1: from /Users/mudge/.rubies/ruby-2.7.1/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
/Users/mudge/.rubies/ruby-2.7.1/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- re2 (LoadError)
13: from /Users/mudge/.gem/ruby/2.7.1/bin/pry:23:in `<main>'
12: from /Users/mudge/.gem/ruby/2.7.1/bin/pry:23:in `load'
11: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/bin/pry:12:in `<top (required)>'
10: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/cli.rb:89:in `parse_options'
9: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/pry_class.rb:146:in `final_session_setup'
8: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/pry_class.rb:104:in `load_requires'
7: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/pry_class.rb:104:in `each'
6: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/pry_class.rb:105:in `block in load_requires'
5: from /Users/mudge/.rubies/ruby-2.7.1/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:156:in `require'
4: from /Users/mudge/.rubies/ruby-2.7.1/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:168:in `rescue in require'
3: from /Users/mudge/.rubies/ruby-2.7.1/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:168:in `require'
2: from /Users/mudge/.gem/ruby/2.7.1/gems/re2-1.2.0/lib/re2.rb:6:in `<top (required)>'
1: from /Users/mudge/.rubies/ruby-2.7.1/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
/Users/mudge/.rubies/ruby-2.7.1/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require': dlopen(/Users/mudge/.gem/ruby/2.7.1/gems/re2-1.2.0/lib/re2.bundle, 9): Library not loaded: /usr/local/opt/re2/lib/libre2.0.dylib (LoadError)
Referenced from: /Users/mudge/.gem/ruby/2.7.1/gems/re2-1.2.0/lib/re2.bundleLooking on Ubuntu, it seems the libre2-dev and libre2-5 package correctly suffix the shared libraries with the soname so I suspect this is a macOS and Homebrew issue only.
If I raise a pull request against Homebrew to stop overwriting the ABI version, this means the gem would fail fast if you try to require it when the underlying libre2 ABI version has changed and seems the most reasonable approach.
This wouldn't require any change to the gem itself.
What do you think?
mudge commented
This has now been merged into Homebrew core; could you please re-test with the latest formula (2020-05-01 revision 1) and let me know if it behaves as you expect?
Perhaps use the 2020-04-01 formula to install re2 with a soname of 0 (though it should be 6):
Then install and compile the gem against that version with gem install re2 and confirm it works as expected:
$ irb -rre2
> RE2('{').error
=> nil
Then upgrade to the latest, fixed version of the re2 Formula (which has a soname of 7) and try to run the same command again:
~> irb -rre2
/Users/mudge/.rubies/ruby-2.7.1/lib/ruby/2.7.0/irb/init.rb:290: warning: LoadError: dlopen(/Users/mudge/.gem/ruby/2.7.1/gems/re2-1.2.0/lib/re2.bundle, 9): Library not loaded: /usr/local/opt/re2/lib/libre2.0.dylib
Referenced from: /Users/mudge/.gem/ruby/2.7.1/gems/re2-1.2.0/lib/re2.bundle
Reason: image not found - /Users/mudge/.gem/ruby/2.7.1/gems/re2-1.2.0/lib/re2.bundle
stanhu commented
Maybe I should create a separate issue, but does gem pristine re2 work for you now? I am seeing:
$ gem pristine re2
Restoring gems to pristine condition...
Building native extensions. This could take a while...
ERROR: While executing gem ... (Gem::Ext::BuildError)
ERROR: Failed to build gem native extension.
current directory: /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/re2-1.1.1/ext/re2
/Users/stanhu/.rbenv/versions/2.6.5/bin/ruby -I /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/2.6.0 -r ./siteconf20200502-7576-t86j84.rb extconf.rb
checking for -lstdc++... yes
checking for stdint.h... yes
checking for rb_str_sublen()... yes
checking for -lre2... yes
checking for re2 requires C++11 compiler... yes
checking for RE2::Match() with endpos argument... yes
creating Makefile
current directory: /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/re2-1.1.1/ext/re2
make "DESTDIR=" clean
current directory: /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/re2-1.1.1/ext/re2
make "DESTDIR="
compiling re2.cc
In file included from re2.cc:9:
In file included from /Users/stanhu/.rbenv/versions/2.6.5/include/ruby-2.6.0/ruby.h:33:
In file included from /Users/stanhu/.rbenv/versions/2.6.5/include/ruby-2.6.0/ruby/ruby.h:2111:
/Users/stanhu/.rbenv/versions/2.6.5/include/ruby-2.6.0/ruby/intern.h:56:19: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
void rb_mem_clear(register VALUE*, register long);
^~~~~~~~~
/Users/stanhu/.rbenv/versions/2.6.5/include/ruby-2.6.0/ruby/intern.h:56:36: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
void rb_mem_clear(register VALUE*, register long);
^~~~~~~~~
re2.cc:261:37: error: no member named 'utf8' in 're2::RE2::Options'
p->pattern->options().utf8() ? "UTF-8" : "ISO-8859-1"));
~~~~~~~~~~~~~~~~~~~~~ ^
re2.cc:37:36: note: expanded from macro 'ENCODED_STR_NEW'
int _enc = rb_enc_find_index(encoding); \
^~~~~~~~
re2.cc:459:35: error: no member named 'utf8' in 're2::RE2::Options'
p->pattern->options().utf8() ? "UTF-8" : "ISO-8859-1"));
~~~~~~~~~~~~~~~~~~~~~ ^
re2.cc:37:36: note: expanded from macro 'ENCODED_STR_NEW'
int _enc = rb_enc_find_index(encoding); \
^~~~~~~~
re2.cc:483:33: error: no member named 'utf8' in 're2::RE2::Options'
p->pattern->options().utf8() ? "UTF-8" : "ISO-8859-1");
~~~~~~~~~~~~~~~~~~~~~ ^
re2.cc:37:36: note: expanded from macro 'ENCODED_STR_NEW'
int _enc = rb_enc_find_index(encoding); \
^~~~~~~~
re2.cc:614:29: error: no member named 'utf8' in 're2::RE2::Options'
p->pattern->options().utf8() ? "UTF-8" : "ISO-8859-1");
~~~~~~~~~~~~~~~~~~~~~ ^
re2.cc:37:36: note: expanded from macro 'ENCODED_STR_NEW'
int _enc = rb_enc_find_index(encoding); \
^~~~~~~~
re2.cc:687:19: error: no member named 'set_utf8' in 're2::RE2::Options'
re2_options.set_utf8(RTEST(utf8));
~~~~~~~~~~~ ^
re2.cc:770:29: error: no member named 'utf8' in 're2::RE2::Options'
p->pattern->options().utf8() ? "UTF-8" : "ISO-8859-1");
~~~~~~~~~~~~~~~~~~~~~ ^
re2.cc:37:36: note: expanded from macro 'ENCODED_STR_NEW'
int _enc = rb_enc_find_index(encoding); \
^~~~~~~~
re2.cc:788:29: error: no member named 'utf8' in 're2::RE2::Options'
p->pattern->options().utf8() ? "UTF-8" : "ISO-8859-1");
~~~~~~~~~~~~~~~~~~~~~ ^
re2.cc:37:36: note: expanded from macro 'ENCODED_STR_NEW'
int _enc = rb_enc_find_index(encoding); \
^~~~~~~~
re2.cc:818:42: error: no member named 'utf8' in 're2::RE2::Options'
return BOOL2RUBY(p->pattern->options().utf8());
~~~~~~~~~~~~~~~~~~~~~ ^
re2.cc:21:23: note: expanded from macro 'BOOL2RUBY'
#define BOOL2RUBY(v) (v ? Qtrue : Qfalse)
^
re2.cc:1015:31: error: no member named 'utf8' in 're2::RE2::Options'
p->pattern->options().utf8() ? "UTF-8" : "ISO-8859-1");
~~~~~~~~~~~~~~~~~~~~~ ^
re2.cc:37:36: note: expanded from macro 'ENCODED_STR_NEW'
int _enc = rb_enc_find_index(encoding); \
^~~~~~~~
re2.cc:1046:39: error: no member named 'utf8' in 're2::RE2::Options'
BOOL2RUBY(p->pattern->options().utf8()));
~~~~~~~~~~~~~~~~~~~~~ ^
re2.cc:21:23: note: expanded from macro 'BOOL2RUBY'
#define BOOL2RUBY(v) (v ? Qtrue : Qfalse)
^
re2.cc:1116:33: error: no member named 'utf8' in 're2::RE2::Options'
p->pattern->options().utf8() ? "UTF-8" : "ISO-8859-1"),
~~~~~~~~~~~~~~~~~~~~~ ^
re2.cc:37:36: note: expanded from macro 'ENCODED_STR_NEW'
int _enc = rb_enc_find_index(encoding); \
^~~~~~~~
re2.cc:1287:31: error: no member named 'utf8' in 're2::RE2::Options'
p->pattern->options().utf8() ? "UTF-8" : "ISO-8859-1");
~~~~~~~~~~~~~~~~~~~~~ ^
re2.cc:37:36: note: expanded from macro 'ENCODED_STR_NEW'
int _enc = rb_enc_find_index(encoding); \
^~~~~~~~
re2.cc:1324:31: error: no member named 'utf8' in 're2::RE2::Options'
p->pattern->options().utf8() ? "UTF-8" : "ISO-8859-1");
~~~~~~~~~~~~~~~~~~~~~ ^
re2.cc:37:36: note: expanded from macro 'ENCODED_STR_NEW'
int _enc = rb_enc_find_index(encoding); \
^~~~~~~~
2 warnings and 13 errors generated.
make: *** [re2.o] Error 1
make failed, exit code 2
Gem files will remain installed in /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/re2-1.1.1 for inspection.
Results logged to /Users/stanhu/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/extensions/x86_64-darwin-19/2.6.0/re2-1.1.1/gem_make.out