fails to intercept oauth2
techieshark opened this issue · 5 comments
Hello 👋
I'm trying to intercept some simple ruby code, like this:
# example.rb
require 'oauth2'
puts "env rubylib is:"
puts ENV['RUBYLIB']
client = OAuth2::Client.new(client_id, client_secret, site)
Using a fresh terminal, I run it and it looks like net/https isn't overridden:
env rubylib is:
/Applications/HTTP Toolkit.app/Contents/Resources/app/httptoolkit-server/overrides/gems
Traceback (most recent call last):
18: from ./example.rb:19:in `<main>'
17: from /Users/user/.gem/ruby/2.6.0/gems/oauth2-1.4.4/lib/oauth2/strategy/client_credentials.rb:20:in `get_token'
16: from /Users/user/.gem/ruby/2.6.0/gems/oauth2-1.4.4/lib/oauth2/client.rb:147:in `get_token'
15: from /Users/user/.gem/ruby/2.6.0/gems/oauth2-1.4.4/lib/oauth2/client.rb:99:in `request'
14: from /Users/user/.gem/ruby/2.6.0/gems/faraday-1.0.0/lib/faraday/connection.rb:492:in `run_request'
13: from /Users/user/.gem/ruby/2.6.0/gems/faraday-1.0.0/lib/faraday/rack_builder.rb:153:in `build_response'
12: from /Users/user/.gem/ruby/2.6.0/gems/faraday-1.0.0/lib/faraday/request/url_encoded.rb:23:in `call'
11: from /Users/user/.gem/ruby/2.6.0/gems/faraday-1.0.0/lib/faraday/adapter/net_http.rb:66:in `call'
10: from /Users/user/.gem/ruby/2.6.0/gems/faraday-1.0.0/lib/faraday/adapter.rb:60:in `connection'
9: from /Users/user/.gem/ruby/2.6.0/gems/faraday-1.0.0/lib/faraday/adapter/net_http.rb:68:in `block in call'
8: from /Users/user/.gem/ruby/2.6.0/gems/faraday-1.0.0/lib/faraday/adapter/net_http.rb:126:in `perform_request'
7: from /Users/user/.gem/ruby/2.6.0/gems/faraday-1.0.0/lib/faraday/adapter/net_http.rb:135:in `request_with_wrapped_block'
6: from /Users/user/.gem/ruby/2.6.0/gems/faraday-1.0.0/lib/faraday/adapter/net_http.rb:149:in `request_via_request_method'
5: from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/net/http.rb:1470:in `request'
4: from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/net/http.rb:919:in `start'
3: from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/net/http.rb:930:in `do_start'
2: from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/net/http.rb:996:in `connect'
1: from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/net/protocol.rb:44:in `ssl_socket_connect'
/System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/net/protocol.rb:44:in `connect_nonblock': SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed (self signed certificate in certificate chain) (OpenSSL::SSL::SSLError)
(Note the error):
SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed (self signed certificate in certificate chain) (OpenSSL::SSL::SSLError)
From faraday/adapter/net_http.rb
, it looks like it's looking for net/https
:
begin
require 'net/https'
rescue LoadError
warn 'Warning: no such file to load -- net/https. ' \
'Make sure openssl is installed if you want ssl support'
require 'net/http'
end
Should the overrides include a net/https
file?
❯ ls "/Applications/HTTP Toolkit.app/Contents/Resources/app/httptoolkit-server/overrides/gems"
http.rb net stripe.rb uri
❯ ls "/Applications/HTTP Toolkit.app/Contents/Resources/app/httptoolkit-server/overrides/gems/net"
http.rb
Thanks
Just an update to confirm a workaround:
Modify faraday/adapter/net_http.rb:
begin
- require 'net/https'
+ require 'net/http'
rescue LoadError
warn 'Warning: no such file to load -- net/https. ' \
'Make sure openssl is installed if you want ssl support'
require 'net/http'
end
Hmm, in theory overriding net/https
shouldn't be required, and just net/http
should be sufficient, because net/https
doesn't actually do anything: https://github.com/ruby/ruby/blob/master/lib/net/https.rb. net/http
includes all the HTTPS logic required all by itself nowadays, so that's where we hook the request setup.
I've just tested and running the below in irb
in an intercepted terminal seems to work fine for me:
require 'net/https'
Net::HTTP.get(URI('https://example.com/'))
That runs successfully, and shows up correctly in the HTTP Toolkit UI. If you test out that minimal case, does that work for you? If not, can you let me know which version of Ruby you're using?
If that does work for you, I suspect this is something to do with Faraday or the rest of your setup. Still a bug either way of course! If you can give me a minimal example and Faraday version I'll look into it further.
Thanks @pimterry.
No luck for me. Is it possible the require_relative 'http'
in Ruby's net/https.rb is getting the local source rather than that from HTTP Toolkit?
Here's the version:
❯ ruby --version
ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.x86_64-darwin19]
❯ which ruby
/usr/bin/ruby
And here's what I see with the minimal example:
❯ irb
WARNING: This version of ruby is included in macOS for compatibility with legacy software.
In future versions of macOS the ruby runtime will not be available by
default, and may require you to install an additional package.
irb(main):001:0> require 'net/https'
=> true
irb(main):002:0> Net::HTTP.get(URI('https://example.com'))
Traceback (most recent call last):
12: from /usr/bin/irb:23:in `<main>'
11: from /usr/bin/irb:23:in `load'
10: from /Library/Ruby/Gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `<top (required)>'
9: from (irb):2
8: from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/net/http.rb:458:in `get'
7: from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/net/http.rb:481:in `get_response'
6: from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/net/http.rb:605:in `start'
5: from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/net/http.rb:919:in `start'
4: from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/net/http.rb:930:in `do_start'
3: from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/net/http.rb:996:in `connect'
2: from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/net/protocol.rb:44:in `ssl_socket_connect'
1: from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/net/protocol.rb:44:in `connect_nonblock'
OpenSSL::SSL::SSLError (SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed (self signed certificate in certificate chain))
I'm back from holidays now, looking into this again, and I've managed to reproduce it! It seems to appear just when using the built-in Ruby release on a Mac. I think the issue is:
- As you suggested,
net/https
avoids thenet/http
hook, by usingrequire_relative
. If you only usenet/https
and never requirenet/http
at all then the hook is never used. - The default Ruby on Mac uses a different OpenSSL build, I think primarily for integration with the system keychain, which ignores OpenSSL's standard
SSL_CERT_FILE
env var.
If you import net/http
too, then the hook handles certificate setup automatically. If you import just net/https
in most environments (except a default Mac Ruby install) it still works anyway, which hid this, because the certificate is trusted automatically by OpenSSL itself due to SSL_CERT_FILE
. If both issues combine though, then you're in trouble.
As an immediate workaround, adding require 'net/http'
before running any code that runs require 'net/https'
does seem to resolve the issue, and should have no downside otherwise. Alternatively, if you install and use Ruby via rbenv
on Mac (or with rvm
or brew
, I suspect) then it works fine out of the box.
To fix this more permanently, I've now added a hook for net/https
too, which just ensures the net/http hook always runs first, before loading the real module. That seems to fix the issue in my case.
Would you mind giving this a quick test, to confirm it fixes the issue for you? You just need to drop the new https.rb
file into HTTP Toolkit's overrides/gems/net
folder on your machine. I'm intending to get a release out in the next few days, once I'm confident that this works.
Hi @techieshark - just to let you know, this fix is now released in server 1.1.0. Next time you open HTTP Toolkit it'll update in the background, and the following time everything should work with Ruby and net/https
as expected. Thanks so much for reporting the issue, it's great to be able to catch more tricky Ruby interception cases like this!