zombocom/derailed_benchmarks

Can't hit the endpoint

Opened this issue · 8 comments

stevesuh commented

Been stumped since yesterday afternoon and paired with another engineer today but still stuck. Any pointers would be appreciated.

Mac OSX 11.6 on Intel

13:37 $ USE_AUTH=true RAILS_ENV=development PATH_TO_HIT=http://localhost:5000/v1/users/reservation_history bundle exec derailed exec perf:mem_over_time
Booting: development
WARNING: Control endpoints are disabled.
         Set ENABLE_CONTROLS=1 to enable them.
         Set ENABLE_CONTROLS=0 in .env.development.local to suppress this warning.
`Redis#exists(key)` will return an Integer by default in redis-rb 4.3. The option to explicitly disable this behaviour via `Redis.exists_returns_integer` will be removed in 5.0. You should use `exists?` instead.
Warning: method `create` already defined in Admin::StoreAdsSettingsController
Warning: method `update` already defined in Admin::StoreAdsSettingsController
Database 'api_dev' already exists
Database 'api_test' already exists
Endpoint: "http://localhost:5000/v1/users/reservation_history"
Auth: true
PID: 77200
253.7890625
Couldn't call app. Bad request to http://localhost:5000/v1/users/reservation_history! Resulted in 403 status.


***RESPONSE BODY***

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8" />
  <title>Action Controller: Exception caught</title>
  <style>
    body {
      background-color: #FAFAFA;
      color: #333;
      margin: 0px;
    }

    body, p, ol, ul, td {
      font-family: helvetica, verdana, arial, sans-serif;
      font-size:   13px;
      line-height: 18px;
    }

    pre {
      font-size: 11px;
      white-space: pre-wrap;
    }

    pre.box {
      border: 1px solid #EEE;
      padding: 10px;
      margin: 0px;
      width: 958px;
    }

    header {
      color: #F0F0F0;
      background: #C52F24;
      padding: 0.5em 1.5em;
    }

    h1 {
      margin: 0.2em 0;
      line-height: 1.1em;
      font-size: 2em;
    }

    h2 {
      color: #C52F24;
      line-height: 25px;
    }

    .details {
      border: 1px solid #D0D0D0;
      border-radius: 4px;
      margin: 1em 0px;
      display: block;
      width: 978px;
    }

    .summary {
      padding: 8px 15px;
      border-bottom: 1px solid #D0D0D0;
      display: block;
    }

    .details pre {
      margin: 5px;
      border: none;
    }

    #container {
      box-sizing: border-box;
      width: 100%;
      padding: 0 1.5em;
    }

    .source * {
      margin: 0px;
      padding: 0px;
    }

    .source {
      border: 1px solid #D9D9D9;
      background: #ECECEC;
      width: 978px;
    }

    .source pre {
      padding: 10px 0px;
      border: none;
    }

    .source .data {
      font-size: 80%;
      overflow: auto;
      background-color: #FFF;
    }

    .info {
      padding: 0.5em;
    }

    .source .data .line_numbers {
      background-color: #ECECEC;
      color: #AAA;
      padding: 1em .5em;
      border-right: 1px solid #DDD;
      text-align: right;
    }

    .line {
      padding-left: 10px;
      white-space: pre;
    }

    .line:hover {
      background-color: #F6F6F6;
    }

    .line.active {
      background-color: #FFCCCC;
    }

    .button_to {
      display: inline-block;
    }

    .hidden {
      display: none;
    }

    a { color: #980905; }
    a:visited { color: #666; }
    a.trace-frames { color: #666; }
    a:hover { color: #C52F24; }
    a.trace-frames.selected { color: #C52F24 }


  </style>

  <script>
    var toggle = function(id) {
      var s = document.getElementById(id).style;
      s.display = s.display == 'none' ? 'block' : 'none';
      return false;
    }
    var show = function(id) {
      document.getElementById(id).style.display = 'block';
    }
    var hide = function(id) {
      document.getElementById(id).style.display = 'none';
    }
    var toggleTrace = function() {
      return toggle('blame_trace');
    }
    var toggleSessionDump = function() {
      return toggle('session_dump');
    }
    var toggleEnvDump = function() {
      return toggle('env_dump');
    }
  </script>
</head>
<body>

<header>
  <h1>Blocked host: localhost</h1>
</header>
<div id="container">
  <h2>To allow requests to localhost, add the following to your environment configuration:</h2>
  <pre>config.hosts &lt;&lt; "localhost"</pre>
</div>


</body>
</html>

Contents of /etc/hosts:

##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
127.0.0.1       localhost
127.0.0.1       jane.localhost
127.0.0.1       business.jane.localhost
127.0.0.1       retail.jane.localhost
127.0.0.1       express.jane.localhost
255.255.255.255 broadcasthost
::1             localhost
# Added by Docker Desktop
# To allow the same kube context to work on the host and the container:
127.0.0.1 kubernetes.docker.internal
# End of section

Contents of /private/etc/hosts:

##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
127.0.0.1       localhost
127.0.0.1       jane.localhost
127.0.0.1       business.jane.localhost
127.0.0.1       retail.jane.localhost
127.0.0.1       express.jane.localhost
255.255.255.255 broadcasthost
::1             localhost
# Added by Docker Desktop
# To allow the same kube context to work on the host and the container:
127.0.0.1 kubernetes.docker.internal
# End of section

Contents of perf.rake:

require 'bundler/setup'

require 'derailed_benchmarks'
require 'derailed_benchmarks/tasks'

namespace :perf do
  task :rack_load do
    STDERR.puts "inside perf"
    require_relative "config/boot"
    require_relative "config/environment"
    DERAILED_APP = RailsBaseApi::Application::Routes
  end
end

DerailedBenchmarks.auth.user = -> { User.find_by(email: "stevesuh44@gmail.com") }

relevant rake route snippet:

 reservation_history_v1_users GET        /v1/users/reservation_history(.:format)                                                             v1/reservations/history#index {:format=>"json"}
stevesuh commented

Updated comment above because of application.rb:

config.host = "localhost:5000"
stevesuh commented

select id, email from users where email = 'stevesuh44@gmail.com';

Screen Shot 2021-10-27 at 1 53 49 PM

stevesuh commented

Switched to an endpoint that doesn't need auth just to try to get it working.

18:47 $ RAILS_ENV=development PATH_TO_HIT=http://127.0.0.1:5000/v1/products/1 bundle exec derailed exec perf:mem_over_time
Booting: development
WARNING: Control endpoints are disabled.
         Set ENABLE_CONTROLS=1 to enable them.
         Set ENABLE_CONTROLS=0 in .env.development.local to suppress this warning.
`Redis#exists(key)` will return an Integer by default in redis-rb 4.3. The option to explicitly disable this behaviour via `Redis.exists_returns_integer` will be removed in 5.0. You should use `exists?` instead.
Warning: method `create` already defined in Admin::StoreAdsSettingsController
Warning: method `update` already defined in Admin::StoreAdsSettingsController
Database 'api_dev' already exists
Database 'api_test' already exists
Endpoint: "http://127.0.0.1:5000/v1/products/1"
PID: 94334
249.4609375
Couldn't call app. Bad request to http://127.0.0.1:5000/v1/products/1! Resulted in 403 status.


***RESPONSE BODY***

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8" />
  <title>Action Controller: Exception caught</title>
  <style>
    body {
      background-color: #FAFAFA;
      color: #333;
      margin: 0px;
    }

    body, p, ol, ul, td {
      font-family: helvetica, verdana, arial, sans-serif;
      font-size:   13px;
      line-height: 18px;
    }

    pre {
      font-size: 11px;
      white-space: pre-wrap;
    }

    pre.box {
      border: 1px solid #EEE;
      padding: 10px;
      margin: 0px;
      width: 958px;
    }

    header {
      color: #F0F0F0;
      background: #C52F24;
      padding: 0.5em 1.5em;
    }

    h1 {
      margin: 0.2em 0;
      line-height: 1.1em;
      font-size: 2em;
    }

    h2 {
      color: #C52F24;
      line-height: 25px;
    }

    .details {
      border: 1px solid #D0D0D0;
      border-radius: 4px;
      margin: 1em 0px;
      display: block;
      width: 978px;
    }

    .summary {
      padding: 8px 15px;
      border-bottom: 1px solid #D0D0D0;
      display: block;
    }

    .details pre {
      margin: 5px;
      border: none;
    }

    #container {
      box-sizing: border-box;
      width: 100%;
      padding: 0 1.5em;
    }

    .source * {
      margin: 0px;
      padding: 0px;
    }

    .source {
      border: 1px solid #D9D9D9;
      background: #ECECEC;
      width: 978px;
    }

    .source pre {
      padding: 10px 0px;
      border: none;
    }

    .source .data {
      font-size: 80%;
      overflow: auto;
      background-color: #FFF;
    }

    .info {
      padding: 0.5em;
    }

    .source .data .line_numbers {
      background-color: #ECECEC;
      color: #AAA;
      padding: 1em .5em;
      border-right: 1px solid #DDD;
      text-align: right;
    }

    .line {
      padding-left: 10px;
      white-space: pre;
    }

    .line:hover {
      background-color: #F6F6F6;
    }

    .line.active {
      background-color: #FFCCCC;
    }

    .button_to {
      display: inline-block;
    }

    .hidden {
      display: none;
    }

    a { color: #980905; }
    a:visited { color: #666; }
    a.trace-frames { color: #666; }
    a:hover { color: #C52F24; }
    a.trace-frames.selected { color: #C52F24 }


  </style>

  <script>
    var toggle = function(id) {
      var s = document.getElementById(id).style;
      s.display = s.display == 'none' ? 'block' : 'none';
      return false;
    }
    var show = function(id) {
      document.getElementById(id).style.display = 'block';
    }
    var hide = function(id) {
      document.getElementById(id).style.display = 'none';
    }
    var toggleTrace = function() {
      return toggle('blame_trace');
    }
    var toggleSessionDump = function() {
      return toggle('session_dump');
    }
    var toggleEnvDump = function() {
      return toggle('env_dump');
    }
  </script>
</head>
<body>

<header>
  <h1>Blocked host: 127.0.0.1</h1>
</header>
<div id="container">
  <h2>To allow requests to 127.0.0.1, add the following to your environment configuration:</h2>
  <pre>config.hosts &lt;&lt; "127.0.0.1"</pre>
</div>


</body>
</html>

Contents of rake.perf:

require 'bundler/setup'

require 'derailed_benchmarks'
require 'derailed_benchmarks/tasks'

namespace :perf do
  task :rack_load do
    STDERR.puts "inside perf"
    require_relative "config/boot"
    require_relative "config/environment"
    DERAILED_APP = RailsBaseApi::Application::Routes
  end
end
stevesuh commented

Figured out the issue. Need to set HTTP_HOST to localhost.

https://stackoverflow.com/questions/65433303/sending-a-rack-request-through-rails-console

Might be helpful to update the README for next chap.

schneems commented

I think the problem here is that PATH_TO_HIT is only supposed to be a path, not a full host.

Does it work if you use PATH_TO_HIT=/v1/users/reservation_history without HTTP_HOST ?

We could detect the case of that starting with http or localhost:<port> and issue a warning. I would be open to a PR adding that check.

stevesuh commented

It tries to go to example.org if you remove HTTP_HOST=localhost

19:42 $ USE_AUTH=true RAILS_ENV=development PATH_TO_HIT=/v1/users/reservation_history bundle exec derailed exec perf:mem_over_time
Booting: development
WARNING: Control endpoints are disabled.
         Set ENABLE_CONTROLS=1 to enable them.
         Set ENABLE_CONTROLS=0 in .env.development.local to suppress this warning.
`Redis#exists(key)` will return an Integer by default in redis-rb 4.3. The option to explicitly disable this behaviour via `Redis.exists_returns_integer` will be removed in 5.0. You should use `exists?` instead.
Warning: method `create` already defined in Admin::StoreAdsSettingsController
Warning: method `update` already defined in Admin::StoreAdsSettingsController
Database 'api_dev' already exists
Database 'api_test' already exists
Endpoint: "/v1/users/reservation_history"
Auth: true
DERAILED_APP: #<DerailedBenchmarks::AuthHelpers::Devise:0x00007f933c5bdd90>
PID: 98962
242.20703125
Couldn't call app. Bad request to /v1/users/reservation_history! Resulted in 403 status.


***RESPONSE BODY***

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8" />
  <title>Action Controller: Exception caught</title>
  <style>
    body {
      background-color: #FAFAFA;
      color: #333;
      margin: 0px;
    }

    body, p, ol, ul, td {
      font-family: helvetica, verdana, arial, sans-serif;
      font-size:   13px;
      line-height: 18px;
    }

    pre {
      font-size: 11px;
      white-space: pre-wrap;
    }

    pre.box {
      border: 1px solid #EEE;
      padding: 10px;
      margin: 0px;
      width: 958px;
    }

    header {
      color: #F0F0F0;
      background: #C52F24;
      padding: 0.5em 1.5em;
    }

    h1 {
      margin: 0.2em 0;
      line-height: 1.1em;
      font-size: 2em;
    }

    h2 {
      color: #C52F24;
      line-height: 25px;
    }

    .details {
      border: 1px solid #D0D0D0;
      border-radius: 4px;
      margin: 1em 0px;
      display: block;
      width: 978px;
    }

    .summary {
      padding: 8px 15px;
      border-bottom: 1px solid #D0D0D0;
      display: block;
    }

    .details pre {
      margin: 5px;
      border: none;
    }

    #container {
      box-sizing: border-box;
      width: 100%;
      padding: 0 1.5em;
    }

    .source * {
      margin: 0px;
      padding: 0px;
    }

    .source {
      border: 1px solid #D9D9D9;
      background: #ECECEC;
      width: 978px;
    }

    .source pre {
      padding: 10px 0px;
      border: none;
    }

    .source .data {
      font-size: 80%;
      overflow: auto;
      background-color: #FFF;
    }

    .info {
      padding: 0.5em;
    }

    .source .data .line_numbers {
      background-color: #ECECEC;
      color: #AAA;
      padding: 1em .5em;
      border-right: 1px solid #DDD;
      text-align: right;
    }

    .line {
      padding-left: 10px;
      white-space: pre;
    }

    .line:hover {
      background-color: #F6F6F6;
    }

    .line.active {
      background-color: #FFCCCC;
    }

    .button_to {
      display: inline-block;
    }

    .hidden {
      display: none;
    }

    a { color: #980905; }
    a:visited { color: #666; }
    a.trace-frames { color: #666; }
    a:hover { color: #C52F24; }
    a.trace-frames.selected { color: #C52F24 }


  </style>

  <script>
    var toggle = function(id) {
      var s = document.getElementById(id).style;
      s.display = s.display == 'none' ? 'block' : 'none';
      return false;
    }
    var show = function(id) {
      document.getElementById(id).style.display = 'block';
    }
    var hide = function(id) {
      document.getElementById(id).style.display = 'none';
    }
    var toggleTrace = function() {
      return toggle('blame_trace');
    }
    var toggleSessionDump = function() {
      return toggle('session_dump');
    }
    var toggleEnvDump = function() {
      return toggle('env_dump');
    }
  </script>
</head>
<body>

<header>
  <h1>Blocked host: example.org</h1>
</header>
<div id="container">
  <h2>To allow requests to example.org, add the following to your environment configuration:</h2>
  <pre>config.hosts &lt;&lt; "example.org"</pre>
</div>


</body>
</html>

Also the README says you can use a full uri.
https://github.com/zombocom/derailed_benchmarks#hitting-a-different-endpoint-with-path_to_hit

schneems commented

You are right there about passing in fully qualified hosts. Thankfully past me documented it :)

It looks like some logic somewhere in your app is blocking connections.

where is this output coming from “Blocked host”?

Can you reproduce this with a “rails new” app?

Michoels commented

I just had this as well.
Without HTTP_HOST=localhost, I got this error:

 <!-- BEGIN <REDACTED>/gems/ruby-3.2.2/gems/actionpack-7.0.8/lib/action_dispatch/middleware/templates/rescues/blocked_host.html.erb -->
 <header>
  <h1>Blocked hosts: </h1>
</header>
<main role="main" id="container">
  <h2>To allow requests to these hosts, make sure they are valid hostnames (containing only numbers, letters, dashes and dots), then add the following to your environment configuration:</h2>
  <pre>
    config.hosts &lt;&lt; ""
  </pre>
  <p>For more details view: <a href="https://guides.rubyonrails.org/configuring.html#actiondispatch-hostauthorization">the Host Authorization guide</a></p>

When I added HTTP_HOST=localhost the error disappeared.
This behavior is documented in the Rails docs here.

The command that worked was:

HTTP_HOST=localhost RAILS_ENV=development PATH_TO_HIT=/sync/3/start USE_AUTH=true bundle exec derailed exec perf:mem