newrelic/elixir_agent

Connection timeout connecting to `collector.eu01.nr-data.net`

edds opened this issue · 12 comments

edds commented

Describe the bug

I'm seeing the following error in my tmp/new_relic.log

[ERROR] - 2022-08-17 19:34:18 - preconnect: (failed_request) {:failed_connect, [{:to_address, {'collector.eu01.nr-data.net', 443}}, {:inet, [:inet], :timeout}]}

Changing the http timeout in NewRelic.Util.HTTP to 2000ms rather than 1000ms seems to remove the error.

Would you accept a PR to increase, or make configurable the timeout? With the low timeout if the preconnect fails we don't get any transaction tracing at all.

Environment

Both locally and remote on Heroku.

  • Elixir & Erlang version (elixir -v):
Erlang/OTP 24 [erts-12.3.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1]
Elixir 1.13.4 (compiled with Erlang/OTP 24)
  • Agent version (mix deps | grep new_relic_agent):

new_relic_agent 1.27.7

Cgboal commented

I've started getting this also, thanks for the fix

infinityfye commented

I started getting this today as well on AWS ECS.

Cgboal commented

This is now happening for me even with a 5_000ms timeotu set.

infinityfye commented

Yeah, on AWS ECS I actually had to raise it all the way to 10_000

edds commented

For extra context there is a thread on the NewRelic discuss about this issue. Feel free to also let them know there.

  • 👍1
mattbaker commented

Whoops, didn't mean to close this one.

I do think we need a better solution that's not opt-in so it's solved for every user, see some of the discussion here.

tl;dr: let's provide a higher timeout just for collector communication and do a scan of the codebase to see if that could conflict with timeouts elsewhere (GenServer calls, for example). If anyone takes it on feel free to tag me

mattbaker commented

(@edds work in #391 allows for a global increase across the board in the mean time)

ahsandar commented

i am seeing this issue from Linode, Kamatera and Cloudcone as well. Is there any docs on how to increase the timeout in 1.27.8 ?

infinityfye commented

i am seeing this issue from Linode, Kamatera and Cloudcone as well. Is there any docs on how to increase the timeout in 1.27.8 ?

Nope. Unfortunately, documentation has not been updated yet.

But you can do this in your config:

config :new_relic_agent,
    httpc_request_options: [
      connect_timeout: 13_000 # choose whatever number is suitable for you here
    ],
    ...
ahsandar commented

Thanks @infinityfye, I have a PR sent for docs update hope that gets merge and is of use to others

  • ❤️1
mattbaker commented

Closing in favor of #395