spandex-project/spandex

Failed to run Custom Tracing example

vovayartsev opened this issue · 6 comments

ENVIRONMENT

  • spandex 1.6.1
  • datadog agent 6.3.3

Hey! Phoenix/Plug works fine for me, but I can not do custom tracing:

defmodule Core.Tracer do
  @moduledoc """
  DataDog tracer
  """
  use Spandex.Tracer, otp_app: :dice
end
defmodule ManuallyTraced do
  alias Core.Tracer
  require Core.Tracer

  # Does not handle exceptions for you.
  def trace_me() do
    _ = Tracer.start_trace("my_trace") #also opens a span
    _ = Tracer.update_span(service: :ecto, type: :db, sql_query: [query: "SELECT * FROM posts", rows: "10"])

    :timer.sleep(1000)
    _ = Tracer.finish_trace()
  end
end

ManuallyTraced.trace_me()

Datadog Agent rejects this trace:

[ TRACE ] 2018-08-01 19:35:16 ERROR (receiver.go:250) - dropping trace reason: invalid span service:"ecto" name:"my_trace" traceID:7629694493567207627 spanID:6427408010887947325 start:1533152115901223000 du...

Am I missing some required params in my example?

Hmm...I think you could turn on verbose logging to see the entire log line in your datadog agent. Its very possible that something is missing in the custom tracing. I've used the custom tracing in the past and didn't have any issues, although I don't use it now.

Thanks, indeed, there is DD_LOG_LEVEL=debug env var.
And the root cause of my problem was missing resource keyword arg to Tracer.trace() macro call.
Should resource become a required argument?

Yes, resource is required by the DataDog API. I hit the same problem getting started.

I'm not sure the best way to handle this. Often, the resource will be known after starting the initial trace, so we don't want to make it required to start. Likewise with something that is likely going to be interwoven into a large amount of data pipelining I hesitate to add a log warning or error message when a span gets sent without a service. In fact, there is a ticket that I think we should use for this, that basically is a strict mode, where failures in spandex actually incur failures and logs in the parent application. With that in mind, we could do this validation there, and have people develop in strict mode.

The ticket is here: #56

We can discuss over there.