Debugging oneway calls

[dnlserrano]

Hey peeps,

First of all, thank you so much for your hard work on elixir-thrift! 🙇 👏

I've decided to open this issue after I asked @jparise for help on Twitter and he advised me to do so. I'm an Elixir and (especially) Thrift noob, so please bear with me. 🐻

Question

How do you debug oneway calls in elixir-thrift?

What

I am trying to call Jaeger Agent's emitBatch (Thirft IDL here) to submit a batch of spans.

To do it, I'm running the default docker-compose.yml from the Jaeger project. I can see traces being submitted when I hit my (running) Jaeger Agent using their Python example.

In trying to do the same via Elixir, I have resorted to the great elixir-thrift lib. It worked fine to auto-generate the code based on the Jaeger IDL. I created a client, called the Agent, but nothing happened. I see an {:ok, nil}, but that's about it... No traces are actually captured by Jaeger.

How

In order to test it, I just did docker-compose up on the Jaeger project to get the Jaeger Query (i.e., UI), Agent and Collector running.

I added 3 files for the IDL (agent.thrift, jaeger.thrift and zipkincore.thrift) to the thrift/ folder and instructed mix to compile them via:

# mix.exs

def project do
  [
    ...
    compilers: [:thrift | Mix.compilers],
    thrift: [
      files: Path.wildcard("thrift/**/*.thrift")
    ],
    ...
  ]
end

Then I wrote the following simples script:

# jaeger.exs

alias Thrift.Generated.Agent.Binary.Framed.Client
alias Thrift.Generated.Batch

{:ok, agent} = Client.start_link(
  "jaeger-agent.jaeger.docker", 5778, []
)

batch = %Batch{
  process: %Thrift.Generated.Process{
    service_name: "spandex_jaeger", tags: []
  },
  spans: [
    %Thrift.Generated.Span{
      annotations: nil,
      binary_annotations: nil,
      debug: true,
      duration: 130,
      id: 222,
      name: "my_elixir_span",
      parent_id: nil,
      timestamp: nil,
      trace_id: 111,
      trace_id_high: nil
    }
  ]
}

Client.emit_batch(agent, batch)

So, that's why I'd like to know how you debug what's happening in the backstage. Do I have a way of understanding why this isn't yielding any actual results? Is the only way prying into the Jaeger Agent code? Can I debug oneway calls in any other way?

Thanks in advance! 🤗

[fishcakez]

Hi @dnlserrano, I would start with trying out our thrift debugging tool to see whats being sent over the wire: https://github.com/pinterest/thrift-tools. Note that we don't use oneway calls with elixir-thrift internally so that might be some gaps 😊 .

Try something like:

$ pip install thrift-tools
$ sudo thrift-tool --iface eth0 --port 5778 dump --show-all --pretty

[dnlserrano]

Thanks for the help @fishcakez, I've been fiddling with it. It's helping me in understanding what's going on under the covers. In the meantime, shall I update the README to have that info (e.g., Debug section pointing to that Python lib)? It would have saved both of us this GitHub issue. 😅 👊

[fishcakez]

Sure, does that mean you found the cause is not elixir-thrift?