Tracing Envoy with consul-envoy:v1.11.2-v1.20.1

[mlindes]

The Datadog example config does not work with the xDS v3 syntax used in later versions like consul-envoy:v1.11.2-v1.20.1
I've managed to sort out some syntax to capture traces, but I'm still not able to see the Envoy spans. Not sure where I'm going wrong. I'd love to see a working example of this

For the tracing json, I'm using:

  envoy_tracing_json = <<EOL
  {
    "http": {
      "name": "envoy.tracers.datadog",
      "typed_config": {
        "@type": "type.googleapis.com/envoy.config.trace.v3.DatadogConfig",
        "collector_cluster": "datadog_8126",
        "service_name": "envoy"
      }
    }
  }
  EOL

This allows traces to be captured from the services, but not envoy.

[nicholasjackson]

Hey @mlindes, let me look into this that example is quite old there but the tracing should work with later envoy versions. As you correctly state it will be a case of updating to change the deprecated v2 syntax. Might take me a couple of days before I get to this but will definitely have this updated by the weekend.

[nicholasjackson]

Hey @mlindes, let me look into this that example is quite old there but the tracing should work with later envoy versions. As you correctly state it will be a case of updating to change the deprecated v2 syntax. Might take me a couple of days before I get to this but will definitely have this updated by the weekend.

[mlindes]

Hey @nicholasjackson - just some followup....
Using your consul-demo-tracing repo with the envoy_tracing_json config noted above & some updates to datadog env vars, I am able to see envoy spans. Interestingly enough, though, I'm not able to reproduce this in ECS Fargate. I'll follow-up on that elsewhere, but would welcome any input you have regarding passing the envoy config as a value for CONSUL_LOCAL_CONFIG to the envoy container.

[nicholasjackson]

@mlindes I am really sorry for the late reply, my Github notifications are a mess.

Are you using Consul ECS?

https://www.consul.io/docs/ecs

If so you can pass that config using the Proxy Defaults configuration entry.

https://www.consul.io/docs/connect/config-entries/proxy-defaults

If you are manually starting the Envoy proxy using the consul connect envoy command this would work too.

Note: You need to set the proxy-defaults before creating the proxy, tracing settings are a bootstrap setting I think, and are not hot updated.

[mlindes]

Sorry for such a slow turnaround on my side.
First, to answer the questions

1. yes, we're using consul ecs
2. testing both - local & with proxy defaults. By "locally", I mean we were passing the proxy config as a json blob in CONSUL_LOCAL_CONFIG with envoy_public_listener_json envoy_tracing_json & envoy_extra_static_clusters_json vars specified. It's json inside json of env vars inside an env var. No fun :)

But we solved the problem. The main issue is that services registered need to be registered in consul as http services, by default they were being registered as tcp services. While they services still performed their functions well, the tracing filters were not being applied.