Transactions in `SENTRY_IGNORED_TRANSACTIONS` are not ignored

[fbad]

Integration

sentry-opentelemetry-agent

Java Version

17

Version

8.2.0

Steps to Reproduce

1. Set SENTRY_IGNORED_TRANSACTIONS env var (e.g. GET *).
2. Run Java application based on Jetty server with Sentry Opentelemetry Java Agent with -javaagent and auto- initialization.

Expected Result

GETs requests are filtered out before to get both to Sentry and OTEL traces exporter.

Actual Result

All spans are sent both to OTEL traces exporter and Sentry without filtering GETs. Names in Sentry and OTEL traces start with GET.

[lcian]

Hi @fbad , thanks for reporting this.

The list of values set by the env var SENTRY_IGNORED_TRANSACTIONS is turned into a java.util.regex regular expression to determine if transaction names match.
So, if you want to filter all transactions that start with GET, you should set SENTRY_IGNORED_TRANSACTION to GET.*.

Please let me know if that works.
I agree that is confusing and someone could think the option takes glob patterns instead, perhaps we should document it better in the changelog.

[fbad]

Hi @lcian, thank you for your prompt response.

Actually, I tried with both regexes, glob patterns and explicitly stating an endpoint, but none of these worked for me unfortunately.

[adinauer]

@fbad are you using an embedded Jetty in a JAR file that you start with the -javaagent option?

[adinauer]

Can you also share what other ENV vars and other Sentry config files (sentry.properties, application.properties / application.yml) you set and if there's other Sentry SDKs you're using.

[adinauer]

I just retested and it works fine for me.

[fbad]

@fbad are you using an embedded Jetty in a JAR file that you start with the -javaagent option?

Yes, correct.

Can you also share what other ENV vars and other Sentry config files (sentry.properties, application.properties / application.yml) you set and if there's other Sentry SDKs you're using.

No other Sentry SDKs, we have these other env vars:

SENTRY_ENVIRONMENT=staging
SENTRY_TRACES_SAMPLE_RATE=0.05
SENTRY_SAMPLE_RATE=1.0
SENTRY_ENABLED=true
SENTRY_DSN=XXX

and these for OTEL:

OTEL_LOGS_EXPORTER=none
OTEL_METRICS_EXPORTER=none
OTEL_TRACES_EXPORTER=otlp
OTEL_EXPORTER_OTLP_ENDPOINT=XXX
OTEL_EXPORTER_OTLP_PROTOCOL=grpc
OTEL_SERVICE_NAME=XXX
OTEL_RESOURCE_ATTRIBUTES=a=b,c=d

I just retested and it works fine for me.

Ok, nice, I can do some more debugging on our side then if required, just wanted to double check this was not a well-known issue. Thank you!

[adinauer]

Thanks for the details @fbad . I see no obvious problems here.
Could it be your ENV vars are not being read correctly, e.g. due to your shell interpreting the regex characters?
Maybe you can put the ENV var value in quotes to fix it.

[fbad]

@adinauer, I shared the env vars like that for simplicity but they are actually injected in a container (this is a Kubernetes deployment) so I don't think it's something to be done with regex characters (btw, I tried also with explicitly stating the endpoint and it did not exclude those transactions anyway).

[adinauer]

Can you inspect / print the output of Sentry.getCurrentScopes().getOptions().getIgnoredTransactions() to see if it actually makes its way into Sentry config?

[fbad]

Hey @adinauer, I was able to run the container locally and debug a little more. In fact, ignored transactions are set and using SENTRY_DEBUG=true and OTEL_TRACES_EXPORTER=console I can see they are correctly filtered from output sent to Sentry but not from output sent to exporter (and then logged to console). I was looking at wrong time frame in Sentry before, sorry about that, that's why I thought both were not ignoring transactions, while it's actually only OTEL exporter.

This is not a big problem for us as we can easily filter them out from OTEL collector, but I thought it was worth to mention in case the expected behaviour was different?

Thank you!

[adinauer]

Yeah that's expected (for now). I'm not sure we could filter it out from other exporters since the transaction name is used for filtering when the span has ended and we don't expect any further updates to it as opposed to filtering in the OpenTelemetry sampler before the span is actually started.