[Feature] `DD_TRACE_REMOVE_AUTOINSTRUMENTATION_ORPHANS` does not remove PDO spans for the `laravelqueue` integration

[stewartmalik]

Describe the feature you'd like

The DD_TRACE_REMOVE_AUTOINSTRUMENTATION_ORPHANS configuration option should also remove calls to PDO for laravelqueue services.

Is your feature request related to a problem?

I'm attempting to add APM tracing to a long running Laravel queue worker that will run in a Kubernetes Pod. I've set the following environment variables in the Pod:

...
            - name: DD_TRACE_CLI_ENABLED
              value: "1"
            - name: DD_TRACE_REMOVE_ROOT_SPAN_LARAVEL_QUEUE
              value: "true"
            - name: DD_TRACE_REMOVE_AUTOINSTRUMENTATION_ORPHANS
              value: "true"
...

Prior to setting the DD_TRACE_REMOVE_ROOT_SPAN_LARAVEL_QUEUE and DD_TRACE_REMOVE_AUTOINSTRUMENTATION_ORPHANS options I was receiving spans for laravelqueue, and pdo services even when the queue worker was sitting there not processing any jobs. It was constantly checking the database for jobs and this was generating these spans.

Adding DD_TRACE_REMOVE_AUTOINSTRUMENTATION_ORPHANS removes the unnecessary laravelqueue spans however the pdo spans still exist.

In the DataDog documentation it states that DD_TRACE_REMOVE_AUTOINSTRUMENTATION_ORPHANS removes the following single span traces:

laravel.event.handle
laravel.provider.load
Predis.Client.__construct
Predis.Client.executeCommand
Predis.Pipeline.executePipeline

In an ideal world I'd like to see single span traces for pdo also removed. The intention is to have only spans generated while executing a job appear in DataDog and not the spans surrounding the running of the queue worker itself.

Describe alternatives you've considered

I investigated using DD_APM_FILTER_TAGS_REJECT however adding an exclusion would likely remove pdo not from just the laravelqueue service but all laravel services as well.

Additional context

No response

[bwoebi]

Hey @stewartmalik,

I think you might be in luck with sampling rules here:

DD_TRACE_SAMPLING_RULES="[{"service": "pdo", "sample_rate": 0}]"

These sampling rules apply by default to the root span only, and so you seem to want to exclude root spans with the pdo service, which fits this perfectly.

[stewartmalik]

Hi @bwoebi,

Apologies for the late response, I've added the changes to our application. In testing your suggestion seems to work as expected 👍

Thank you for your assistance.

Not sure if you want to keep this issue open? I still believe that DD_TRACE_REMOVE_AUTOINSTRUMENTATION_ORPHANS should be removing the singular PDO spans, but the workaround above works fine.

[stewartmalik]

Confirming setting DD_TRACE_SAMPLING_RULES is working as expected in our system.

The queue worker runs and doesn't report PDO spans every time it checks the database for a new job, and spans are correctly still recorded when a job is processing.

[PROFeNoM]

Hey @stewartmalik 👋

Happy to hear that the workaround addressed your issue :-)

Not sure if you want to keep this issue open? I still believe that DD_TRACE_REMOVE_AUTOINSTRUMENTATION_ORPHANS should be removing the singular PDO spans, but the workaround above works fine.

Yes. DD_TRACE_REMOVE_AUTOINSTRUMENTATION_ORPHANS should also remove these singular PDO spans so that no further configuration is needed.

Could you please confirm with me that the following would be the list of the operation names to be removed?

PDOStatement.execute
PDO.commit
PDO.prepare
PDO.__construct
PDO.exec

Thanks! 😃

[iamluc]

Hi @stewartmalik
We've just released version 1.2.0, which fixes this issue.