Tracing support for bullmq-pro
Opened this issue · 4 comments
Hi
We use bullmq-pro and elastic-apm for tracing. To get this working we have implemented a separate layer for continuing traces from scheduling a job to worker running the job. This however requires us to do this boilerplate for each project we work on. Would there be any possibility of having an autoinstrumenter for this? Or if you know of a project that does that already that would be even better.
Like I imagine that we could store the current tracing context to the job object when calling .add and then set the parent from that data inside the Worker when starting to work on the job. This would require monkeypatching the bullmq-pro classes so some support from taskforcesh to at least avoid breaking that functionality would be nice -- even better would be to have this be part of the bullmq-pro project.
Our manual setup uses the job data to carry this information and has wrapper around the worker function to set the context. However this requires us to have one more thing to remember and maintain.
@sakari we are planning to add pluggable support for telemetry frameworks such as open-telemetry. It would be quite valuable for us if you could show us an example of how you are currently wrapping the data so that we can design the most flexible solution that covers most cases used by BullMQ users.
we have a layer on top of bullmq that allows us to have middlewares. One such middleware changes the job data to contain the tracing information using elastic apm currentTraceParent. Another middleware envelopes the worker function and sets the tracing context using startTransaction.
However elastic-apm autoinstrumenting would be much simpler setup and would also cover cases where we do not for some reason use the layer on top of bullmq. If we could have a way to add "header" information to the job that does not mess with the job data I think monkeypatching .add and Worker to do the necessary would be enough and would limit the amount of setup we have to do to minimum.
With autoinstrumenting you mean using this? https://github.com/elastic/apm-agent-nodejs
From what I see here: https://www.elastic.co/guide/en/apm/agent/nodejs/current/supported-technologies.html it seems that if BullMQ Pro supported OTel it should work automatically.
yes. yes it seems so if I read the doc correctly.