Instrumental/instrumental_agent-ruby

Resque workers need to have the agent be synchronous

netshade opened this issue · 3 comments

When using the agent in a Resque worker, the forked process that runs the job needs to have the agent be synchronous in order to flush any metrics sent during the job to the server.

Right now you can fix this problem by doing:

Resque.after_fork do |job|
  I.synchronous = true
end

Consider making this automatic?

Note: the Resque fix above can cause a hanging worker in cases where the Instrumental collector is down. Use with caution for now.

I'm flagging this issue as closed, with the introduction of the Instrumental::Agent#flush method. Users of Resque should add the following to their Rails initializers in order to have metrics recorded in a Resque job correctly flushed to the server:

Resque.after_perform { I.flush }
Resque.on_failure { I.flush }

It should be noted that this behavior is specific to scripts which use Kernel.exit! as Resque does. In most cases where a script exits normally, the Instrumental Agent will correctly use an .at_exit hook to flush all pending metrics to our servers. Only in cases where .exit! is called and the at_exit handlers for the process are not called will any pending metrics not be flushed to our server.

Note that the Instrumental::Agent#flush method was introduced in version 0.9.0.