how does an `enqueue`d job know when it is completed

[davetayls]

I'm enqueueing jobs but i can't work out how the job returned in the enqueue callback can be notified of when it is completed by a separate worker.

I was expecting to be able to do something like:

queue.enqueue('jobname', function(err, job) {
  job.on('complete', function() { console.log('complete'); });
});

[scttnlsn]

It is likely that the job is being processed by a worker in a separate process (from the enqueuing process) so you'll need some sort of inter-process pubsub to make this work. I had originally baked this into Monq by default (using https://github.com/scttnlsn/mubsub) but later removed it since it's easy enough to add that functionality where needed and is often more overhead than is needed for the majority of cases.

I'd recommend having each worker publish events about the job lifecycle to some sort of message broker. You can forward some of the emitted worker events (i.e. dequeued, complete, failed, etc.) to the broker and even publish progress events if it makes sense for your workers. In the enqueueing process, listen for the appropriate events to monitor the progress of a particular job. The alternative is to have the enqueueing process periodically poll Mongo to reload the job and check its status.

Does that make sense?

[davetayls]

I see, that sounds ok. I'll take a look at your mubsub project. In fact I don't need it for the immediate use case but I'm pretty sure I will in the future so I didn't want to tie myself in to something where I didn't have the option. thanks for you work on this.

[brianreese]

I have a worker which spawns a child process. The parent process listens for events on the spawned child process, and eventually calls the enqueued job's callback once complete - however, my worker doesn't seem to ever end the current job and grab a new one.

I don't need to notify the enqueuing process of the job's progress, just trying to figure out how to have my worker process more than a single item from the queue. Are there any special considerations when processing my queue in this way (e.g. incorporating the child process), or am I maybe just missing something in the way I've set up my worker?

[brianreese]

Nevermind, please ignore my previous comment. The job wasn't actually completing, and the promise I setup never actually resolved.