yajq

YAJQ - Yet Another Job Queue

Yajq is a redis backed job queue, with a simple wire format

Installation

npm install yajq

Usage

фыв// adding job to queue
var jq = require("yajq").createQueue();
jq.add_job("echo", {'data': 'some string'}, function (err, res) {
    if(err) {
        console.log("ERROR: %s", err);
    }
    else {
        console.log(res);
    }
});

// executing jobs
var worker = require('yajq').createWorker();
worker.on("echo", function (params, done) {
    done(null, params.data);
});
worker.start();

Wire Format

Jobs are rpush'ed to the redis list yajq:jobs or yajq:jobs:#{name} for named queues. Job description in the queue is a simple JSON object

{
    'job_id'    : 'a551c40f-9257-4fde-bcb6-b43cd3d78575', // random uuid to identify the job
    'caller_id' : 41955, // id of the process that added the job
    'name'      : "some_task", // name of the task that should be performed. Think RPC function name
    'params'    : "" // any valid JSON value, parameters for the job
}

job_id and caller_id can be omitted if you don't need the response

Worker process pops an object from the list and does whatever it has to do. After the worker process is done with the job, it rpush'es a response object on the response queue. The name of the response queue is yajq:done:#{caller_id} ('yajq:done:41955' in the example above). The response format looks like this:

{
    'job_id'   : 'a551c40f-9257-4fde-bcb6-b43cd3d78575', // uuid of the job
    'error'    : "", // error description if there was an error processing the job, if there was no error this key is omitted
    'response' : "" // any valid JSON value
}

API

yajq.createQueue([port], [host], [options])

Create Job Queue object. Parameters are passed directly to redis.createClient();

Queue.add_job(job_name, [params], [callback (err, res)])

Add job to default "yajq:jobs" queue. params and callback are optional.

Queue.add_job_to(queue_name, job_name, [params], [callback (err, res)])

Add job to "yajq:jobs:#{queue_name}" queue. params and callback are optional.

yajq.createWorker([port], [host], [options])

Create worker object. Parameters are passed directly to redis.createClient();

Worker.on(job_name, callback(params, done(err, res)))

Worker is an EvenEmitter. When when worker is done with the job, it should call done(err, res) function to pass the response to the caller. err should be null if there was no error.

Worker.start([queue_name])

Start processing the job queue. If queue_name is omitted, worker starts on default yajq:jobs queue.