Katar
A modular queue server for node.js
Features
- Built on ES6 Harmony generators
- Support for multiple queues
- Completely modular
- Pluggable data store
- Process jobs using distributed workers over http, tcp or ipc
Usage
Installation
npm install katar --save
katar and its subcomponents harness the full power of ES6 generators and as such require node 0.11.9 or higher, and must run node with the --harmony flag. If you don't like typing this, add an alias to your shell profile:
alias node='node --harmony'
Example
Here's an example of a complex queue server that lets you create multiple queues and use HTTP polling workers
// use mongo for storing tasksvar katarDb = 'localhost/queue';// initialise a katar servervar katar = db: katarDb ;// create a data queue that fetches data from a urlvar dataQueue = katar;// create a youtube queue that downloads videos from youtube // this queue is persistent - this means once a task is completed, it is retained in the database for historical reasonsvar ytQueue = katar; // insert a data taskdataQueue; // insert a data task with high priority // supported values - HIGH, MEDIUM, LOWdataQueue; // insert a youtube request with an object data type // all data types supported by the data adapter are supported as data is passed straight to the adapterytQueue; // create a http server that workers can poll and fetch jobs // server is a koa-framework objectvar workerServer = katar: katar port: 3000; // add any custom routes as needed // see koa-framework npm module for usageworkerServerapiv1router;
Modularity
Data store
The queue system can be backed by any database. Simply create a data adapter that has the required methods
Available adapters:
- MemoryDb
npm install katar-memorydb
- MongoDb
npm install katar-mongodb
To simplify the task of creating data adapters, a test suite is published that can be used to test the functionality of the adapter meets the needs of katar.
npm install katar-db-test
Workers
Katar allows workers to subscribe for new jobs, poll the server or push new jobs to the server
Pull based
Create a server that listens for worker requests. Workers can poll the server on a timely basis to fetch new jobs and report when jobs have been finished.
Channels:
- HTTP:
npm install katar-worker-http
Changelog
0.0.1 - Alpha
- Initial release