Async-ForkQueue

Async-ForkQueue is based on ForkQueue, but it allows setting a level of concurrency where each forked process will run that many at a time and provides a api for creating worker functions.

Install

  npm install async-forkqueue

API

var Queue = require('async-forkqueue');
var num_workers = 4;
var concurrency = 4;
 
var queue = new Queue(num_workers, concurrency, module_path);
 
for (var i = 0; i < 100; i++) {
  queue.push(i);
}
 
queue.end(callback);

Worker modules

Worker modules are spawned with child_process.fork.

A simple worker is below.

module.exports = function (payload, cb) {
  // Do something with the payload
  cb();
}