chopchop-js

Asynchronous processing utilities for the browser (AMD) and for node.

npm install chopchop-js
1 downloads in the last week
4 downloads in the last month

chopchop.js

Utility library for dealing with asynchronous processes in javascript.

  • Keep the event loop spinning be chopping up your long running work in small batches.
  • Use the map reduce paradigm to structure programs
  • Pool asynchronous processes into single API endpoints
  • Use promises to represent asynchronous work (from web workers/nodejs utilities/...)

Project page

Repository

Installation

Node (server)

Use the node package manager to install the module library.

npm install chopchop-js

And then use it in a script

var cc = require('chopchop-js');

AMD (browser)

Grab the module and place it wherever your amd loader can find it.

require({
 paths:{chopchop: 'path/to/wherever/i/dropped/the/module/file/chopchop'}
},['chopchop'], function(cc){
   ... cc is the chopchop module ...
});

Documentation

Go to the Javascript API documentation.

Tutorials

Go to the map reduce demo. It illustrates various aspects of the API.

At a glance

Use chopchop for this:

Keep event loop spinning

Prevent the main event loop from locking up by running long running processes in small batches.

define(['chopchop'], function(cc)}
    cc
        .map([...,...,...], function(){
            return ...
        })
        .then(function(mappedCollection){
            ...
        });
});

Avoid callback hell

Promises or thenables help you avoid callback spaghetti.

This is better.

doSomeAsyncWork(input)
 .then(doSomethingElse)
 .then(doMore)
 .then(wrapUp);

This is worse.

doSomeAsyncWork(input, function(err,data){
    if (!err){
        doSomethingElse(data,function(err,data){
            if (!err){
                doMore(data, function(err,data){
                    if (!err){
                        wrapUp(data);
                    }
                });
            }
        });
    }
});

chopchop is build from the ground up to support the former. Use chopchop to bootstrap popular api's to use promises instead.

Bootstrap node utilities

fs = require('fs');
chch = require('../src/chopchop');

read = chch.strapAsync(fs.readFile);

read('./testWorker.js', 'utf8')
  .then(function (data) {
    //success
  }, function (err) {
    //error
  });

Bootstrap workers

Consider worker 'sumWorker.js';

onmessage = function(oEvent) {
    var dt = oEvent.data;
    var acc = dt[0];
    var value = dt[1];
    postMessage(acc + value);
};

In the browser:

var sumWorker = new Worker('sumWorker.js');
var sum = cc.strapWorker(sumWorker, {
    mapArguments: function (a, b) {
        return [a, b];//the return will be "postMessage" argument
    }
});

sum(1,2)
    .then(function(s){
    //s === 3
});

map reduce

chopchop offers a simple map reduce framework.

Below is the canonical word count example. Map, reduce, and data iteration may all be asynchronous.

//Map function: map each word to the number of times it occurs in the text.
function countWords(text) {
  var s = text.split(/[^\w]/g);
  var map = {};
  for (var i in s) {
    if (!map[s[i]]) {
      map[s[i]] = 0;
    }
    map[s[i]] += 1;
  }
  return map;
}

//Reduce function: aggregate the results by summing.
function sum(a, b) {
  return a + b;
}

//inputs
var data = [
  "you load sixteen tons",
  "and what do you get",
  "another day older",
  "and deeper in debt"
];

cc
    .mapReduce(countWords, sum, data)
    .then(function(result){
        /*
        JSON.stringify(result, null, " ") === "{
         "and": 2,
         "deeper": 1,
         "in": 1,
         "debt": 1,
         "what": 1,
         "do": 1,
         "you": 2,
         "get": 1,
         "another": 1,
         "day": 1,
         "older": 1,
         "load": 1,
         "sixteen": 1,
         "tons": 1
        }";
        */
    });

Project setup

To setup the dev project.

Checkout the mercurial repo:

> hg clone https://bitbucket.org/trgn/chopchop.js

Setup the project (only required for building releases)

> node setup.js

To build a release from the source code, run the "release" jake task

> cd build
> "../node_modules/.bin/jake" -f build.js release

Directories under version control:

  • build: build scripts
  • src: the source code
  • testsrc: the test source code
  • lib: 3rd party libs (amd module loaders, minifiers, qunit)
  • demo: demo of project (dev)

Not versioned:

  • node_modules: module dependencies
  • release: node module, amd module, docs, demo site
  • tmp: for tmp files, generated during build process.
npm loves you