comment-chunk-helper

A utility function that makes it easier to write code chunkers for Sense engines.

Rationale

Common code operations like parsing and syntax highlighting have strong library support. While writing Sense engines, however, we found that another code operation we call chunking is often awkward to implement. Code chunking is splitting code up into logical units such as comment and statement blocks, which are the top-level executable units of Sense's interactive dashboards. Chunking allows dashboards to display code, comments, results and output in an order that closely matches the source code. As such, the goal is stylistic rather than formal.

As an example, we might chunk JavaScript source as follows:

// Here are some line comments.
// Successive line comments should be grouped together in a chunk.
// These chunks are often markdown-formatted and displayed as documentation.

var a = 2;

var b = 2;  var c = 3;

/*
Here's a block comment. Block comments can be markdown-formatted and displayed
as documentation too.
*/

var square = function (x) {
  return x * x;
};

if (true) {
  console.log(square(3)); 
}
else {
  console.log('Not displaying the answer.');
}

Most languages have parsers available that get you most of the way to a chunker. Parsers usually output a sequence of abstract syntax tree nodes, which correspond to the code chunks, and the line and column at which each chunk begins and ends. If you can find such a parser for your language, this package will help you create a chunker relatively quickly. If your parser doesn't make a record of comments, no problem- this module will go back through the code and pick them up.

Usage

var parserFunc = function(code, cb) {
  // You have to supply this function.
  
  // If a syntax error or other error occurs during parsing, it should call
  cb("the error message")
  
  // If the code parses, it should call
  cb(false, [
    {start: {line: 0, column: 0}, end: {line: 0, column: 3}}, // the type is "code" by default.
    {start: {line: 1, column: 0}, end: {line: 10, column: 4}, type: "blockComment"}
    ...
  ])
  
  // Note that the end line and columns are inclusive upper bounds and that
  // indexing begins at zero, so the statemnt
  "var x;"
  // would be 
  {start: {line: 0, column: 0}, end: {line: 0, column: 5}}.
}

var chunk = require('comment-chunk-helper').chunk({
  parser: parserFunc,
  lineComment: "//", 
  blockComment: {left: "/*", right: "*/"}
});

chunk(code, cb);
// The single argument passed to the callback will be an array
// of chunks, that is, an argument of the form
[
  {
    type: ("code" or "error" or "comment" or "blockComment"),
    value: "the code, comment, or error message"
  }
]

Note that the chunker's callback does not take an error argument. If the parser passes an error argument to its callback, that argument will be wrapped in a single chunk of type 'error' and passed to the chunker's callback. The dashboard implementation can decide what to do with error chunks. Simply passing them to the engine as code is often a good idea.

Support

• Email: support@senseplatform.com
• Google Group: https://groups.google.com/forum/?fromgroups#!forum/sense-users
• IRC: #senseplatform on irc.freenode.net