linestream

a readable stream emitting lines (from files, streams)

npm install linestream
1 downloads in the last day
30 downloads in the last week
73 downloads in the last month

LineStream.js

[Node.js] ReadableStream of lines

Installation

$ npm install linestream

OR

$ git clone git://github.com/shinout/LineStream.git

sample

with file

var stream = require('linestream').create(filename, {bufferSize: 300});

stream.on('data', function(line, isEnd) {
  console.log(line); // each line comes here
  console.log(isEnd); // if it is the end of data or not.
})

stream.on('end', function() { // emitted at the end of file
  console.log('end');
});

stream.on('error', function(e) { // emitted when an error occurred
  console.error(e);
});

with Stream (like HttpResponse)

var https = require('https');
var req = https.request({host: 'github.com'}, function(response) {
  var stream = require('linestream').create(response);

  stream.on('data', function(line) {
    console.log(line); // each line comes here
  });
});
req.end();

API Documentation

LineStream extends ReadableStream.

See Node.js#Stream for ReadableStream's API.

  • LineStream.create(source, options)
  • on "data"
  • LineStream.tsv(source, [options], fn)
  • stream.after(rstream1, rstream2, ...)

LineStream.create(source, options)

Creates an instance of LineStream.

source is the one of the followings.

  • (string) filename. Then reads the file and emit each lines.
  • "-". Then resumes process.stdin and reads from it.
  • (ReadableStream) stream. Then reads lines from the stream.

(Object) options is optional.

key type description example
separator string line separator. "\n" by default.
"\r"
trim boolean If true, separator are not appended in the line. true by default.
false
filter function filter function before emitting lines.
each line is passed to the function as the first argument.
function (line) { return line.length }
comment string Registers the marks of one-line comment. If the mark comes in the first position of a line, the line is filtered. "#"
fieldSep string A field separator. It is used with fieldNum options. "\t"
fieldNum string the required number of the fields.
If not matched, the line is filtered.
6
empty boolean If true, empty lines (after trimmed) are filtered. true

Other options are passed to fs.createReadStream(filename, options) if the first argument is a string.

See fs.createReadStream()

on "data"

Data event of LineStream. Two arguments are passed.

  • line (string) each line
  • isEnd (boolean) whether the line is final or not.

example

stream.on("data", function(line, isEnd) {
  console.log([line, isEnd].join('\t'));
});

LineStream.tsv(source, [options], fn)

Creates an instance of LineStream, with field separated by "\t".

source and options are the same as LineStream.create(source, options).

fn is called on "data" event. Three arguments are passed.

  • data (Array)
  • line (string)
  • isEnd (boolean)

data is equivalent to line.split("\t")

Other arguments are the same as original "data" event.

Returns an instance of LineStream.

stream.after(rstream1, rstream2, ...)

Pauses the stream until all passed readable streams come to an end.

var ids = {};
var comingLines = LineStream.tsv('-', function(data, line, isEnd) { // reads from process.stdin
  var id = data[0];
  ids[id] = true;
});

LineStream.tsv('file1', function(data, line, isEnd) { // reads from "file1"
  var id = data[0];
  if (ids[id]) console.log(line);
})
.after(comingLines); // resumes after comingLines finished,
npm loves you