nf
If you've used perl oneliners before you're expecting when you pass -pe
to
node that you'll evaluate the script once per input line
$ echo -e "foo\nbar" | perl -pe '"baz"'
foo
bar
However if you try it:
$ echo -e "foo\nbar" | node -pe '"baz"'
baz
nf
is a small package that lets you perform arbitrary javascript/node actions
once per input line:
$ npm install -g nf
$ echo -e "foo\nbar" | nf -pe '"baz"'
baz
baz
Valid command line arguments:
-e
-- evaluate the given script (if not passed assumes last argument is a javascript file to load)-n
-- evaluate the given script once per input line, the current line is available in the global__line
-p
-- evaluate the given script once per input line with an implicit print of the return value (implies-n
)
Here's a small example interpreting the current line
$ echo -e "foo\nbaz\nfoobar" | nf -pe '__line.replace(/foo/, "bar")'
bar
baz
barbar
Here is apache.js
which transforms an apache log line to json:
var pattern = /^\s+\s+\s+\[\]\s+\s+\s+\s+""\s+\s+""\s+""\s+""$/; { var m = line; if !m return JSON; var obj = host: m1 a: m2 user: m3 date: m4 method: m5 url: m6 version: m7 statusCode: m8 contentLength: m9 refer: m10 userAgent: m11 ; return JSON;}__line;
then using jsontool you can find the uniq
userAgents
cat access.log | nf -p ./apache.js | json -ga userAgent | sort -r | uniq | head 10
Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:21.0) Gecko/20100101 Firefox/21.0
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.110 Safari/537.36
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/536.30.1 (KHTML, like Gecko) Version/6.0.5 Safari/536.30.1
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.37 Safari/537.36
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.110 Safari/537.36
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.110 Safari/537.36
Mozilla/4.0 (compatible;)
scripts
Scripts (either in string form or from a file) are not the same thing as a node
module; most but not all the globals you expect are there. You can require
in
your script, but notably exports
and module
are missing as their utility
would be suspect.