nodutils

Node utilities to manage common operations over: string, numeric, array, date, file, url, cache, properties files, geocoding

npm install nodutils
1 downloads in the last week
23 downloads in the last month

Utilities for NodeJS

Installation

            npm install nodutils

Usage

            var utils = require("nodutils");

String

  • utils.string.trim(str) or String.trim()

  • utils.string.ltrim(str) or String.trim()

  • utils.string.rtrim(str) or String.trim()

  • utils.string.toi(str) or String.toi()

    • Converts to integer
  • utils.string.tof(str[,decimals]) or String.tof([decimals])

    • Converts to float, with number of decimals
  • utils.string.dropDiacritics(str) or String.dropDiacritics()

    • Converts accents, diacritics into a plain letter
  • utils.string.isNumber(str) or String.isNumber()

  • utils.string.stripHtml(str) or String.stripHtml()

    • Strip all html tags and leaves only text
  • utils.string.count(str,substr[,flags]) or String.count(substr[,flags]).

    • It counts the number of ocurrences of substr. Flags can be "i" (ignore case) and/or "d" (drop accents)
  • utils.string.reverse(str) or String.reverse()

  • utils.string.toHtml(str) or String.toHtml()

    • Converts diacritics and almost all chars into html entities
  • utils.string.fromHtml(str) or String.fromHtml()

    • Converts into diacritics html encoded entities

Number

  • utils.number.round(num[,decimals]) or Number.round([decimals])

    • Rounds number to the given number of decimals

Array

  • utils.array.max(array) or Array.max()

    • Returns the max value in an array of numbers
  • utils.array.min(array) or Array.min()

    • Returns the min value in an array of numbers
  • utils.array.uniques(array) or Array.uniques()

    • Returns an array of uniques values in the original array
  • utils.array.aggregate(array) or Array.aggregate()

    • Returns an array of uniques values and counts its occurrences, sorted descending (sample code)

         ["a","b","c","d",1,2,"a","b","c","a","b","a",1,0,1].aggregate();
      
         returned array
      
         [ [ 'a', 4 ], [ '1', 3 ], [ 'b', 3 ], [ 'c', 2 ], [ '0', 1 ], [ '2', 1 ], [ 'd', 1 ] ]
      
  • utils.array.contains(array,value[,flags]) or Array.contains(value[,flags])

    • Returns true or false

    • Flags can be "i" (ignore case) and/or "d" (drop accents)

Date

  • utils.date.diff(date1,date2[,unit])

    • Unit = "d": days,"h": hours,"m": minutes,"s": seconds. Default unit is millis
  • utils.date.millis([date])

    • Returns a timestamp in millis from current date or date passed as param (string or date object)
  • utils.date.frommillis(millis)

    • Returns a date object from millis passed as parameter

File

  • utils.file.write(file,data[,options],function(err){})

    • options = "w" write or "a" append
  • utils.file.read(file[,encoding],function(err,data){})

  • utils.file.exists(file,function(exists){})

    • exists: true or false
  • utils.file.getModTime(file,function(date){})

    • Date object is given to the callback as an argument
  • utils.file.remove(file,callback)

  • utils.file.createpath(path,callback)

Url

            //get sample
            utils.url.get("www.bbc.com",function(html){
                if(html){
                    console.log(html);
                }
            });            

            //post sample
            utils.url.post('httpbin.org/post',{post_data:{data:'lorem ipsum dolor sit amet'}}, 
                function(resp){
                    if(resp){
                        console.log(resp);
                    }
                }
            );
  • utils.url.get(url[,options],function(content,statuscode,headers){})

  • utils.url.post(url[,options],function(content,statuscode,headers){})

    • Support for http and https
    • Support for proxy requests (in url. E.g: "url"=http://www.proxy.com:8080/www.urltobeproxied.com)
    • It is possible to set only url or options, but options need to set host, path, ...
    • Options is an object with some props:

      • "encoding" default is "utf-8"
      • "post_data" (for post()) is an object

        with the vars (post_data:{a:1,b:2})

        or with the body in "data" key (post_data:{data:"whatever"})

      • "headers" (object)

      • "auth" (string)
      • "forceparse": if the url is with proxy data is better to set to true

Cache

  • utils.cache.getPath()

    • Get the current cache dir (default is "./cache")
  • utils.cache.setPath(path,callback)

    • Set the cache dir (and create if it doesn't exists)

    • It's recommended to use absolute paths ("/apps/myapp/cache")

  • utils.cache.setOptions({path : "/mypath", size : 1}},callback)

    • Set the cache dir (and create if it doesn't exists) and cache max size

    • Cache size is in MB

  • utils.cache.set(key,data[,expiretime],callback)

    • Expiretime is in seconds. If not informed, then unlimited
  • utils.cache.get(key,function(value){})

  • utils.cache.remove(key,callback)

(sample code)

Properties

  • utils.props.load(path_to_file,function(props){})

    • properties in the file are loaded into a JSON object passed in callback

    • Format of the properties file is

            key1=value1
            key2=value2
            key3=value3
            key4=123456789
      
  • utils.props.save(path_to_file,properties,callback)

    • properties param is a json object. You can save properties dynamically

Geo

Utilities for basic geocoding and checking distance between points. Limited to Google and Yahoo API limits, but you could geocode 50.000 addresses/points for Yahoo and 2.500 for Google. Utils.geo will use one or other depending of availability of the service. (sample code)

  • utils.geo.getDistance(point1,point2[,earthRadius])

    • Calculates the distance between point1 and point2. Points are arrays: [latitude,longitude]

    • Distance is in km.

    • earthRadius default is 6371 km.

  • utils.geo.isInside(point,center,radius[,earthRadius])

    • Checks if a point (array of two positions [latitude,longitude]) is inside a circle (center + radius). Center is an array of two positions.

    • radius is in KM

    • earthRadius default is 6371 km.

  • utils.geo.geocode(address,options,callback)

    • Returns an array of two positions [latitude,longitude] for the given address.
  • utils.geo.rgeocode(latitude,longitude,options,callback)

    • Reverse geocode. Returns an object for the given latitude and longitude.

Both, geocode and rgeocode return the same object

            { 
                number : 'xxxxxxxxxxxx',
                street: 'xxxxxxxxxxxx',
                postal: 'xxxxxxxxxxxx',
                city: 'xxxxxxxxxxxx',
                county: 'xxxxxxxxxxxx',
                state: 'xxxxxxxxxxxx',
                country: 'xxxxxxxxxxxx',
                lat: 1.11,
                lon: 2.22 
            }

You can set the locale in the options object {"locale" : "en_GB"}

Samples

  • Caching twitter request due to twitter api limits (it uses url and cache utilities)

           var utils = require("nodutils");
           var twitterquery = "davidayalas";
           var twitterurl = "http://api.twitter.com/1/statuses/user_timeline.json?screen_name=";
    
           utils.cache.get(twitterquery, function(content){
               if(!content){
                   utils.url.get(twitterurl+twitterquery,function(result){
                       utils.cache.set(twitterquery,result,300);
                       console.log(result);
                   });
               }else{
                   console.log(content);
               }
           });
    
  • Easy "tagcloud" from url content (it uses url, string and array utilities)

              var utils = require("nodutils");
    
              utils.url.get("www.bbc.com",function(content){
                  var topwords = [];
                  if(content){
                      var topwords = content.stripHtml().split(" ").aggregate().filter(function(i){
                          return i[0].length<=3 || i[0].indexOf("&")>-1?false:true;
                      }).slice(0,50);
                  }
                  console.log(topwords);
              });  
    
  • Calculate the distance between two unestructured addresses (it uses geo)

              var utils = require("nodutils");
    
              utils.geo.geocode("madrid,spain", function(p1){
                  utils.geo.geocode("barcelona,spain", function(p2){
                      console.log("distance between Madrid and Barcelona is: ");
                      if(p1 && p2){
                          console.log(utils.geo.getDistance([p1.lat, p1.lon],[p2.lat, p2.lon]) + " km");
                      }
                  });
              });
    
  • Simple http server for caching and serve only valid twitter users (stored in properties file)

  • Bulk geocode from url content. Tested with more than 5000 addresses. (it uses geo, url, string, file)

              var utils = require("nodutils");
    
              utils.url.get("http://w20.bcn.cat/opendata/DonaRecurs.aspx?arbre=general&recurs=TAULA_CARRERS&fitxer=1121",{"encoding":"iso-8859-1"}, function(content,code){
                  if(content){
                      content.split("\r\n").forEach(
                          function(element, index, array){
                              var add = element.slice(element.lastIndexOf(";")+1);
                              if(add){    
                                  utils.geo.geocode((add.count("carrer","i")>0?add:"Carrer "+add)+",barcelona,spain", function(p){
                                      if(p && typeof(p)=="object" && p.lat && p.lon){
                                          console.log(element + ";" + p.lat + ";" + p.lon);
                                          utils.file.write(__dirname + "/geocoded-addresses.txt", element + ";" + p.lat + ";" + p.lon + "\r\n", "a");
                                      }
                                  });
                              }
                          }
                      );
                  }
              });
    
npm loves you