scroungejs

compress, concatenate and/or timestamp your files as part of a build process for your web site.

npm install scroungejs
47 downloads in the last day
128 downloads in the last week
288 downloads in the last month

scroungejs

(c)Bumblehead, 2012,2013 MIT-license
scrounge

OVERVIEW:

Scroungejs processes scripts, stylesheets and template files.

Scroungejs reads files from a 'source' directory and writes to an 'output' directory. The output may contain minified, concatenated results from source scripts, stylesheets and templates. It is made to be flexible and features are selectively enabled as needed. Scroungejs concatenates files in an ordered way -each file appears after the files it depends on. No config file, just commented meta-data at each file.

Perform these tasks:

  • Easily choose deployment options: Concatenate, or not. Minify, or not. Timestamp, or...

  • Preprocess markup on each build, to auto add or update include elements:

    <!-- ex. include for concatenated application -->
    <script src="app/appScriptsConcatenated.js" type="text/javascript"></script>
    <!-- ex. include for unconcatenated application -->
    <script src="app/appScriptA.js" type="text/javascript"></script>
    <script src="app/appScriptB.js" type="text/javascript"></script>
    
  • Yield output for multiple applications with one group of files.

  • Preprocess scripts before writing them to 'output'

    • Remove calls to 'console.log' with precision using an ast-tree (UglifyJS2).
    • Remove node.js requires statements from scripts to share them with client

Advantages scroungejs has over other deployment tools:

  • usable as a command line tool -no config file or framework.
  • usable with node.js.
  • output does not interfere with the node.js requirejs module system.
  • compression and concatenation are enabled/disabled for each build.
    • unconcatenated, unminified are helpful during development.
  • html files do not need unusual tags or attributes added to them.
  • it works with almost any collection of javascript files.
  • by default, it does not add javascript to files it generates.
  • it is packaged with an emacs config file.

Scroungejs relies on other packages, notably: css-clean and UglifyJS2.

Read about how it works to learn more, or get started with scroungejs.


INSTALL:

Scroungejs may be downloaded directly or installed through npm.

  • npm

    $ npm install scroungejs
    
  • Direct Download

    $ git clone https://github.com/iambumblehead/scroungejs.git
    $ cd scroungejs && npm install
    

GET STARTED:

  1. Before Starting...

    'Examples demonstrate usage from a shell. Scroungejs is also usable from a javascript file. Each environment uses the same modifiers, but syntax is different. 'Both examples would produce the same output.

    shell

    $ node ./scrounge.js \  
     --inputPath=./getStarted \
     --isTimestamped=true \
     --isCompressed=true
    

    javascript file

    var scroungejs = require('scroungejs');
    scroungejs.build({  
     inputPath : './getStarted',
     isTimestamped : true,  
     isCompressed : true
    }, function (err, res) {
     if (err) return console.log(err);
     console.log('finished!')
    });
    
  2. Visit the scroungejs directory.

    $ cd /path/to/scroungejs
    
  3. Call scroungejs with node.

    $ node ./scrounge.js
    [...] read: files (2/2)
    [...] write: cmpr/scrounge.js
    [...] write: cmpr/index.js
    [...] finish: 00:00:23 (mm:ss:ms)
    
  4. Specify an input path.

    The directory named 'getStarted' and its contents are provided with the Scrounge package.

    $ node ./scrounge.js -i ./getStarted
    [...] read: files (2/2) 
    [...] write: getStarted/cmpr/fileB.js
    [...] write: getStarted/cmpr/fileA.js
    [...] finish: 00:00:25 (mm:ss:ms)
    
  5. Use compression and timestamping modifiers.

    $ node ./scrounge.js -i ./getStarted \
    --isTimestamped=true --isCompressed=true
    [...] read: files (2/2) 
    [...] ugly: (fileA.js, .js, 1/2) getStarted/fileB.js
    [...] write: getStarted/cmpr/fileB_2012.07.07-15:25:57.js
    [...] ugly: (fileB.js, .js, 2/2) getStarted/fileA.js
    [...] write: getStarted/cmpr/fileA_2012.07.07-15:25:46.js
    [...] finish: 00:00:25 (mm:ss:ms)
    
  6. Define a dependency in fileB.js.

    scroungejs will concatenate dependency-related files. Dependencies are defined in .js and .css files using the 'Requires' property.

    Open fileB.js and add the following line to the top: // Requires: fileA.js.

    File properties are explained in section File Properties.

    ./getStarted/fileB.js:

    // Requires: fileA.js
    
  7. Concatenate files

    Dependency-related files are recognized as a tree. A tree is composed of one file that depends on other files 'and so on. Here fileB.js is a source file that begins a dependency tree.

    When tree information is collected a visual representation of each tree is printed.

    isConcatenated may be defined as true or false or it may be defined with comma separated names of trees or extension types, such as '.js'. Only matching files will be concatenated.

    $ node ./scrounge.js -i ./getStarted --isConcatenated=true
    [...] read: files (2/2)  
    
    fileB.js  
    └── fileA.js  
    
    [...] join: (fileA.js, .js, 1/2) getStarted/fileA.js
    [...] join: (fileB.js, .js, 2/2) getStarted/fileB.js
    [...] write: getStarted/cmpr/fileB.js
    [...] finish: 00:00:27 (mm:ss:ms)
    
  8. Specify an Output Directory

    If a specified output directory does not exist it is created.

    $ node ./scrounge.js -i ./getStarted --isConcatenated=true \
    --outputPath=./app/public/cmpr
    [...] read: files (2/2)  
    
    fileB.js  
    └── fileA.js  
    
    [...] join: (fileA.js, .js, 1/2) getStarted/fileA.js
    [...] join: (fileB.js, .js, 2/2) getStarted/fileB.js
    [...] write: app/public/cmpr/fileB.js
    [...] finish: 00:00:27 (mm:ss:ms)
    
  9. Build a larger tree found in ./getStarted/app/.

    Multiple trees may be discovered and concatenated and .css files that associate with a tree will be discovered.

    $ node ./scrounge.js -i ./getStarted/app --isConcatenated=true \
    --outputPath=./app/public/cmpr --isRecursive=true --trees=app.js
    [...] read: files (12/12)  
    
    app.js
    └─┬ ViewsAll.js
    . ├─┬ ViewB.js
    . │ └─┬ CtrlsAll.js
    . │   ├─┬ CtrlB.js
    . │   │ └── ModelB.js
    . │   └─┬ CtrlA.js
    . │     └── ModelA.js
    . └── ViewA.js  
    
    [...] join: (app.js, .css, 1/1) getStarted/app/views/ViewA.css
    [...] write: app/public/cmpr/app.css
    [...] join: (app.js, .js, 1/9) getStarted/app/models/ModelB.js
    [...] join: (app.js, .js, 2/9) getStarted/app/controls/CtrlB.js
    [...] join: (app.js, .js, 3/9) getStarted/app/models/ModelA.js
    [...] join: (app.js, .js, 4/9) getStarted/app/controls/CtrlA.js
    [...] join: (app.js, .js, 5/9) getStarted/app/controls/CtrlsAll.js
    [...] join: (app.js, .js, 6/9) getStarted/app/views/ViewB.js
    [...] join: (app.js, .js, 7/9) getStarted/app/views/ViewA.js
    [...] join: (app.js, .js, 8/9) getStarted/app/views/ViewsAll.js
    [...] join: (app.js, .js, 9/9) getStarted/app/app.js
    [...] write: app/public/cmpr/app.js
    [...] finish: 00:00:17 (mm:ss:ms)
    
  10. Define filters for the build process.

    When --extensionType=.js is given, files with other extensions are ignored by the build process.

    modifiers are explained in section Modifiers.

    $ node ./scrounge.js -i ./getStarted/app --isConcatenated=true \
    --outputPath=./app/public/cmpr --isRecursive=true --tress=app.js \
    --extensionType=.js
    [...] read: files (12/12)  
    
    app.js
    └─┬ ViewsAll.js
    . ├─┬ ViewB.js
    . │ └─┬ CtrlsAll.js
    . │   ├─┬ CtrlB.js
    . │   │ └── ModelB.js
    . │   └─┬ CtrlA.js
    . │     └── ModelA.js
    . └── ViewA.js  
    
    [...] join: (app.js, .css, 1/1) getStarted/app/views/ViewA.css
    [...] write: app/public/cmpr/app.css
    [...] join: (app.js, .js, 1/9) getStarted/app/models/ModelB.js
    [...] join: (app.js, .js, 2/9) getStarted/app/controls/CtrlB.js
    [...] join: (app.js, .js, 3/9) getStarted/app/models/ModelA.js
    [...] join: (app.js, .js, 4/9) getStarted/app/controls/CtrlA.js
    [...] join: (app.js, .js, 5/9) getStarted/app/controls/CtrlsAll.js
    [...] join: (app.js, .js, 6/9) getStarted/app/views/ViewB.js
    [...] join: (app.js, .js, 7/9) getStarted/app/views/ViewA.js
    [...] join: (app.js, .js, 8/9) getStarted/app/views/ViewsAll.js
    [...] join: (app.js, .js, 9/9) getStarted/app/app.js
    [...] write: app/public/cmpr/app.js
    [...] finish: 00:00:12 (mm:ss:ms)
    
  11. Controlling the tree display.

    To make scroungejs display the tree and stop, use the modifier stop=tree

    By default, each dependency is only displayed once. This makes the tree smaller when, for example, a library is required by many files. To view the entire tree, use the modifier treeView=full

    $ node ./scrounge.js -i ./getStarted/app --isConcatenated=true \
    --isRecursive=true --stop=tree --treeView=full
    [...] read: files (12/12)
    
    app.js
    └─┬ ViewsAll.js
    . ├─┬ ViewB.js
    . │ └─┬ CtrlsAll.js
    . │   ├─┬ CtrlB.js
    . │   │ └── ModelB.js
    . │   └─┬ CtrlA.js
    . │     └── ModelA.js
    . └─┬ ViewA.js
    .   └── CtrlsAll.js
    
  12. Update index.mustache using scroungejs.

    The index file contains a scrounge element.

    ./getStarted/index.mustache

    <!doctype html>
    <html>
      <head>
        <script src="/scr/libFile.js" type="text/javascript"></script>
      </head>
      <body>  
        <!-- <scrounge type=".js"> -->
        <!-- </scrounge> -->
      </body>
    </html>
    
  13. modify the basepage index.mustache with scroungejs

    Call scroungejs without concatenation on the basepage.

    using -s makes the output 'silent'

    $ node ./scrounge.js -s -i ./getStarted/app \
    --basepage=./getStarted/index.mustache --isRecursive=true \
    --outputPath=./app/public/cmpr
    

    ./getStarted/index.mustache

    <!doctype html>
    <html>
    <head>
    <script src="/app/lib/library.js" type="text/javascript"></script>
    </head>
    <body>
    <!-- <scrounge type=".js"> -->
    <script src="/app/public/cmpr/library.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/app2.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/ModelB.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/CtrlB.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/ModelA.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/CtrlA.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/CtrlsAll.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/ViewB.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/ViewA.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/ViewsAll.js" type="text/javascript"></script>
    <script src="/app/public/cmpr/app.js" type="text/javascript"></script>     
    <!-- </scrounge> -->
    </body>
    </html>
    
  14. modify the basepage index.mustache and use concatenation.

    Call scroungejs with concatenation on this basepage. scroungejs correctly modifies the basepage for concatenated files.

    $ node ./scrounge.js -s -i ./getStarted/app --isConcatenated=true \
    --basepage=./getStarted/index.mustache --isRecursive=true \
    --outputPath=./app/public/cmpr
    

    ./getStarted/index.mustache

    <!doctype html>  
    <html>  
    <head>  
        <script src="/app/lib/library.js" type="text/javascript"></script>  
    </head>  
    <body>  
        <!-- <scrounge type=".js"> -->  
        <script src="/app/public/cmpr/library.js" type="text/javascript"></script>  
        <script src="/app/public/cmpr/app2.js" type="text/javascript"></script>  
        <script src="/app/public/cmpr/app.js" type="text/javascript"></script>  
        <!-- </scrounge> -->  
    </body>  
    </html>
    
  15. define a public path and update index.mustache.

    This index page may reference a root web directory served from app/public. In this case, files copied to /app/public/cmpr/ should be accessed through the path /cmpr.

    Scroungejs will modify a basepage to reference scripts from a publicPath.

    $ node ./scrounge.js -s -i ./getStarted/app --isConcatenated=true \
    --basepage=./getStarted/index.mustache --isRecursive=true \
    --outputPath=./app/public/cmpr --publicPath=/cmpr
    

    ./getStarted/index.mustache

    <!doctype html>  
    <html>  
    <head>  
        <script src="/app/lib/library.js" type="text/javascript"></script>
    </head>  
    <body>  
        <!-- <scrounge type=".js"> -->  
        <script src="/cmpr/library.js" type="text/javascript"></script>
        <script src="/cmpr/app.js" type="text/javascript"></script>  
        <script src="/cmpr/app2.js" type="text/javascript"></script>  
        <!-- </scrounge> -->  
    </body>  
    </html>
    
  16. define tree attributes.

    If a tree attribute is defined in a scrounge element, only the file or files that result from the named tree are added to the scrounge element.

    Trees may be defined in a scrounge element attribute or by using the --trees modifier. When trees are defined, scrounge will output the defined trees only.

    ./getStarted/index.mustache

    <!doctype html>  
    <html>  
    <head>  
        <script src="/app/lib/library.js" type="text/javascript"></script>  
        <!-- <scrounge type=".css" trees="app.js"> -->     
        <!-- </scrounge> -->  
    </head>  
    <body>  
        <!-- <scrounge type=".js" trees="app.js"> -->  
        <!-- </scrounge> -->  
    </body>  
    </html>
    
    $ node ./scrounge.js -s -i ./getStarted/app
    --basepage=./getStarted/index.mustache --isRecursive=true --publicPath=/cmpr
    

    ./getStarted/index.mustache

    <!doctype html>  
    <html>  
    <head>  
        <script src="/app/lib/library.js" type="text/javascript"></script>  
        <!-- <scrounge type=".css" trees="app.js"> -->
        <script src="/cmpr/app.css" type="text/javascript"></script>  
        <!-- </scrounge> -->     
    </head>  
    <body>  
        <!-- <scrounge type=".js" trees="app.js"> -->  
        <script src="/cmpr/app.js" type="text/javascript"></script>  
        <!-- </scrounge> -->  
    </body>  
    </html>
    
  17. You are now ready to use scroungejs.

    scrounge


FILE PROPERTIES:

Each file processed by scroungejs may affect the build process, with meta-data added to each file.

./getStarted/app/views/ViewA.js

   // Filename: ViewA.js
   // Timestamp: 2011.06.03-22:10:20
   // Author(s): Bumblehead (www.bumblehead.com)
   // Requires: CtrlsAll.js, ModelA.js

./getStarted/app/views/ViewA.css

   /* Filename: Main.css
    * Timestamp: 2011.06.03-22:10:20
    * Author(s): Bumblehead (www.bumblehead.com)
    */
  • Filename: filename
    by default, filename is the file's system filename.
    the file will be recognized by this name when processed by scroungejs.

  • Requires: filename, more filenames
    by default, file has no dependencies.
    the file depends on files with these filenames.

  • Timestamp: YYYY.MM.DD-H:MM:SS
    by default, timestamp is Date.now().
    the file will associate with this timestamp. concatenated files will produce a file using the most recent timestamp found.

  • Authors: authorname, more authornames
    by default, file associates with no author.
    the file associates with the given author(s). concatenated files will produce a file that associates with all defined authors.

  • DoNotCompress: bool
    by default, false If true, scroungejs will skip compression of this file.


BASEPAGE:

scroungejs may add include elements for the js and css files it processes. A basepage containing one or more scrounge elements will be modified.

example.html

   <!doctype html>  
   <html>  
     <head>  
       <!-- <scrounge type=".js"> -->  
       <!-- </scrounge> -->  
     </head>  
     <body></body>   
   </html>

scroungejs adds js/css include elements in the body of scrounge elements. Something like the following could be added to the scrounge element above.

example.html

  <!-- <scrounge type=".js"> -->
  <script src="cmpr/app1.js" type="text/javascript"></script>
  <script src="cmpr/app2.js" type="text/javascript"></script>
  <script src="cmpr/app3.js" type="text/javascript"></script>
  <!-- </scrounge> -->
  `

Each time you modify a basepage, the body of the tags is remade by scroungejs. A 'tree' attribute will affect focus this process.

example.html

  <!-- <scrounge type=".css" tree="app1.js,app2"> -->
  <script src="cmpr/app1.js" type="text/javascript"></script>
  <script src="cmpr/app2.js" type="text/javascript"></script>
  <!-- </scrounge> -->

example scrounge elements are given below

example.html

  <!-- <scrounge type=".css" tree="Map.css,Main.js"> -->
  <!-- </scrounge> -->
  <!-- <scrounge type=".js" tree="Main.js,Crypto.js"> -->
  <!-- </scrounge> -->
  <!-- <scrounge type=".css" tree="Main.js"> -->
  <!-- </scrounge> -->

MODIFIERS:

  • --inputPath= path, -i path, default: ./

    a systempath to a directory or file. multiple paths may be given, separated with a comma, for example: --inputPath=./path/1,./path/2.

  • --outputPath= path, -o path, default: ./cmpr

    a systempath to a directory or file.

  • --publicPath= path, -p path, default: null
    a path to the files created by scrounge. examples describe it best

    for the file ./getStarted/app/cmpr/app.js, with publicPath /cmpr/app.js:

    a resulting basepage element:

    <script src="/cmpr/app.js" type="text/javascript"></script>';
    

    for the file ./getStarted/app/cmpr/app.js, with publicPath http://www.site.com/cmpr/app.js:

    a resulting basepage element:

    <script src="http://www.site.com/cmpr/app.js" type="text/javascript"></script>';
    

    a public path is not useful without a --basepage argument as it affects only paths generated for the specified basepage.

  • --isBasepageSourcePaths= bool, default: false
    basepage include tags will reference scripts in source directory. disables compression, concatenation and copying of files. uses publicPath

    ex.

    $ node scrounge.js -l --isRecursive=true \        
      --isBasepageSourcePaths=true \  
      --basepage=~/Software/kuaweb/sources/index.html \  
      --inputPath=~/Software/kuaweb/sources/appSrc --publicPath=/appSrc
    
  • --isSourcePathUnique= bool, default: false
    add a unique argument to reference paths in include elements

    if enabled, an include element may appear as follows:

    <script src="Main.js?u=1370491167925" type="text/javascript"></script>';
    <link href="Main.css?u=1370491167926" rel="stylesheet" type="text/css">
    
  • --extnTemplate= str, default: ''
    prompts scroungejs to recognize the given string as an extention for template files. For example, --extnTemplate=.mustache.

    If ViewSignin.js is included in scroungejs' output, ViewSignin.mustache will be included as well and it will be copied to the specified output directory.

  • --extnStylesheet= str, default: '.css'
    prompts scroungejs to recognize the given string as an extention for stylesheet files. For example, --extnStylesheet=.less.

    If ViewSignin.js is included in scroungejs' output, ViewSignin.less or maybe viewSignin.css will be included as well and it will be copied to the specified output directory.

    .less support is recently added and does not support cross-file references.

    Only .css and .less are supported at this time.

  • --isCompressed= bool, extension, tree, default: false
    compress all files and/or trees before writing them to disk. If an extension or a treename are given only files that associate with the treename or extension will be compressed.

  • --isConcatenated= bool, extension, tree, default: false
    concatenate all js/css files that form a tree before writing them to disk. If an extension or a treename are given only files that associate with the treename or extension will be concatenated.

  • --isRecursive= bool, default: false
    discover files in nested directories on the given path.

  • --isWarning= bool, default: false
    raise warnings around trace statements.

  • --isLines= bool, default: false
    each compressed script on its own line.

  • --isClosure= bool, default: false
    compressed js code will added in the body of anonymous self-calling function.

  • --isSilent= bool, -s, default: false
    supress console messages.

  • --isMintFilter= bool, default: false
    only process files that include _mint in their filename. this should only be used for the special case when a build directory has many files that should not be included in the build process. _mint distinguishes a file that should be included in the build process from a file that should not be included in the build process.

  • --isTimestamped= bool, extension, tree, default: false
    add a timestamp to name of the output files. If an extension or treename are given only files that associate with the treename or extension will be timestamped

  • --isRemoveRequires= bool, default: true
    remove 'requires' statements from javascript files. true by default.

  • --isRemoveConsole= bool, default: false
    remove 'console.log' statements from javascript files. uses UglifyJS2 ast to comprehensively remove console.log

  • --isUpdateOnly= bool, default: false
    modify include tags in a basepage only. do not build scrounge elements.

  • --extensionType= type, -t type, default: null
    process files of one type only, js or css.

  • --forceTimestamp= timestamp, default: null
    all timestamped files will use the given timestamp.

  • --basepage= basepage, -b basepage, default: null
    update scrounge tags in the defined basepage.

    scroungejs will not modify or process a file that does not contain scrounge tags.


License:

scrounge

(The MIT License)

Copyright (c) 2012 Bumblehead chris@bumblehead.com

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

npm loves you