npm-metadata-raw

Retrieve the metadata for every package on NPM and store it in a level* database

npm install npm-metadata-raw
6 downloads in the last week
11 downloads in the last month

npm-metadata-raw Flattr this!experimental

Retrieve the metadata for every package on NPM and store it in a level* database. Once you've acquired the data, you can then run your own manual queries over it very quickly and without hitting the registry.

The initial sync takes a long time, but this module only downloads the most recent changes not already stored - so after the first run it should be relatively speedy.

Usage

npm-metadata-raw

metadb = require('npm-metadata-raw')(db)

Takes a level* database db and returns a modified one for querying. Once synced, each package is stored in metadb by their name - so you could do this to get the metdata for the disc module:

var db = require('level')(__dirname + '/npm-meta')
var metadata = require('npm-metadata-raw')
var sublevel = require('level-sublevel')

var metadb = metadata(sublevel(db))

metadb.sync(function(err) {
  if (err) throw err
  metadb.get('disc', function(err, meta) {
    if (err) throw err
    console.log(JSON.parse(meta).name) // "disc"
  })
})

metadb.sync([finished])

Sync with the NPM registry, optionally passing the finished callback for when the process is complete. This must be done at least once before you can access any data.

metadb.sync().on('found', name)

When syncing, an initial pass is done over a complete list of packages in the registry. The found event is emitted once for each module in the list.

metadb.sync().on('data', json, name, date)

After the first pass, a second pass will retrieve and store the metadata for each out of date package. When retrieved, the module data is emitted under the data event.

metadb.sync().on('end')

Emitted when the whole process is complete.

License

MIT. See LICENSE.md for details.

npm loves you