sooty

0.0.26 • Public • Published

sooty2011

sooty

Beerpay Beerpay GitHub stars

A bit of oofle dust to scrape websites

it useful ★ ★ ★

Features

  • Scrape by query selector
  • Built on headless chrome

Installation

npm install --save sooty

Dependencies

Usage

import Sooty from 'sooty';

const scrapper = new Sooty({
  url: 'https://google.com',              // go to google search page
  interactions: {
    search: [
      {                                   // fill out Google search
        fields: { q: 'funny cat videos' }
      },
      {                                   // press "Google Search" button
        click: 'input[name=btnK]',
        waitUntil: 'networkidle0'
      }
    ]
  },
  queries: {
    catVideos: {                          // query google search results
      selector: '.rc .r a',
      requires: ['search']
    }
  }
});
scrapper.run().then(results => {
  results.catVideos.forEach(catVideo => {
    console.log(`${catVideo}\n`);
  });
});

Support

Submit an issue

Screenshots

Contribute a screenshot

Contributing

Review the guidelines for contributing

License

MIT License

Jam Risser © 2018

Changelog

Review the changelog

Credits

Support on Beerpay (actually, I drink coffee)

A ridiculous amount of coffee was consumed in the process of building this project.

Add some fuel if you'd like to keep me going!

Beerpay Beerpay

Readme

Keywords

none

Package Sidebar

Install

npm i sooty

Weekly Downloads

23

Version

0.0.26

License

MIT

Unpacked Size

1.49 MB

Total Files

14

Last publish

Collaborators

  • codejamninja