Streaming high performance scraper
Want to see pretty graphs? Log in now!
npm install hyperscrape
|2||downloads in the last week|
|28||downloads in the last month|
|Last Published By|
|Version||0.2.1 last updated a month ago|
|Dependencies||request, cheerio, streamz|
Hyperscrape is a stream-component that accepts urls from upstream and pushes parsed pages downstream. If the input is an object that contains an
url property, the output will be added to that object. Each output object contains
responseHeaders and the cheerio parsed content in
Hyperscrape is initialized by two arguments. First argument is the maximum number of concurrent requests allowed and the second argument contains options for the hyperquest stream. If an
url is defined in the options object, it will be passed on to the stream as the first url to process.