node-crawler

Node.JS Multithreaded Web Crawler with rules to parse site

npm install node-crawler
5 downloads in the last day
17 downloads in the last week
51 downloads in the last month

Web Crawler

Run

node .bin/crawle.js \
    --start-point="http://dom.ria.ua/ru/search?period=per_day&limit=100&category=0&with_photo=1&page=0" \
    --rule-config=rules.js \
    --rule-group=dom.ria.ua \
    --rule-index=index
npm loves you