easy-crawler

curling data form resource.

Install

npm install easy-crawler

Docs

declare a new DataCurl object and aimed the file save folder

var data = new DataCurl({
    "file_save": "./data_save", 
    "input_file": "./sample/task"
});

Task file

add ing a line with the following params. Seperate these params with comma.

• url
• data name

see sample:

https://github.com/chilijung/easy-crawler/blob/master/sample/task

File structure( while saving )

the root file while saving should be the data name of the data, inside the folder should save the files with timestamps.

Sample data

http://cloud.culture.tw/frontsite/trans/SearchShowAction.do?method=doFindTypeJ&category=8,test-data

• url
• data name: data name can't have space

License

MIT @chilijung