NStar Bills Table
Module to interact with the bills table on the NGrid account homepage. This module uses cheerio to parse the account homepage html
Installation
npm install -S nstar-bills-table
Usage
var billsTable = var $ = 'account homepage html parsed by cheerio'var pageNumbers = billsTablevar billNumbers = billsTable
Methods
The following methods are supported by the module
getPageNumbers
getBillNumbers
loadNextPage
viewBill
getPageNumbers
The bills table can have 1 or more pages. The page number appears in the bottom left corner with text that looks like "Page 1 of 3". The getPageNumbers
method takes the account homepage html parsed by cheerio as a parameter and returns an object with the page number data
var billsTable = var inspect = var $ = 'account homepage html parsed by cheerio'var pageNumbers = billsTable
In the example above, pageNumbers is an object that looks like the following. Note that pageNumbers uses 1-based indexing to match with what appears on the NStar website
var pageNumbers = currentPageNumber: 1 // 1-based index numPages: 3
getBillNumbers
The bills table can have 0 or more bill rows. The getBillNumbers
method takes the account homepage html parsed by cheerio as a parameter and returns an array of bill numbers
var billsTable = var inspect = var $ = 'account homepage html parsed by cheerio'var billNumbers = billsTable
In the example above, billNumbers is an array that looks like the following...
var billNumbers = "bf3fa73d284759d702564b355a1b29b6" "2c32c071f784980f5cf2afa9ef06b5eb"
viewBill
When the scraper finds a new bill in the bills table on the account homepage. That bill must be downloaded and parsed. To download the bill, use the viewBill
function. In the bills table, in each row there is a *View Bill button. When you click on this button your browser issues a post request. The viewBill
function here emulates that same post request to view the bill.
When submitting the post requestion the following fields are required. Note that the number of __VIEWSTATE
fields can vary. The __VIEWSTATE#
, __PREVIOUSPAGE
and __EVENTVALIDATION
values are extracted from the account homepage html
"__VIEWSTATEFIELDCOUNT": "dsoifjs" "__VIEWSTATE1": "aazbsan2" "__VIEWSTATE2": "masdfn0adfsjh2nz0j32" "__EVENTVALIDATION": "sd01nsd023mdsf9k32msal" "__PREVIOUSPAGE": "asdfasfde" "ctl00$Main$lvBills$ctrl0$btViewB.x": "1" "ctl00$Main$lvBills$ctrl0$btViewB.y" "1"
The x & y values in the keys ctl00$Main$lvBills$ctrl0$btViewB.x
and ctl00$Main$lvBills$ctrl0$btViewB.y
keys refer to the pixel location of where the user clicked on the View Bill button. The values of x & y are thus irrelevant for our use here. The important part is the ctrl0
part of the key. The first bill in the table will have ctrl0
, then second bill row will have crtl1
, third crtl2
and so on.
When submitting the post form, NStar requires that a valid user agent header by supplied
loadNextPage
The bills table on the account homepage is paginated with about 12 bills per page. If there are more then 12 bills available for a a given login, the bills will be split across multiple pages of the bills table. To switch pages, this module issues a post request as required by the NStar server
When submitting the post request, the following fields are required. Note that the number of __VIEWSTATE
fields can vary. The viewstates are extracted from the account homepage
"__EVENTTARGET": "ctl00$Main$lvBills$viewPager" "__EVENTARGUMENT": "2" // the desired page index (1-based indexing) "__VIEWSTATEFIELDCOUNT": "dsoifjs" "__VIEWSTATE": "rrtisp3kds92mvmvvzpal3" "__VIEWSTATE1": "aazbsan2" "__VIEWSTATE2": "masdfn0adfsjh2nz0j32" "__EVENTVALIDATION": "sd01nsd023mdsf9k32msal" "__PREVIOUSPAGE": "dsdfmxcv09jasdf"
The __EVENTTARGET
should always be the same static value of ctl00$Main$lvBills$viewPager
. The __EVENTARGUMENT
value should match the desired bills table page that the scraper is requesting. Thus to go to the third page, __EVENTARGUMENT = '3'
As with other requests to the NStar website, when submitting the post form, NStar requires that a valid user agent header be supplied as well