github-resource-converter
Convert GitHub Issues and Pull Requests to JSON and CSV from a Terminal or Node.js app.
Table of contents
- 1. Overview
- 2. Installation
- 3. Usage
- 4. API
- 4.1.
grc.authenticate({token, type, key}): void
- 4.2.
grc.getAll({owner, repo}): Promise
- 4.3.
grc.issues.getForRepo({owner, repo}): Promise
- 4.4.
grc.logger
- 4.5.
grc.options
- 4.6.
grc.pullRequests.getForRepo({owner, repo}): Promise
- 4.7.
grc.save({data, dest}): Promise
- 4.8.
grc.toCsv({data=[]}): Promise
- 4.1.
- 5. Version
- 6. Contributing
- 7. License
1. Overview
github-resource-converter
(alias grc
) exports your GitHub and GitHub Enterprise repositories' Issues and Pull Requests to CSV
and JSON
file formats. It's helpful whenever you need to:
-
Use spreadsheets to analyze, modify, print, or summarize large amounts of data with Pivot Tables and other important financial or statistical operations
Example:
# Convert all GitHub issues to CSV:$ github-resource-converter \--owner foo \--repo bar# => Saved "foo-bar-issues-export.2018-03-20T02_11_04_356Z.csv". -
Share data with other tools like GitLab and JIRA.
Example:
# Convert and save all GitHub Enterprise# Pull Requests as JSON (using the grc alias):$ grc \--owner gregswindle \--repo github-resource-converter \--resource-type pr \--dest './docs/reports/export.json'# Saved "docs/reports/gregswindle-github-resource-converter-pr-export.2018-03-20T02_18_33_682Z.json".
2. Installation
- Required:
github-resource-converter
is written in JavaScript (CommonJS) for Node.js version
which must be installed prior to use. Node.js requires npm, which is used for installing dependencies. (npm installs with Node.js.)
-
Recommended: To avoid rate-limiting, you should create a personal access token and save your personal access token.
-
MacOS and Unix
$ mkdir -p /usr/local/etc/github-resource-center/envvars/$ touch /usr/local/etc/github-resource-center/envvars/.env$ echo "GITHUB_ACCESS_TOKEN="{your-personal-access-token-value}" > \/usr/local/etc/github-resource-center/envvars/.env -
Windows
> md -p C:\usr\local\etc\github-resource-center\envvars\> touch C:\usr\local\etc\github-resource-center\envvars\.env> echo "GITHUB_ACCESS_TOKEN="{your-personal-access-token-value}" >>C:\usr\local\etc\github-resource-center\envvars\.env
-
2.1. For Terminal/command-line usage
# Install globally to execute from a Terminal/command-line $ npm i -g github-resource-converter
2.2. As a application dependency
# Install as a dependency within a Node.js app $ npm i --save github-resource-converter
3. Usage
The following examples assume that
github-resource-converter
is installed globally and invoked from a Terminal (command-line interface)
3.1. Formatting
--dest export.[csv|json]
You can convert GitHub (Enterprise) Issues and Pull Requests into two file formats:
CSV
andJSON
.
- CSV is the default format.
- JSON formatting requires a
--dest
value with a.json
file extension.
3.2. Resource types
--resource-type, -t
Convert and export GitHub (Enterprise) by
--resource-type
:issues
,pull_requests
, or both (all
).
-
issues
is the defaultresource-type
. -
prs
require a--resource-type
or-t
value ofpr
prs
pull_request
pull_requests
-
--resource-type all
will export allissues
andprs
into a single file.
3.3. Filtering
Filtering is currently unavailable.
If you're interested in CONTRIBUTING to features like filters--e.g., only select "open" issues--we're happily accepting pull requests!
3.4. Exporting
3.4.1. Issues
-
CSV
GitHub:
# GET https://api.github.com/rrepos/:owner/:repo/issues$ github-resource-converter \--owner gregswindle \--repo github-resource-converter \--dest './docs/reports/export.csv'GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/issues$ grc --base-url https://api.ecorp.com/api/v3 \--owner evilcorp \--repo ecoin -
JSON
GitHub:
# GET https://api.github.com/repos/:owner/:repo/issues$ github-resource-converter \--owner gregswindle \--repo github-resource-converter \--dest './docs/reports/export.json'GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/issues$ grc --base-url https://api.ecorp.com/api/v3 \--owner evilcorp \--repo ecoin--dest ./export.json
3.4.2. Pull Requests
-
CSV
GitHub:
# GET https://api.github.com/repos/:owner/:repo/pulls$ github-resource-converter \--owner gregswindle \--repo github-resource-converter \--resource-type prGitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/pulls$ grc --base-url https://api.ecorp.com/api/v3 \--owner evilcorp \--repo ecoin \--resource-type pr -
JSON
GitHub:
# GET https://api.github.com/repos/:owner/:repo/pulls$ github-resource-converter \--owner gregswindle \--repo github-resource-converter \--resource-type pr--dest './export.json'GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/pulls$ grc --base-url https://api.ecorp.com/api/v3 \--owner evilcorp \--repo ecoin \--resource-type pr--dest './export.json'
3.4.3. All (open and closed issues and pull requests)
-
CSV
GitHub:
# GET https://api.github.com/repos/:owner/:repo/pulls$ github-resource-converter \--owner gregswindle \--repo github-resource-converter \--resource-type all -
GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/pulls$ grc --base-url https://api.ecorp.com/api/v3 \--owner evilcorp \--repo ecoin \--resource-type all -
JSON
GitHub:
# GET https://api.github.com/repos/:owner/:repo/pulls$ github-resource-converter \--owner gregswindle \--repo github-resource-converter \--resource-type all--dest './export.json' -
GitHub Enterprise:
# GET https://api.ecorp.com/api/v3/repos/:owner/:repo/pulls$ grc --base-url https://api.ecorp.com/api/v3 \--owner evilcorp \--repo ecoin \--resource-type all--dest './export.json'
3.5. Command-line flags
--owner, -o
Required. The GitHub account name or organization name.
--repo, -r
Required. The name of the GitHub (or GitHub Enterprise) repository.
--base-url
The GitHub REST API v3 URL origin, or a GitHub Enterprise URL origin and path-prefix.
Default value: https://api.github.com
--dest, -d
The destination path and file name of the CSV.
Default value: ./export.csv --no-auto-filename
Disable automatic file naming.
Default value: false --resource-type, -t
Declares whether to convert and export Issues, Pull Requests, or All.
Default value: issues Valid values: all, issue, issues, pr, prs, pull_request, pull_requests
3.6. Errors
Errors are written to the console (stdout
) as JSON:
# Attempt to fetch issues from a repository that doesn't exist $ grc --owner example --repo error[2018-03-20T02:31:24.737Z] ERROR: github-resource-converter/48219 on User.router.home: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/issues/#list-issues-for-a-repository"} HttpError: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/issues/#list-issues-for-a-repository"} at response.text.then.message at <anonymous> at process._tickCallback
3.7. Info
The --help
flag displays all options:
$ grc --help Convert and export GitHub resources--Issues and Pull Requests--to CSV and JSON. Usage $ grc [options] [info] $ github-resource-converter [options] [info] Options --base-url The GitHub REST API v3 URL origin, or a GitHub Enterprise URL origin and path-prefix. [Default: 'https://api.github.com'] --dest, -d The CSV's destination path and file name. [Default: './resources.csv'] --no-auto-filename Disable automatic file naming. [Default: false] --owner, -o The GitHub account name or organization name. --repo, -r The name of the GitHub (or GitHub enterprise) repository. --resource-type, -t "issues", "prs", or "all". [Default: 'issues'] Info --help Show this dialog. --version Display the installed semantic version. Examples $ grc --owner github --repo hub // => Exported CSV to /path/of/cwd/issues.csv. $ grc --owner github --repo hub -dest './reports/issues/YYYY-MM-DD.csv' // => Exported CSV to /path/to/reports/issues/YYYY-MM-DD.csv. $ grc --owner example --repo error // => [2018-03-19T08:04:06.596Z] ERROR: github-resource-converter/00000 on localhost: Cannot destructure property `data` of 'undefined' or 'null'. TypeError: Cannot destructure property `data` of 'undefined' or 'null'. at paginate (/p/a/t/h/github-resource-converter/lib/base-resource-converter.js:39:16) at <anonymous> at process._tickCallback (internal/process/next_tick.js:188:7
Use the --version
flag to see which version you have installed:
$ github-resource-converter --version# => 1.0.1
4. API
Test the
github-resource-converter (grc)
API in your Web browser .
grc.authenticate({token, type, key}): void
4.1. Most GitHub API calls don't require authentication. Rules of thumb:
- If you can see the information by visiting the site without being logged in, you don't have to be authenticated to retrieve the same information through the API.
- If you want to change data, you have to be authenticated.
octokit/rest.js. (2018). GitHub. Retrieved 21 March 2018, from https://github.com/octokit/rest.js#authentication
4.1.1. Parameters
Name | Type | Description | Notes |
---|---|---|---|
key | String | ||
token | String | ||
type | Enum | basic , oauth , oauth-key-secret , token , and integration |
4.1.2. Example
// Token (https://github.com/settings/tokens)grc
grc.getAll({owner, repo}): Promise
4.2. Retrieve all open and closed Issues and Pull Requests from a GitHub project.
/repos/:owner/:repo/issues/repos/:owner/:repo/pulls
grc.getAll
combines the results ofgrc.issues.getForRepo
andgrc.pullRequests.getForRepo
into a singleArray<object>
.For more information, see
grc.issues.getForRepo
andgrc.pullRequests.getForRepo
below.
grc.issues.getForRepo({owner, repo}): Promise
4.3. Retrieve all open and closed issues for a repository with this proxy method for octokit.issues.getForRepo
.
/repos/:owner/:repo/issues
4.3.1. Parameters
Field | Type | Description |
---|---|---|
owner | string | |
repo | string | |
milestone | string | |
state | string |
open, closed, or all Default value: Allowed values:
|
assignee | string |
String User login, |
creator | string |
The user that created the issue. |
per_page | number |
A custom page size up to 100. Default is 30. Default value: |
labels | string |
String list of comma separated Label names. Example: bug,ui,@high |
sort | string |
Default value: Allowed values:
|
direction | string |
Default value: Allowed values:
|
since | date |
Timestamp in ISO 8601 format: YYYY-MM-DDTHH:MM:SSZ |
page | number |
Page number of the results to fetch. |
mentioned | string |
String User login. |
4.3.2. Examples
-
async/await:
const result = await grcissues -
Promise:
grcissues
grc.logger
4.4. A trentm/node-bunyan
logger instance that uses the LONG
thlorenz/bunyan-format
writeable
stream for output.
bunyan.INFO
is the default log LEVEL.
Logger domain: null _events: {} _eventsCount: 0 _maxListeners: undefined _level: 30 streams: type: 'stream' stream: BunyanFormatWritable closeOnExit: false level: 30 raw: false serializers: null src: false fields: name: 'github-resource-converter' hostname: 'Gregorys-MBP-2.fios-router.home' pid: 76698 : void : void : void : void : void : void
grc.options
4.5. Contains default values for api
, cli
, and meta
data.
Toggle view of grc.options.
"api": "baseUrl": "https://api.github.com" "headers": "Accept": "application/vnd.github.v3+json, application/vnd.github.symmetra-preview+json" "user-agent": "gregswindle/github-resource-converter v1.0.1" "owner": null "repo": null "requestMedia": "application/vnd.github.v3+json" "timeout": 0 "cli": "flags": "baseUrl": "default": "https://api.github.com" "type": "string" "dest": "alias": "d" "default": "./export.csv" "type": "string" "no-auto-filename": "default": false "type": "boolean" "owner": "alias": "o" "type": "string" "repo": "alias": "r" "type": "string" "resource-type": "alias": "t" "default": "issues" "type": "string" "meta": "input": "flags": "noAutoFilename": false "baseUrl": "https://api.github.com" "dest": "./export.csv" "d": "./export.csv" "resourceType": "issues" "t": "issues" "pkg": "name": "github-resource-converter" "description": "Convert and export GitHub resources--Issues and Pull Requests--to CSV and JSON." "version": "1.0.1" "author": "name": "Greg Swindle" "email": "greg@swindle.net" "url": "https://github.com/gregswindle" "bin": "grc": "lib/cli.js" "bugs": "url": "https://github.com/gregswindle/github-resource-converter/issues" "commitplease": "style": "angular" "types": "build" "chore" "ci" "docs" "feat" "fix" "perf" "refactor" "revert" "style" "test" "scope": "\\S+.*" "contributors": "dependencies": "@octokit/rest": "15.2.6" "bunyan": "1.8.12" "bunyan-format": "0.2.1" "dotenv": "5.0.1" "fs-extra": "5.0.0" "insight": "0.10.1" "jsonexport": "2.0.11" "lodash": "4.17.5" "meow": "4.0.0" "devDependencies": "@semantic-release/changelog": "2.0.1" "@semantic-release/git": "4.0.1" "@semantic-release/npm": "3.2.4" "ajv": "6.4.0" "ajv-keywords": "3.1.0" "codacy-coverage": "2.1.1" "commitplease": "3.2.0" "coveralls": "3.0.0" "eslint": "4.19.1" "eslint-config-prettier": "^2.4.0" "eslint-config-scanjs": "1.0.0-beta4" "eslint-config-standard": "11.0.0" "eslint-config-xo": "0.20.1" "eslint-plugin-import": "2.10.0" "eslint-plugin-jsdoc": "3.5.0" "eslint-plugin-json": "1.2.0" "eslint-plugin-no-unsafe-innerhtml": "1.0.16" "eslint-plugin-node": "6.0.1" "eslint-plugin-prettier": "^2.2.0" "eslint-plugin-promise": "3.7.0" "eslint-plugin-security": "1.4.0" "eslint-plugin-standard": "3.0.1" "eslint-plugin-unicorn": "4.0.2" "eslint-plugin-xss": "0.1.9" "fixpack": "2.3.1" "husky": "^0.14.3" "jest": "22.4.3" "jest-runner-eslint": "0.4.0" "lec": "^1.0.1" "lint-staged": "7.0.3" "markdown-magic": "0.1.21" "markdown-magic-dependency-table": "1.3.2" "markdown-magic-install-command": "1.3.1" "markdown-magic-package-scripts": "1.2.1" "nsp": "^3.2.1" "prettier": "1.11.1" "semantic-release": "15.1.5" "engines": "node": ">=8.0.0" "eslintIgnore": "lib/__tests__/coverage/**" "files": "lib" "homepage": "https://github.com/gregswindle/github-resource-converter/#readme" "jest": "automock": false "collectCoverage": true "coverageDirectory": "lib/__tests__/coverage" "coverageThreshold": "global": "branches": 100 "functions": 100 "lines": 100 "statements": 100 "moduleDirectories": "node_modules" "lib" "testMatch": "<rootDir>/lib/__tests__/**/*.test.js" "<rootDir>/lib/__tests__/*.test.js" "watchman": false "jest-runner-eslint": "cliOptions": "config": ".github/config/jest-runner-eslint.config.json" "fix": true "keywords": "QA" "convert" "converter" "converter" "csv" "export" "github" "google sheets" "issue" "json" "pr" "pull request" "pull-request" "quality assurance" "quality-assurance" "reporting" "reports" "save" "sheets" "spreadsheets" "testing" "license": "MIT" "lint-staged": "*.js": "npm run lint:js" "git add" "*.json": "npm run lint:json" "git add" "*.md": "npm run lint:md" "npm run docs" "git add" "main": "lib/index.js" "prettier": "semi": false "singleQuote": true "release": "generateNotes": "preset": "angular" "prepare": "@semantic-release/changelog" "@semantic-release/npm" "@semantic-release/git" "verifyConditions": "@semantic-release/npm" "@semantic-release/git" "repository": "type": "git" "url": "git+https://github.com/gregswindle/github-resource-converter.git" "scripts": "docs": "npm run docs:toc" "docs:toc": "md-magic --config '.github/config/markdown.config.js' --path '**/*.md' --ignore 'node_modules'" "lint": "npm run lint:js && npm run lint:json && npm run lint:manifest && npm run lint:md" "lint:js": "node_modules/.bin/eslint -c .github/config/.eslintrc.yml --ext .js . --fix" "lint:json": "prettier ./**/*.json --ignore-path '.github/config/.prettierignore' --write" "lint:manifest": "fixpack" "lint:md": "prettier ./**/*.md -prose-wrap always --write" "posttest:ci:coverage:codacy": "cat ./lib/__tests__/coverage/lcov.info | codacy-coverage" "precommit": "lint-staged" "prepare": "npm run lint && npm run security" "prepublish": "lec lib/cli.js -c LF && npm run security && semantic-release -d" "prepublishOnly": "npm run prepare" "pretest": "npm run lint" "security": "npm run security:nsp:scan" "security:nsp:scan": "nsp check" "test": "jest" "test:config": "jest --showConfig" "test:watch": "jest ./lib/__tests__/*.test.js --watch" "test:watch:all": "jest ./lib/__tests__/*.test.js --watchAll" "readme": "ERROR: No README data found!" "_id": "github-resource-converter@1.0.1" "help": "\n Convert and export GitHub resources--Issues and Pull Requests--to CSV and JSON.\n\n Usage\n\n $ grc [options] [info]\n $ github-resource-converter [options] [info]\n\n Options\n --base-url The GitHub REST API v3 URL origin, or a\n GitHub Enterprise URL origin and path-prefix.\n [Default: 'https://api.github.com']\n --dest, -d The CSV's destination path and file name.\n [Default: './resources.csv']\n --no-auto-filename Don't append an ISO 8601-like timestamp to the\n output file.\n [Default: false]\n --owner, -o The GitHub account name or organization name.\n --repo, -r The name of the GitHub (or GitHub enterprise)\n repository.\n --resource-type, -t \"issues\", \"prs\", or \"all\".\n [Default: 'issues']\n\n Info\n\n --help Show this dialog.\n --version Display the installed semantic version.\n\n Examples\n\n $ grc --owner github --repo hub\n // => Exported CSV to /path/of/cwd/issues.csv.\n\n $ grc --owner github --repo hub -dest './reports/issues/YYYY-MM-DD.csv'\n // => Exported CSV to /path/to/reports/issues/YYYY-MM-DD.csv.\n\n $ grc --owner example --repo error\n // =>\n [2018-03-19T08:04:06.596Z] ERROR: github-resource-converter/00000 on localhost: Cannot destructure property `data` of 'undefined' or 'null'.\n TypeError: Cannot destructure property `data` of 'undefined' or 'null'.\n at paginate (/p/a/t/h/github-resource-converter/lib/base-resource-converter.js:39:16)\n at <anonymous>\n at process._tickCallback (internal/process/next_tick.js:188:7)\n"
grc.pullRequests.getForRepo({owner, repo}): Promise
4.6. Retrieve an array of all open and closed pull requests for a GitHub or GitHub Enterprise repository with this proxy method for
octokit.pullRequests.getAll
.
/repos/:owner/:repo/pulls
4.6.1. Parameters
Field | Type | Description |
---|---|---|
owner | string | |
repo | string | |
state | string |
Default value:
Allowed values:
|
head | string |
Filter pulls by head user and branch name in the format of user:ref-name. Example: github:new-script-format. |
base | string |
Filter pulls by base branch name. Example: gh-pages. |
sort | string |
Possible values are:
Default value:
Allowed values:
|
direction | string |
Default value:
Allowed values:
|
page | number |
Page number of the results to fetch. |
per_page | number |
A custom page size up to 100. Default is 30. Default value:
|
4.6.2. Examples
-
async/await:
const grc =const getAllPullRequests = async params =tryconst WHITESPACE = 2const prs = await grcpullRequestsgrcloggercatch errgrcloggerawait -
Promises:
const grc =grcpullRequests
grc.save({data, dest}): Promise
4.7. Export a collection of Issues or Pull Requests to your local filesystem.
4.7.1. Parameters
Field | Type | Description |
---|---|---|
data | JSON | object | |
dest | string |
Default value:
Allowed values:
|
4.7.2. Examples
-
async/await:
// Save as JSONconst result = await grc -
Promise:
// Save as CSVgrc
grc.toCsv({data=[]}): Promise
4.8. Converts (deeply) nested JSON into CSV format, returning a Promise<string>
.
4.8.1. Parameters
Field | Type | Description |
---|---|---|
data | JSON | object |
4.8.2. Examples
-
async/await:
const result = await grc -
Promise:
grc
5. Version
6. Contributing
We welcome contributions with GitHub issues and pull requests.
Before embarking on a significant change, please adhere to the following guidelines:
-
Create an issue—e.g., a defect ("bug") report or a feature request—to propose changes.
Exceptions:
If you're working on documentation and fixing something simple like a typo or an easy bug, go ahead and make a pull request.
-
Follow the CONTRIBUTING guidelines.
Why:
Standards and guidelines make communication easier. If you're willing and able to program—or want to learn how— following the guidelines will increase the likelihood of adding your changes to the software product.
-
Why:
It's more fun when everybody's friendly and respectful.
-
Make a pull request when you're ready for other to review your changes (or you get stuck somewhere).
PR novices:
🙋 Never created a pull request? No problem. 🆓 Take this free online training . (It even covers most of the conventions in the CONTRIBUTING guidelines!)
7. License
MIT © Greg Swindle.
Read the NOTICE for all third-party software that github-resource-converter
uses.