This is a light utility to analyze history of failing tests from an Azure DevOps build.

• install node so that npx works in your shell
• install the Azure CLI and run az login in your shell

npx ado-test-history --org <org> --project <project> --buildId <buildId>

This utility was designed against a particular ADO build/repo so the defaults may not generalize for others. In order of usefulness:

• --buildDefinitionBaselineID 123 allows you to run this utility against something like a PR build. The visualization shows historical test results from definition 123 (ideally something stable like CI)
• --buildReferenceNumberFilter X The ADO API filters records to a particular build definition, but includes all branches. Setting X here limits records to builds whose name includes X (master by default)
• --verbose will create JSON files in a log directory. This should work for most customization since you can parse the files yourself.

The utility prints commits associated with builds that failed once after two consecutive passed builds.

If buildDefinitionBaselineID is set, we suggest commits regardless, even if the latest baseline test is passing.

Otherwise, we only suggest commits if the latest baseline test is failing.