HyperFlow Tools
A collection of small useful tools for the HyperFlow workflow engine
-
hflow-info
: print various information about a workflow and annotate workflow json file. -
hflow-dot
: convert HyperFlow workflow graph to Graphviz dot format. -
hflow-metis
: convert HyperFlow workflow graph to Metis format (graph partitioning tool). -
hflow-patoh
: convert HyperFlow workflow graph to Patoh format (graph partitioning tool). -
hflow-convert-makeflow
: convert Makeflow mf/json workflow to HyperFlow json.
Requirements
- Node v12.x
Installation
npm install -g https://github.com/hyperflow-wms/hflow-tools/archive/master.tar.gz
Usage
hflow-info
Usage:
hflow-info <workflow.json path> [--print] [-p <metis_partition_file>] [-f <file_dir>]
hflow-info -h|--help
Options:
-h --help Prints this
--print Print output workflow json (to stderr)
-p <partition_file> Add partitioning info based on partitioning file generated by metis
-f <file_dir> Add information about worklow files which are stored in <file_dir>
hflow-dot
hflow-dot: converts HyperFlow workflow.json to graphviz dot format
Usage:
hflow-dot [-p] [--png] <workflow-json-file-path>
hflow-dot -h|--help
Options:
-h --help Prints this
--png Generate png
-p Generates graph according to paritioning info
To generate image in another format, use dot
:
dot -Tpdf workflow.dot -o workflow.pdf
hflow-metis
hflow-metis: converts HyperFlow workflow.json to Metis graph format
Usage:
hflow-metis [--ew] [--nw] [--ns] [--sn] [--lw=<npart>] [--pwgts=<pwgts>] <workflow-json-file-path>
hflow-metis -h|--help
Options:
-h --help Prints this
--ew Add edge weights (not implemented, probably not needed)
--nw Add node weights (requested cpu)
--ns Add node size (communication volume)
--sn Add special storage node
--lw=<npart> Add level weights for 'npart' partitions
--pwgts=<pwgts> Partition size weights, e.g. '0.3,0.7'
Using Metis to generate a partitioning:
gpmetis -objtype vol workflow.metis 2
Where vol
is the recommended optimization objective (minimizes communication between partitions), and 2
is the number of partitions.
hflow-patoh
Usage:
hflow-patoh [--ew] [--nw] [--ns] [--sn] [--lw=<npart>] [--pwgts=<pwgts>] <workflow-json-file-path>
hflow-patoh -h|--help
Options:
-h --help Prints this
--ew Add edge weights (communication volume)
--nw Add node weights (requested cpu)
--sn Add special storage node
--lw=<npart> Add level weights for 'npart' partitions
--pwgts=<pwgts> Partition size weights, e.g. '0.3,0.7'
hflow-convert-makeflow
hflow-convert-makeflow: converts workflows in Makeflow mf/json format into HyperFlow json
Usage:
hflow-convert-makeflow <makeflow MF/JSON file>