cloudd
node.js cloud engine Apart from all the buzz words, key aspect of this software is to specify set of tasks that are interdependent in a config file (similar to DAG input on condor) and run them.
Basics
- Task - is the smallest unit of execution (preferrable should be an idempotent task)
- Job - collection of tasks with dependencies (acyclic)
Features
- job execution engine
- Task could be a javascript unit or executable that would be exec'ed
- Command line to check the status of jobs, tasks and current running processes
- Command line Submit of jobs
- instantiate jobs cron style
- Good unit test coverage
Upcoming
- Failure of tasks to be handled
- Job load balancing among cloudd hosts
- Persistence of state
- clustering among cloudd hosts
Example
- Lets see how to define a Hello My World Job
- Job
Hello My World
comprise of three tasks
Hello
TaskMy
TaskWorld
Task
- Dependencies
My
would be run only afterHello
World
would be run only afterMy
- Job
- Sample yaml config file would look like (see in examples/hellomyworld.yaml)
name: Hello Worlddescription: Sample 'hello my world' jobjobs: my: executable: echo "my" hello: executable: echo "hello" world: executable: echo "world"dependencies: - my-world: parent: [my] child: [world] - hello-my: parent: [hello] child: [my]
To try this example
Have two terminals, one for the server and another one for submitting the job
# terminal 1
cloudd server
# terminal 2
cloudd submit HW examples/hellomyworld.yaml
Usage
To run the server
cloudd server
Command line utilities
# to show current running tasks
cloudd ps
# to show total tasks that are ready to run (ordered list)
cloudd tasks
# to show total jobs in the system (non-completed)
cloudd jobs
# to submit a new job
cloudd submit job-id sample-job-file.yaml
# to submit a cron job (say every 5 seconds)
cloudd cron "*/5 * * * * *" job-id sample-job-file.yaml
Config file
- id's for the job and task cannot have '#'
- input is a field that would be passed to task