Note
Name is an optional parameter in all below functions.
Description | Functions and examples |
---|---|
Create InputTypes | input_type_1 = InputTypes("Types1", [int, str])
|
Create InputValue | input_value_1 = InputValues("Values1", [1, "hello"])
|
Create InputArray | input_array_12 = InputArray("Array12", [input_value_1, input_value_2])
|
Description | Functions and examples |
---|---|
Create a Task | task_f1 = Task("A", FUNCTION, "myfunc", input_type_1)
task_x1 = Task("X", EXECUTABLE, "./bin/x", input_type_1)
|
Create a TaskArray | task_array_xyz = TaskArray("xyz", [task_f1, task_x1])
task_array_12 = TaskArray("tasks12", [task_f1, task_f2])
|
Description | Functions and examples |
---|---|
Sequence | output = sequence("my_sequence", task_array_12, input_array_12)
|
Data parallel | output = parallel("abc", task_array_12, input_array_12)
|
Split | output = split("split", task_x1, input_value_1, spl_taskarray, spl_input_array)
|
Merge | output = merge("sync", syn_ta, syn_ia, task_x1, input_value_1)
|
Constant | Description |
---|---|
FUNCTION | Function in current program |
EXECUTABLE | Different program to execute |
PREVIOUS.task_name.i[output_no] | Output number of previous task task_name |
Below is a skeleton to help kickstart writing your Tigres code. This will allow you to specify different Execution plugins. You can download the Tigres skeleton program here
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | """
My Tigres Program
"""
import os
import sys
from tigres import *
from tigres.utils import Execution, TigresException, State
def main(execution):
# Setup up the Tigres Program using __file__ to build the log file name
start(log_dest=os.path.splitext(__file__)[0] + '.log', execution=execution)
# Set the logging level
set_log_level(Level.INFO)
# Heart of Tigres Program here
# Create DOT (plain text graph) file
dot_execution()
# End the Tigres Program
end()
if __name__ == "__main__":
# Simple Usage Here
if len(sys.argv) <= 1:
print("Usage: {} ({})>".format(sys.argv[0], "|".join(Execution.LOOKUP.keys())))
exit()
try:
main(Execution.get(sys.argv[1]))
except TigresException as e:
print(e)
task_check = check('task', state=State.FAIL)
for task_record in task_check:
print(".. State of {} = {} - {}".format(task_record.name, task_record.state, task_record.errmsg))
|
This program allows you to execute your workflow with any of the available execution plugins. If you execute this python code, you will see all the available execution plugins:
$ python hello_world_parallel.py
Usage: hello_world_parallel.py (EXECUTION_SLURM|EXECUTION_DISTRIBUTE_PROCESS|EXECUTION_LOCAL_THREAD|EXECUTION_LOCAL_PROCESS|EXECUTION_SGE)>
Execute the script with EXECUTION_LOCAL_THREAD:
$ hello_world_parallel.py EXECUTION_LOCAL_THREAD
There is no output to the screen but you will see a log file in the current directory:
$ ls
hello_world_parallel.log hello_world_parallel.py
In lines 7-8 the Tigres API is loaded into the python program namespace. Line 7 imports all functions and classes from tigres while line 8 imports the tigres.utils.Execution, tigres.utils.TigresException and tigres.utils.State.
from tigres import *
from tigres.utils import Execution, TigresException, State
The main function has some boiler plate code that is run before and after any templates are executed.
def main(execution):
# Setup up the Tigres Program using __file__ to build the log file name
start(log_dest=os.path.splitext(__file__)[0] + '.log', execution=execution)
# Set the logging level
set_log_level(Level.INFO)
# Heart of Tigres Program here
# Create DOT (plain text graph) file
dot_execution()
# End the Tigres Program
end()
Before any templates are executed, the Tigres program is initialized with a call to start(). This function configures and initializes the Tigres program. In the above code, the program specifies the log file name (log_dest) as well as the execution mechanism (e.g. execution='EXECUTION_LOCAL_THREADS'). The function start() will end a previously started Tigres program. The verbosity of the logs is set using set_log_level(). After this call, all messages at a numerically higher level (e.g. DEBUG if level is INFO) will be skipped (See Level).
After the last template is executed, the execution can be visualized using dot_execution(). This function creates a DOT file (plain text graph) for a visual representation of the completed workflow. The function dot_execution() writes a graph file of the current Tigres execution workflow that can be visualized using the visualization tools of your choice such as GraphViz. Lastly, the Tigres program is finalized with a call to end().
The driver program at the bottom of the file, does some argument checking, executes main() and prints any raised errors. If a tigres.utils.TigresException is raised, the message is printed and then the monitoring information is checked for more detailed error messages using tigres.check(). Line 26, searches the monitoring information for the currently running program for all Tigres tasks that have a state of State.FAIL.
if __name__ == "__main__":
# Simple Usage Here
if len(sys.argv) <= 1:
print("Usage: {} ({})>".format(sys.argv[0], "|".join(Execution.LOOKUP.keys())))
exit()
try:
main(Execution.get(sys.argv[1]))
except TigresException as e:
print(e)
task_check = check('task', state=State.FAIL)
for task_record in task_check:
print(".. State of {} = {} - {}".format(task_record.name, task_record.state, task_record.errmsg))