Progress Logs

If the user opts to use the progress reporting functions already described then progress logs will be created as files. The progress logs are created in the remote job directory. They can be downloaded along with the job's results from HPC Workflow Manager context menu of the main window.

The progress logs have the *.plog extension. Each log file of a job corresponds to a node of the cluster where the job is running on. For example, progress_0.plog, progress_1.plog would be the progress logs for two nodes, counting from zero.

There are two log formats available:

• XML files, and
• Comma-separated values (CSV) files.

You can choose the format of the log by calling the parSelectProgressLogger(type) where type="xml" or type="csv" function from the macro language. The XML format is used by default if this function is not called.

Examples

The following is an example of a progress log from a single node in each one of the two available formats.

There are four tasks in this example, all have finished. Each task of a node must have a unique description. Different nodes may have different tasks. If a task description exists in more than one node it will be displayed on the same line in HPC Workflow Manager's dashboard's progress tab.

Also listed are the total number of nodes and the timestamp when the progress log was last updated (in UNIX Epoch time format in milliseconds). It is used by HPC Workflow Manager not to parse an old progress log file again while a job is running.

XML

<?xml version="1.0" encoding="UTF-8"?>
<job>
	<nodes>2</nodes>
	<task id="0">
		<description>Message overlay generation.</description>
		<progress>100</progress>
	</task>
	<task id="1">
		<description>Flatten message overlays.</description>
		<progress>100</progress>
	</task>
	<task id="2">
		<description>Add noise multiple times.</description>
		<progress>100</progress>
	</task>
	<task id="3">
		<description>Draw random lines.</description>
		<progress>100</progress>
	</task>
	<lastUpdated>1643900602419</lastUpdated>
</job>

CSV

2
1644242435125
0,Message overlay generation.
1,Flatten message overlays.
2,Add noise multiple times.
3,Draw random lines.
0,10
0,20
0,30
0,40
0,50
0,60
0,70
0,80
0,90
0,100
1,0
1,100
2,0
2,10
2,20
2,30
2,40
2,50
2,60
2,70
2,80
2,90
2,100
3,0
3,1
3,2
3,3
3,4
3,5
3,6
3,7
3,8
3,9
3,10
3,11
3,12
3,13
3,14
3,15
3,16
3,17
3,18
3,19
3,20
3,21
3,22
3,23
3,24
3,25
3,26
3,27
3,28
3,29
3,30
3,31
3,32
3,33
3,34
3,35
3,36
3,37
3,38
3,39
3,40
3,41
3,42
3,43
3,44
3,45
3,46
3,47
3,48
3,49
3,50
3,51
3,52
3,53
3,54
3,55
3,56
3,57
3,58
3,59
3,60
3,61
3,62
3,63
3,64
3,65
3,66
3,67
3,68
3,69
3,70
3,71
3,72
3,73
3,74
3,75
3,76
3,77
3,78
3,79
3,80
3,81
3,82
3,83
3,84
3,85
3,86
3,87
3,88
3,89
3,90
3,91
3,92
3,93
3,94
3,95
3,96
3,97
3,98
3,99
3,100

main hub:house: