Skip to content

Commit

Permalink
Add more cases for std stream logging (#3347)
Browse files Browse the repository at this point in the history
Previously, stdout/err logs not output if the streams were not set
(i.e. defaulting to None) and logged with the entire stream target
if set.

This PR adds more cases here with case specific logging: if stdout/err
is not redirected, that is now logged. If tuple mode is used, the
tuple is unpacked and logged as separate fields. If an unknown format
is specified an error is logged (but no exception is raised - it is
not the job of this code to validate the stdout/err specifications)

This PR is in preparation for a new case to be added in an upcoming
PR which requires more serious formatting that was not accomodated
by the previous implementation.
  • Loading branch information
benclifford authored Apr 11, 2024
1 parent 1140cff commit 906d544
Showing 1 changed file with 14 additions and 4 deletions.
18 changes: 14 additions & 4 deletions parsl/dataflow/dflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -1391,10 +1391,20 @@ def load_checkpoints(self, checkpointDirs: Optional[Sequence[str]]) -> Dict[str,

@staticmethod
def _log_std_streams(task_record: TaskRecord) -> None:
if task_record['app_fu'].stdout is not None:
logger.info("Standard output for task {} available at {}".format(task_record['id'], task_record['app_fu'].stdout))
if task_record['app_fu'].stderr is not None:
logger.info("Standard error for task {} available at {}".format(task_record['id'], task_record['app_fu'].stderr))
tid = task_record['id']

def log_std_stream(name: str, target) -> None:
if target is None:
logger.info(f"{name} for task {tid} will not be redirected.")
elif isinstance(target, str):
logger.info(f"{name} for task {tid} will be redirected to {target}")
elif isinstance(target, tuple) and len(target) == 2:
logger.info(f"{name} for task {tid} will be redirected to {target[0]} with mode {target[1]}")
else:
logger.error(f"{name} for task {tid} has unknown specification: {target!r}")

log_std_stream("Standard out", task_record['app_fu'].stdout)
log_std_stream("Standard error", task_record['app_fu'].stderr)


class DataFlowKernelLoader:
Expand Down

0 comments on commit 906d544

Please sign in to comment.