Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

typing updates for Task #194

Merged
merged 2 commits into from
Jul 3, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 20 additions & 20 deletions sisyphus/task.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,35 +3,35 @@
import logging
import sys
import time
import typing
from typing import Optional, Union, Any, Sequence, Dict, List
import subprocess as sp
from ast import literal_eval

import sisyphus.tools as tools
import sisyphus.global_settings as gs


class Task(object):
class Task:
"""
Object to hold information what function should be run with which requirements.
"""

def __init__(
self,
start,
resume=None,
rqmt=None,
args=None,
mini_task=False,
start: str,
resume: Optional[str] = None,
rqmt: Optional[Dict[str, Any]] = None,
args: Optional[Sequence[Union[List[Any], Any]]] = None,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
args: Optional[Sequence[Union[List[Any], Any]]] = None,
args: Optional[Sequence[Any]] = None,

can already be Any and List[Any] is contained in Any.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea I know, but sometimes I prefer this to give some more specific type hints to what types would be specifically expected here, or specially handled, even though other types (any other types) would also be ok.

Also, I basically just translated the typing from the earlier docstring. I also wondered a bit about why this is here at all, and I thought there is indeed maybe a special meaning when there is a list?

mini_task: bool = False,
update_rqmt=None,
parallel=0,
tries=1,
continuable=False,
parallel: int = 0,
tries: int = 1,
continuable: bool = False,
):
"""
:param str start: name of the function which will be executed on start
:param str resume: name of the function which will be executed on resume, often set equal to start
:param dict[str] rqmt: job requirements
:param start: name of the function which will be executed on start
:param resume: name of the function which will be executed on resume, often set equal to start
:param rqmt: job requirements
Might contain:
"cpu": number of cpus
"gpu": number of gpus
Expand All @@ -40,14 +40,14 @@ def __init__(
"multi_node_slots": amount of slots, distributed potentially over multiple nodes.
E.g. maps to `--ntasks <multi_node_slots>` parameter for Slurm,
and to `-pe <pe_name> <multi_node_slots>` for SGE (SGE parallel environment (PE)).
:param typing.Sequence[typing.Union[typing.List[object],object]] args: job arguments
:param bool mini_task: will be run on engine for short jobs if True
:param args: job arguments
:param mini_task: will be run on engine for short jobs if True
:param (dict[str],dict[str])->dict[str] update_rqmt: function to update job requirements for interrupted jobs
:param int parallel: if set to > 0, groups jobs for individual arguments together into the number of batches
specified here. Will then submit at max `parallel` jobs into the engine at a time.
:param int tries: how often this task is resubmitted after failure
:param bool continuable: If set to True this task will not set a finished marker, useful for tasks that can be
continued for arbitrarily long, e.g. adding more epochs to neural network training
:param parallel: if set to > 0, groups jobs for individual arguments together into the number of batches
specified here. Will then submit at max `parallel` jobs into the engine at a time.
:param tries: how often this task is resubmitted after failure
:param continuable: If set to True this task will not set a finished marker, useful for tasks that can be
continued for arbitrarily long, e.g. adding more epochs to neural network training
"""
if rqmt is None:
rqmt = {}
Expand Down
Loading