Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FIX: Allow any iterable for ShellOutSpec requires #631

Merged
merged 2 commits into from
Mar 23, 2023
Merged

FIX: Allow any iterable for ShellOutSpec requires #631

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
51a0e3f
FIX: Allow any iterable for ShellOutSpec requires
ghisvail Mar 21, 2023
d061b35
REF: Use ensure_list instead of list for the str case
ghisvail Mar 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions pydra/engine/specs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -636,11 +636,12 @@ def _check_requires(self, fld, inputs):

if "requires" in fld.metadata:
# if requires is a list of list it is treated as el[0] OR el[1] OR...
if all([isinstance(el, list) for el in fld.metadata["requires"]]):
field_required_OR = fld.metadata["requires"]
required_fields = list(fld.metadata["requires"])
ghisvail marked this conversation as resolved.
Show resolved Hide resolved
if all([isinstance(el, list) for el in required_fields]):
field_required_OR = required_fields
# if requires is a list of tuples/strings - I'm creating a 1-el nested list
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little confused by this one. The type checking is divorced from the semantics. Why should this be a list of lists of tuples or strings? It's hard to reason about the effects of converting an arbitrary iterable to a list when the meaning of the tuples and strings (both iterables) are opaque.

Instead of having this here in this function, I'd rather have a separate function:

def canonicalize_requires(requires):
    """Canonicalize a requirement to a nested list of lists of requirements

    Fully specified requirements are a list of lists of strings or tuples that have the form
    ``[["abc", "def"], ...]`` or ``[[("abc", "def"), ("ghi", "jkl")], ...]``. For brevity, we permit the following
    forms.

    >>> # some example code

    Non-list iterables are also accepted:

    >>> canonicalize_requirements({"abc", "def"})
    [["abc", "def"]]
    """
    ...

Copy link
Collaborator Author

@ghisvail ghisvail Mar 23, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not want to change the semantics too much in this PR so I did not attempt to refactor the code. The code flow could be improved.

The idea remains that I should be able to do:

metadata={
    ...,
    "requires": {"foo", "bar"},
}

or

metadata={
    ...,
    "requires": ["foo", "bar"],
}

and the most straightfoward way was to normalize Iterable to list internally first.

elif all([isinstance(el, (str, tuple)) for el in fld.metadata["requires"]]):
field_required_OR = [fld.metadata["requires"]]
elif all([isinstance(el, (str, tuple)) for el in required_fields]):
field_required_OR = [required_fields]
else:
raise Exception(
f"requires field can be a list of list, or a list "
Expand Down