Add node resource overrides #523
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
EngHabu
previously approved these changes
Jun 25, 2021
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
Codecov Report
@@ Coverage Diff @@
## master #523 +/- ##
=======================================
Coverage 85.37% 85.37%
=======================================
Files 369 369
Lines 28199 28276 +77
Branches 2281 2290 +9
=======================================
+ Hits 24074 24140 +66
- Misses 3505 3510 +5
- Partials 620 626 +6
Continue to review full report at Codecov.
|
EngHabu
previously approved these changes
Jun 25, 2021
flytekit/models/task.py
Outdated
| @@ -100,8 +100,8 @@ def to_flyte_idl(self): | |||
| :rtype: flyteidl.core.tasks_pb2.Resources | |||
| """ | |||
| return _core_task.Resources( | |||
| requests=[r.to_flyte_idl() for r in self.requests], | |||
| limits=[r.to_flyte_idl() for r in self.limits], | |||
| requests=[r.to_flyte_idl() for r in self.requests] if self.requests is not None else None, | |||
There was a problem hiding this comment.
we should revert this - locally at least, unit tests passed for me. #525
There was a problem hiding this comment.
but now that we can initialize the model with optionally empty requests or limits, we can't, right?
they fail for me
FAILED tests/flytekit/unit/models/core/test_workflow.py::test_workflow_template - TypeError: 'NoneType' object is not iterable
FAILED tests/flytekit/unit/models/core/test_workflow.py::test_task_node - TypeError: 'NoneType' object is not iterable
FAILED tests/flytekit/unit/models/core/test_workflow.py::test_node_task_with_no_inputs - TypeError: 'NoneType' object is not iterable
FAILED tests/flytekit/unit/models/core/test_workflow.py::test_node_task_with_inputs - TypeError: 'NoneType' object is not iterable
FAILED tests/flytekit/unit/models/core/test_workflow.py::test_branch_node - TypeError: 'NoneType' object is not iterable
There was a problem hiding this comment.
then why is this passing? #525
| @@ -272,7 +273,9 @@ def get_serializable_node( | |||
| inputs=entity.bindings, | |||
| upstream_node_ids=[n.id for n in upstream_sdk_nodes], | |||
| output_aliases=[], | |||
| task_node=workflow_model.TaskNode(reference_id=task_spec.template.id), | |||
| task_node=workflow_model.TaskNode( | |||
| reference_id=task_spec.template.id, overrides=TaskNodeOverrides(resources=entity._resources) | |||
There was a problem hiding this comment.
These aren't the Resource models though right? They're the internal dataclass. Don't they need to be translated?
There was a problem hiding this comment.
There was a problem hiding this comment.
oh i see. can we update the type hint on line 36 of node.py then?
There was a problem hiding this comment.
ah yes, thanks for the catch
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
flytekit/models/task.py
Outdated
| def __init__(self, requests, limits): | ||
| def __init__( | ||
| self, | ||
| requests: typing.Optional[typing.List[ResourceEntry]] = None, |
Signed-off-by: Katrina Rogan <[email protected]>
wild-endeavor
approved these changes
Jun 25, 2021
wild-endeavor
pushed a commit
that referenced
this pull request
Jun 30, 2021
Signed-off-by: wild-endeavor <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Katrina Rogan [email protected]
TL;DR
Add node resource overrides
Type
Are all requirements met?
Complete description
How did you fix the bug, make the feature etc. Link to any design docs etc
Tracking Issue
flyteorg/flyte#475
flyteorg/flyte#1170
Follow-up issue
NA