Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for python pickle type in flytekit/flyte #667

Merged
merged 32 commits into from
Oct 27, 2021
Merged
Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fixed test
Signed-off-by: Kevin Su <pingsutw@apache.org>
  • Loading branch information
pingsutw committed Oct 26, 2021
commit 1dd1fd40537e984c1433c8e89bf417cab1641f45
1 change: 1 addition & 0 deletions .github/workflows/pythonbuild.yml
Original file line number Diff line number Diff line change
@@ -10,6 +10,7 @@ jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does this do?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we will not cancel all in-progress jobs if any unit test failed.
For example, Sometimes the test only failed with Python 3.9, but it will cancel the other tests with Python 3.7 or 3.8. It will be easy to debug If I know the test fail with which python version.

matrix:
python-version: [3.7, 3.8, 3.9]
spark-version-suffix: ["", "-spark2"]
1 change: 0 additions & 1 deletion flytekit/common/translator.py
Original file line number Diff line number Diff line change
@@ -123,7 +123,6 @@ def get_serializable_task(
)
if settings.should_fast_serialize() and isinstance(entity, PythonAutoContainerTask):
entity.reset_command_fn()
print(tt)
return task_models.TaskSpec(template=tt)


1 change: 0 additions & 1 deletion flytekit/core/promise.py
Original file line number Diff line number Diff line change
@@ -22,7 +22,6 @@
from flytekit.models.core import literals as _literals_models
from flytekit.models.core import workflow as _workflow_model
from flytekit.models.core.literals import Primitive
from flytekit.types.pickle.pickle import FlytePickleTransformer


def translate_inputs_to_literals(
3 changes: 1 addition & 2 deletions tests/flytekit/unit/core/test_type_hints.py
Original file line number Diff line number Diff line change
@@ -1427,7 +1427,6 @@ def foo3(a: typing.Dict) -> typing.Dict:

with pytest.raises(
TypeError,
match="Not a collection type simple: STRUCT\nmetadata {\n fields {\n key: 'python_class_name'\n"
" value {\n string_value: 'VT'\n }\n }\n}\n but got a list \\[{'hello': 2\\}]",
match="Not a collection type simple: STRUCT",
):
foo3(a=[{"hello": 2}])