Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return error code on fail #1408

Merged
merged 6 commits into from
Jan 24, 2023
Merged

Return error code on fail #1408

merged 6 commits into from
Jan 24, 2023

Conversation

pingsutw
Copy link
Member

@pingsutw pingsutw commented Jan 11, 2023

Signed-off-by: Kevin Su [email protected]

TL;DR

aws batch get the status of job from return code. Therefore, we should return error code once we catch the error.

blocked by flyteorg/flyteplugins#306

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

Complete description

image

Tracking Issue

https://flyte-org.slack.com/archives/C01P3B761A6/p1664972219043289
flyteorg/flyte#2979

Signed-off-by: Kevin Su <[email protected]>
@codecov
Copy link

codecov bot commented Jan 11, 2023

Codecov Report

Merging #1408 (cafad0c) into master (b6605bc) will increase coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1408      +/-   ##
==========================================
+ Coverage   69.22%   69.24%   +0.02%     
==========================================
  Files         300      300              
  Lines       28126    28147      +21     
  Branches     2647     2649       +2     
==========================================
+ Hits        19470    19491      +21     
  Misses       8142     8142              
  Partials      514      514              
Impacted Files Coverage Δ
tests/flytekit/unit/bin/test_python_entrypoint.py 99.55% <100.00%> (+0.04%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@@ -161,6 +161,12 @@ def _dispatch_execute(
logger.info(f"Engine folder written successfully to the output prefix {output_prefix}")
logger.debug("Finished _dispatch_execute")

if os.environ.get("FLYTE_FAIL_ON_ERROR", "").lower() == "true" and _constants.ERROR_FILE_NAME in output_file_dict:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a weird way this code is written. But, looks good to me

@pingsutw pingsutw merged commit 99722d5 into master Jan 24, 2023
ByronHsu pushed a commit to ByronHsu/flytekit that referenced this pull request Feb 1, 2023
* AWS batch return error code once it fails

Signed-off-by: Kevin Su <[email protected]>

* AWS batch return error code once it fails

Signed-off-by: Kevin Su <[email protected]>

* update tests

Signed-off-by: Kevin Su <[email protected]>

* Update tests

Signed-off-by: Kevin Su <[email protected]>

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: byhsu <[email protected]>
samhita-alla pushed a commit that referenced this pull request Feb 2, 2023
* AWS batch return error code once it fails

Signed-off-by: Kevin Su <[email protected]>

* AWS batch return error code once it fails

Signed-off-by: Kevin Su <[email protected]>

* update tests

Signed-off-by: Kevin Su <[email protected]>

* Update tests

Signed-off-by: Kevin Su <[email protected]>

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Samhita Alla <[email protected]>
eapolinario pushed a commit that referenced this pull request Feb 8, 2023
* AWS batch return error code once it fails

Signed-off-by: Kevin Su <[email protected]>

* AWS batch return error code once it fails

Signed-off-by: Kevin Su <[email protected]>

* update tests

Signed-off-by: Kevin Su <[email protected]>

* Update tests

Signed-off-by: Kevin Su <[email protected]>

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
eapolinario pushed a commit that referenced this pull request Feb 22, 2023
* AWS batch return error code once it fails

Signed-off-by: Kevin Su <[email protected]>

* AWS batch return error code once it fails

Signed-off-by: Kevin Su <[email protected]>

* update tests

Signed-off-by: Kevin Su <[email protected]>

* Update tests

Signed-off-by: Kevin Su <[email protected]>

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
eapolinario added a commit that referenced this pull request Feb 23, 2023
* Force flyteidl==1.2.9

Signed-off-by: Eduardo Apolinario <[email protected]>

* Sanitize query template input in sqlite task (#1359)

Signed-off-by: Eduardo Apolinario <[email protected]>

Signed-off-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>

* TypeTransformer for reading and writing from TensorFlowRecord format (#1240)

* first commit

Signed-off-by: Ryan Nazareth <[email protected]>

* add tensorflow example tf record transformer

Signed-off-by: Ryan Nazareth <[email protected]>

* refactor

Signed-off-by: Ryan Nazareth <[email protected]>

* correct tfexample description

Signed-off-by: Ryan Nazareth <[email protected]>

* fix test_native.py

Signed-off-by: Ryan Nazareth <[email protected]>

* add tensorflow docs and reqs

Signed-off-by: Ryan Nazareth <[email protected]>

* add tensorflow docs and reqs1

Signed-off-by: Ryan Nazareth <[email protected]>

* tensorflow import in init

Signed-off-by: Ryan Nazareth <[email protected]>

* fix failing tests

Signed-off-by: Ryan Nazareth <[email protected]>

* add tensorflow pinned version to reqs

Signed-off-by: Ryan Nazareth <[email protected]>

* pin grpcio-status to remove protobuf error

Signed-off-by: Ryan Nazareth <[email protected]>

* add suggested changes

Signed-off-by: Ryan Nazareth <[email protected]>

* redesign transformer

Signed-off-by: Ryan Nazareth <[email protected]>

* remove old script

Signed-off-by: Ryan Nazareth <[email protected]>

* fix type reference for TFREcordDataset

Signed-off-by: Ryan Nazareth <[email protected]>

* refactor

Signed-off-by: Ryan Nazareth <[email protected]>

* refactor

Signed-off-by: Ryan Nazareth <[email protected]>

* spacing and uppercase

Signed-off-by: Ryan Nazareth <[email protected]>

* redesign with tfdir and tfrecordfile subclass

Signed-off-by: Ryan Nazareth <[email protected]>

* fix conflicts and typos

Signed-off-by: Ryan Nazareth <[email protected]>

* address majority of comments

Signed-off-by: Ryan Nazareth <[email protected]>

* refactor

Signed-off-by: Ryan Nazareth <[email protected]>

* fix test with flytefile and metadata annotated

Signed-off-by: Ryan Nazareth <[email protected]>

* fix check for example records in directory

Signed-off-by: Ryan Nazareth <[email protected]>

* refactor and correct typing

Signed-off-by: Ryan Nazareth <[email protected]>

* lint

Signed-off-by: Ryan Nazareth <[email protected]>

* import annotated from typing_extensions

Signed-off-by: Ryan Nazareth <[email protected]>

* tweak to tests to test case when Config not passed in as type

Signed-off-by: Ryan Nazareth <[email protected]>

* add suggested changes

Signed-off-by: Ryan Nazareth <[email protected]>

* add task for tfrecord dir with no config in test

Signed-off-by: Ryan Nazareth <[email protected]>

* get filenames from local dir instead of remote

Signed-off-by: Ryan Nazareth <[email protected]>

Signed-off-by: Ryan Nazareth <[email protected]>

* update ray plugin dependency (#1361)

Signed-off-by: Kevin Su <[email protected]>

Signed-off-by: Kevin Su <[email protected]>

* Set default format of structured dataset to empty (#1159)

* Set default format of structured dataset to empty

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

* last error (#1364)

Signed-off-by: Yee Hing Tong <[email protected]>

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Co-authored-by: Yee Hing Tong <[email protected]>

* Adds CLI reference for pyflyte (#1362)

* Adds pyflyte CLI reference guide

Signed-off-by: Samhita Alla <[email protected]>

* bump python version

Signed-off-by: Samhita Alla <[email protected]>

* bump python version

Signed-off-by: Samhita Alla <[email protected]>

* resolve docs error

Signed-off-by: Samhita Alla <[email protected]>

* set nested to none

Signed-off-by: Samhita Alla <[email protected]>

* remove flyteidl version constraint

Signed-off-by: Samhita Alla <[email protected]>

* update requirements

Signed-off-by: Samhita Alla <[email protected]>

Signed-off-by: Samhita Alla <[email protected]>

* Signaling (#1133)

Signed-off-by: Yee Hing Tong <[email protected]>

* Adding created and updated at to ExecutionClosure model (#1371)

Signed-off-by: Yee Hing Tong <[email protected]>

* Add Databricks config to Spark Job (#1358)

Signed-off-by: Kevin Su <[email protected]>

* Add overwrite_cache option the to calls of remote and local executions (#1375)

Signed-off-by: H. Furkan Vural <[email protected]>

Implemented cache overwrite feature is added on flytekit as well for the completeness. In order to support the cache eviction RFC, an overwrite parameter was added, indicating the data store should replace an existing artifact instead of creating a new one on local calls.

* Remove project/domain from being overridden with execution values in serialized context (#1378)

Signed-off-by: Yee Hing Tong <[email protected]>

* Use TaskSpec instead of TaskTemplate for fetch_task and avoid network when loading module (#1348)

Signed-off-by: Ketan Umare <[email protected]>

* Register Databricks config (#1379)

* Register databricks plugin

Signed-off-by: Kevin Su <[email protected]>

* Update databricks plugin

Signed-off-by: Kevin Su <[email protected]>

* register databricks

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

Signed-off-by: Kevin Su <[email protected]>
Co-authored-by: Yee Hing Tong <[email protected]>

* PodSpec should not require primary_container name (#1380)

For Pod tasks, if the primary_container_name is not specified, it should default.

Signed-off-by: Ketan Umare <[email protected]>

* fix(pyflyte): change -d to -D for --destination-dir as -d is already for --domain (#1381)

Co-authored-by: Eduardo Apolinario <[email protected]>

* Handle Optional[FlyteFile] in Dataclass type transformer (#1393)

* Add support for Optional to dataclass transformer

Signed-off-by: Eduardo Apolinario <[email protected]>

* Add one more test

Signed-off-by: Eduardo Apolinario <[email protected]>

* Add one more test

Signed-off-by: Eduardo Apolinario <[email protected]>

* Fix serialization of optional flyte types

Signed-off-by: Eduardo Apolinario <[email protected]>

Signed-off-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>

* add FastSerializationSettings to docs (#1386)

Signed-off-by: Niels Bantilan <[email protected]>

Signed-off-by: Niels Bantilan <[email protected]>
Co-authored-by: Kevin Su <[email protected]>

* Added more pod tests and an example pod task (#1382)

* Added more pod tests and an example pod task

Signed-off-by: Ketan Umare <[email protected]>

* fixing test and name

Signed-off-by: Ketan Umare <[email protected]>

* updated

Signed-off-by: Ketan Umare <[email protected]>

Signed-off-by: Ketan Umare <[email protected]>

* Convert default dict to json string in pyflyte run (#1399)

Signed-off-by: Kevin Su <[email protected]>

Signed-off-by: Kevin Su <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>

* docs: update register help, non-fast version is supported (#1402)

Signed-off-by: Patrick Brogan <[email protected]>

* Update log level for structured dataset (#1394)

Signed-off-by: Kevin Su <[email protected]>

* Add Niels to code owners (#1404)

Signed-off-by: Kevin Su <[email protected]>

* Signal use (#1398)

Signed-off-by: Yee Hing Tong <[email protected]>

* User Documentation Proposal (#1200)

Signed-off-by: Kevin Su <[email protected]>

* Add support MLFlow plugin (#1274)

* MLFlow plugin in progress

Signed-off-by: Ketan Umare <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* update test

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* update readme

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* dwip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* change experiment name

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* Add mlflow to index.rst

Signed-off-by: Kevin Su <[email protected]>

* use experiment name that user provided

Signed-off-by: Kevin Su <[email protected]>

* update doc-requirements.txt

Signed-off-by: Kevin Su <[email protected]>

* Add backend plugin deployment

Signed-off-by: Kevin Su <[email protected]>

* generate doc for method

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

* update docstring

Signed-off-by: Niels Bantilan <[email protected]>

* update docstring

Signed-off-by: Niels Bantilan <[email protected]>

* Update tracking.py

Signed-off-by: Niels Bantilan <[email protected]>

Signed-off-by: Ketan Umare <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Niels Bantilan <[email protected]>
Co-authored-by: Kevin Su <[email protected]>
Co-authored-by: Niels Bantilan <[email protected]>

* fix remote API reference (#1405)

Signed-off-by: Niels Bantilan <[email protected]>

Signed-off-by: Niels Bantilan <[email protected]>

* Read structured dataset from a folder  (#1406)

* Read polars dataframe in a folder

Signed-off-by: Kevin Su <[email protected]>

* Read polars dataframe in a folder

Signed-off-by: Kevin Su <[email protected]>

* Load huggingface and spark plugin implicitly

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* remove _pyspark alias

Signed-off-by: Yee Hing Tong <[email protected]>

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Co-authored-by: Yee Hing Tong <[email protected]>

* Update default config to work out-of-the-box with flytectl demo (#1384)

Signed-off-by: Niels Bantilan <[email protected]>

* Add dask plugin #patch (#1366)

* Add dummy task type to test backend plugin

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Add docs page

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Add dask models

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Add function to convert resources

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Add tests to `dask` task

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Remove namespace

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Update setup.py

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Add dask to `plugin/README.md`

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Add README.md for `dask`

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Top level export of `JopPodSpec` and `DaskCluster`

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Update docs for images

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Update README.md

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Update models after `flyteidl` change

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Update task after `flyteidl` change

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Raise error when less than 1 worker

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Update flyteidl to >= 1.3.2

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Update doc requirements

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Update doc-requirements.txt

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Re-lock dependencies on linux

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Update dask API docs

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Fix documentation links

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Default optional model constructor arguments to `None`

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Refactor `convert_resources_to_resource_model` to `core.resources`

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Use `convert_resources_to_resource_model` in `core.node`

Signed-off-by: Bernhard Stadlbauer <[email protected]>

* Incorporate review feedback

Signed-off-by: Eduardo Apolinario <[email protected]>

* Lint

Signed-off-by: Eduardo Apolinario <[email protected]>

Signed-off-by: Bernhard Stadlbauer <[email protected]>
Signed-off-by: Bernhard Stadlbauer <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>

* Add support for overriding task configurations (#1410)

Signed-off-by: Kevin Su <[email protected]>

* Warning if git is not installed (#1414)

* warning if git is not installed

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

Signed-off-by: Kevin Su <[email protected]>

* Flip the settings for channel and logger (#1415)

Signed-off-by: Yee Hing Tong <[email protected]>

* Preserving Exception in the LazyEntity fetch (#1412)

* Preserving Exception in the LazyEntity fetch

Signed-off-by: Ketan Umare <[email protected]>

* updated lint error

Signed-off-by: Ketan Umare <[email protected]>

* more tests

Signed-off-by: Ketan Umare <[email protected]>

Signed-off-by: Ketan Umare <[email protected]>

* [Docs] SynchronousFlyteClient API reference #3095 (#1416)

Signed-off-by: Peeter Piegaze <[email protected]>

Signed-off-by: Peeter Piegaze <[email protected]>
Co-authored-by: Peeter Piegaze <[email protected]>
Co-authored-by: Haytham Abuelfutuh <[email protected]>

* Return error code on fail (#1408)

* AWS batch return error code once it fails

Signed-off-by: Kevin Su <[email protected]>

* AWS batch return error code once it fails

Signed-off-by: Kevin Su <[email protected]>

* update tests

Signed-off-by: Kevin Su <[email protected]>

* Update tests

Signed-off-by: Kevin Su <[email protected]>

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>

* wrapping flyte entity in a task node in call to flyte node constructor, not sure if integration tests are actually running (#1422)

Signed-off-by: Yee Hing Tong <[email protected]>

Signed-off-by: Yee Hing Tong <[email protected]>

* Sqlalchemy multiline query (#1421)

* SQLAlchemyTask should handle multiline strings for query template

Signed-off-by: Niels Bantilan <[email protected]>

* sqlalchemy supports multi-line query

Signed-off-by: Niels Bantilan <[email protected]>

* update base sql task

Signed-off-by: Niels Bantilan <[email protected]>

* remove space

Signed-off-by: Niels Bantilan <[email protected]>

* fix snowflake tests

Signed-off-by: Niels Bantilan <[email protected]>

* fix lint

Signed-off-by: Niels Bantilan <[email protected]>

* fix test

Signed-off-by: Niels Bantilan <[email protected]>

Signed-off-by: Niels Bantilan <[email protected]>

* Sklearn type transformer should be automatically loaded with import flytekit (#1423)

* add flytekit.extras.sklearn to main __init__ import

Signed-off-by: Niels Bantilan <[email protected]>

* fix docs

Signed-off-by: Niels Bantilan <[email protected]>

* add temporary docs/requirements.txt for onnx plugins

Signed-off-by: Niels Bantilan <[email protected]>

---------

Signed-off-by: Niels Bantilan <[email protected]>

* Bump isort to 5.12.0 (#1427)

Signed-off-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>

* Fixes guess type bug in UnionTransformer (#1426)

Signed-off-by: Ketan Umare <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>

* Add `pod_template` and `pod_template_name` arguments for `PythonAutoContainerTask`, its downstream tasks, and `@task`. (#1425)

* Add `pod_template` and `pod_template_name` arguments for `PythonAutoContainerTask`, its downstream tasks, and `@task`

Signed-off-by: byhsu <[email protected]>

* clean

Signed-off-by: byhsu <[email protected]>

* fix test

Signed-off-by: byhsu <[email protected]>

* Fix taskmetadata

Signed-off-by: byhsu <[email protected]>

* add kubernetes in setup.py

Signed-off-by: byhsu <[email protected]>

* address comments

Signed-off-by: byhsu <[email protected]>

* Regenerate requirements using python 3.7

Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: byhsu <[email protected]>

* keep container validation

Signed-off-by: byhsu <[email protected]>

* bump idl version

Signed-off-by: byhsu <[email protected]>

* Regenerate requirements using python 3.7

Signed-off-by: Eduardo Apolinario <[email protected]>

* Regenerate doc-requirements.txt

Signed-off-by: Eduardo Apolinario <[email protected]>

* fix

Signed-off-by: byhsu <[email protected]>

---------

Signed-off-by: byhsu <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Co-authored-by: byhsu <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>

* Auto Backfill workflow (#1420)

* Fix primitive decoder when evaluating Promise (#1432)

Signed-off-by: Samhita Alla <[email protected]>

* set maximum python version to 3.10 (#1433)

* set maximum python version to 3.10

Signed-off-by: Niels Bantilan <[email protected]>

* remove unneeded python version check

Signed-off-by: Niels Bantilan <[email protected]>

* fix lint

Signed-off-by: Niels Bantilan <[email protected]>

---------

Signed-off-by: Niels Bantilan <[email protected]>

* Revert "Remove project/domain from being overridden with execution values in serialized context (#1378)" (#1460)

* Revert "Remove project/domain from being overridden with execution values in serialized context (#1378)"

This reverts commit b3bfef5.

* Import os

Signed-off-by: Eduardo Apolinario <[email protected]>

* Lint

Signed-off-by: Eduardo Apolinario <[email protected]>

---------

Signed-off-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>

* Support checkpointing in local mode from cached tasks (#1457)

* support checkpointing in local mode from cached tasks

* clear cache before tests

---------

Co-authored-by: Stef Nelson-Lindall <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>

* Deprecate FlyteSchema (#1418)

* Deprecate FlyteSchema

Signed-off-by: Kevin Su <[email protected]>

* Remove version

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>

* Use scarf images (#1434)

* Use scarf images

Signed-off-by: Eduardo Apolinario <[email protected]>

* Use scarf names in tests.

Signed-off-by: Eduardo Apolinario <[email protected]>

---------

Signed-off-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>

* add undocumented objects/functions to flytekit api ref (#1502)

* add reference_launch_plan to flytekit api ref

Signed-off-by: Niels Bantilan <[email protected]>

* import in init, add docstrings

Signed-off-by: Niels Bantilan <[email protected]>

* add more to references

Signed-off-by: Niels Bantilan <[email protected]>

* fix lint

Signed-off-by: Niels Bantilan <[email protected]>

* update

Signed-off-by: Niels Bantilan <[email protected]>

* fix up docstrings

Signed-off-by: Niels Bantilan <[email protected]>

---------

Signed-off-by: Niels Bantilan <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Samhita Alla <[email protected]>

* Use non-root user in default flytekit image (#1417)

Signed-off-by: Kevin Su <[email protected]>

* Fix PyTorch transformer (#1510)

Signed-off-by: Samhita Alla <[email protected]>

* Fix mypy errors (#1313)

* wip

Signed-off-by: Kevin Su <[email protected]>

* Fix mypy errors

Signed-off-by: Kevin Su <[email protected]>

* Fix mypy errors

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* fix tests

Signed-off-by: Kevin Su <[email protected]>

* fix tests

Signed-off-by: Kevin Su <[email protected]>

* fix test

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* Update type

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* update dev-requirements.txt

Signed-off-by: Kevin Su <[email protected]>

* Address comment

Signed-off-by: Kevin Su <[email protected]>

* upgrade torch

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Co-authored-by: Yee Hing Tong <[email protected]>

* Compile the workflow only at compile time (#1311)

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* wip

Signed-off-by: Kevin Su <[email protected]>

* add tests

Signed-off-by: Kevin Su <[email protected]>

* add tests

Signed-off-by: Kevin Su <[email protected]>

* support dynamic task

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* lazy compile

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

* add tests

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* update test

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* Get the origin type when serializing dataclass (#1508)

* Get the origin type when serializing dataclass

Signed-off-by: Kevin Su <[email protected]>

* test

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* update test

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>
Co-authored-by: Niels Bantilan <[email protected]>

* Fix bad merge

Signed-off-by: Eduardo Apolinario <[email protected]>

* Delay initialization of SynchronousFlyteClient in FlyteRemote (#1514)

* Delay initialization of SynchronousFlyteClient in FlyteRemote

Signed-off-by: Eduardo Apolinario <[email protected]>

* Fix spark plugin flyteremote test.

Signed-off-by: Eduardo Apolinario <[email protected]>

* Lint

Signed-off-by: Eduardo Apolinario <[email protected]>

---------

Signed-off-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>

* Set flytekit and flyteidl bounds in plugins tests

Signed-off-by: Eduardo Apolinario <[email protected]>

* Revert "Fix mypy errors (#1313)"

This reverts commit 3798450.

Signed-off-by: Eduardo Apolinario <[email protected]>

* Fix requirements in dask and ray plugins

Signed-off-by: Eduardo Apolinario <[email protected]>

* Fix papermill tests requirements

Signed-off-by: Eduardo Apolinario <[email protected]>

* Fix doc-requirements

Signed-off-by: Eduardo Apolinario <[email protected]>

* dask plugin requirements

Signed-off-by: Eduardo Apolinario <[email protected]>

* Revert "Add dask plugin #patch (#1366)"

This reverts commit 41a9c7a.

Signed-off-by: Eduardo Apolinario <[email protected]>

---------

Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Ryan Nazareth <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Samhita Alla <[email protected]>
Signed-off-by: Ketan Umare <[email protected]>
Signed-off-by: Niels Bantilan <[email protected]>
Signed-off-by: Patrick Brogan <[email protected]>
Signed-off-by: Bernhard Stadlbauer <[email protected]>
Signed-off-by: Bernhard Stadlbauer <[email protected]>
Signed-off-by: Peeter Piegaze <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: byhsu <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Ryan Nazareth <[email protected]>
Co-authored-by: Kevin Su <[email protected]>
Co-authored-by: Yee Hing Tong <[email protected]>
Co-authored-by: Samhita Alla <[email protected]>
Co-authored-by: H. Furkan Vural <[email protected]>
Co-authored-by: Ketan Umare <[email protected]>
Co-authored-by: mcloney-ddm <[email protected]>
Co-authored-by: Niels Bantilan <[email protected]>
Co-authored-by: pbrogan12 <[email protected]>
Co-authored-by: bstadlbauer <[email protected]>
Co-authored-by: Peeter Piegaze <[email protected]>
Co-authored-by: Peeter Piegaze <[email protected]>
Co-authored-by: Haytham Abuelfutuh <[email protected]>
Co-authored-by: ByronHsu <[email protected]>
Co-authored-by: byhsu <[email protected]>
Co-authored-by: Stef Lindall <[email protected]>
Co-authored-by: Stef Nelson-Lindall <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants