pyflyte run remote file #1670
Merged
pyflyte run remote file #1670
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ChungYujoyce
requested review from
wild-endeavor,
kumare3,
eapolinario,
pingsutw and
cosmicBboy
as code owners
Signed-off-by: ChungYujoyce <[email protected]>
Signed-off-by: ChungYujoyce <[email protected]>
Signed-off-by: ChungYujoyce <[email protected]>
ChungYujoyce
force-pushed
the
downloadfile
branch
from
0faca87 to
6717973
Compare
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Codecov Report
@@ Coverage Diff @@
## master #1670 +/- ##
==========================================
+ Coverage 71.03% 71.04% +0.01%
==========================================
Files 336 337 +1
Lines 30798 30821 +23
Branches 5589 5595 +6
==========================================
+ Hits 21876 21897 +21
Misses 8375 8375
- Partials 547 549 +2
|
pingsutw
previously approved these changes
Jun 8, 2023
Signed-off-by: Kevin Su <[email protected]>
|
Can you write an example |
pingsutw
approved these changes
Jun 13, 2023
|
Congrats on merging your first pull request! 🎉 |
eapolinario
pushed a commit
that referenced
this pull request
Jun 29, 2023
Signed-off-by: ChungYujoyce <[email protected]> Signed-off-by: Kevin Su <[email protected]> Co-authored-by: Kevin Su <[email protected]>
eapolinario
added a commit
that referenced
this pull request
Jun 30, 2023
* FLYTECTL_CONFIG env var higher precedence, config flag respected in pyflyte package (#1662) Signed-off-by: Yee Hing Tong <[email protected]> * Change flytekit Pytorch, TFJob and MPI plugins to use new kubeflow config (#1627) * upgrade tensorflow plugin to v1 Signed-off-by: Yubo Wang <[email protected]> * minor fix Signed-off-by: Yubo Wang <[email protected]> * fix tests and lints Signed-off-by: Yubo Wang <[email protected]> * move models file into task make backward compatible Signed-off-by: Yubo Wang <[email protected]> Signed-off-by: Yubo Wang <[email protected]> * add code example in README Signed-off-by: Yubo Wang <[email protected]> * bump flyteidl Signed-off-by: Yubo Wang <[email protected]> * add pytorch Signed-off-by: Yubo Wang <[email protected]> * add mpi and fix requirements.txt Signed-off-by: Yubo Wang <[email protected]> * lint and fmt Signed-off-by: Yubo Wang <[email protected]> * Regenerate requirements files using python 3.8 Signed-off-by: eduardo apolinario <[email protected]> --------- Signed-off-by: Yubo Wang <[email protected]> Signed-off-by: eduardo apolinario <[email protected]> Co-authored-by: Yubo Wang <[email protected]> Co-authored-by: eduardo apolinario <[email protected]> * Root cert should be byte string when loading from caCertFilePath (#1669) Signed-off-by: Yee Hing Tong <[email protected]> * Explicitly set the content type for flyte deck (#1658) * Set content type for flyte deck Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * nit Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * nit Signed-off-by: Kevin Su <[email protected]> * nit Signed-off-by: Kevin Su <[email protected]> --------- Signed-off-by: Kevin Su <[email protected]> * Use protos of new kubeflow.pytorch plugin instead of legacy pytorch plugin (#1678) Signed-off-by: Fabio Grätz <[email protected]> Co-authored-by: Fabio Grätz <[email protected]> * More time info for time line deck (#1680) * more visualization Signed-off-by: Yicheng-Lu-llll <[email protected]> * more visualization Signed-off-by: Yicheng-Lu-llll <[email protected]> --------- Signed-off-by: Yicheng-Lu-llll <[email protected]> * Add http_proxy to client & Fix deviceflow (#1611) * Add http_proxy to client & Fix deviceflow RB=3890720 Signed-off-by: byhsu <[email protected]> * nit Signed-off-by: byhsu <[email protected]> * lint! Signed-off-by: byhsu <[email protected]> --------- Signed-off-by: byhsu <[email protected]> Co-authored-by: byhsu <[email protected]> * Pass verify flag to all authenticators (#1641) Signed-off-by: byhsu <[email protected]> * feat: Add Auth0/audience support for ClientCredentials flow (#1639) * feat: Add Auth0/audience support for ClientCredentials flow Signed-off-by: tnam <[email protected]> * refactor: Remove unneeded variables & condense code Signed-off-by: tnam <[email protected]> * refactor: Reduce verbosity of code Signed-off-by: tnam <[email protected]> * refactor(chore): Remove unused commented code Signed-off-by: tnam <[email protected]> * fix: Missing comma in input args - authenticator.py 213 Signed-off-by: tnam <[email protected]> * style: Run pre-commit against all files Signed-off-by: tnam <[email protected]> --------- Signed-off-by: tnam <[email protected]> Co-authored-by: tnam <[email protected]> * pyflyte run remote file (#1670) Signed-off-by: ChungYujoyce <[email protected]> Signed-off-by: Kevin Su <[email protected]> Co-authored-by: Kevin Su <[email protected]> * upload deck.html only with deck enable (#1693) Signed-off-by: Yicheng-Lu-llll <[email protected]> * Add dask plugin #patch (#1366) * Add dummy task type to test backend plugin Signed-off-by: Bernhard Stadlbauer <[email protected]> * Add docs page Signed-off-by: Bernhard Stadlbauer <[email protected]> * Add dask models Signed-off-by: Bernhard Stadlbauer <[email protected]> * Add function to convert resources Signed-off-by: Bernhard Stadlbauer <[email protected]> * Add tests to `dask` task Signed-off-by: Bernhard Stadlbauer <[email protected]> * Remove namespace Signed-off-by: Bernhard Stadlbauer <[email protected]> * Update setup.py Signed-off-by: Bernhard Stadlbauer <[email protected]> * Add dask to `plugin/README.md` Signed-off-by: Bernhard Stadlbauer <[email protected]> * Add README.md for `dask` Signed-off-by: Bernhard Stadlbauer <[email protected]> * Top level export of `JopPodSpec` and `DaskCluster` Signed-off-by: Bernhard Stadlbauer <[email protected]> * Update docs for images Signed-off-by: Bernhard Stadlbauer <[email protected]> * Update README.md Signed-off-by: Bernhard Stadlbauer <[email protected]> * Update models after `flyteidl` change Signed-off-by: Bernhard Stadlbauer <[email protected]> * Update task after `flyteidl` change Signed-off-by: Bernhard Stadlbauer <[email protected]> * Raise error when less than 1 worker Signed-off-by: Bernhard Stadlbauer <[email protected]> * Update flyteidl to >= 1.3.2 Signed-off-by: Bernhard Stadlbauer <[email protected]> * Update doc requirements Signed-off-by: Bernhard Stadlbauer <[email protected]> * Update doc-requirements.txt Signed-off-by: Bernhard Stadlbauer <[email protected]> * Re-lock dependencies on linux Signed-off-by: Bernhard Stadlbauer <[email protected]> * Update dask API docs Signed-off-by: Bernhard Stadlbauer <[email protected]> * Fix documentation links Signed-off-by: Bernhard Stadlbauer <[email protected]> * Default optional model constructor arguments to `None` Signed-off-by: Bernhard Stadlbauer <[email protected]> * Refactor `convert_resources_to_resource_model` to `core.resources` Signed-off-by: Bernhard Stadlbauer <[email protected]> * Use `convert_resources_to_resource_model` in `core.node` Signed-off-by: Bernhard Stadlbauer <[email protected]> * Incorporate review feedback Signed-off-by: Eduardo Apolinario <[email protected]> * Lint Signed-off-by: Eduardo Apolinario <[email protected]> Signed-off-by: Bernhard Stadlbauer <[email protected]> Signed-off-by: Bernhard Stadlbauer <[email protected]> Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> * Do not bring dask changes Signed-off-by: eduardo apolinario <[email protected]> * Remove readthedocs Signed-off-by: eduardo apolinario <[email protected]> * Linting Signed-off-by: eduardo apolinario <[email protected]> * Force scipy<1.11.0 is on windows Signed-off-by: eduardo apolinario <[email protected]> --------- Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Yubo Wang <[email protected]> Signed-off-by: eduardo apolinario <[email protected]> Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Fabio Grätz <[email protected]> Signed-off-by: Yicheng-Lu-llll <[email protected]> Signed-off-by: byhsu <[email protected]> Signed-off-by: tnam <[email protected]> Signed-off-by: ChungYujoyce <[email protected]> Signed-off-by: Bernhard Stadlbauer <[email protected]> Signed-off-by: Bernhard Stadlbauer <[email protected]> Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Yee Hing Tong <[email protected]> Co-authored-by: Yubo Wang <[email protected]> Co-authored-by: Yubo Wang <[email protected]> Co-authored-by: eduardo apolinario <[email protected]> Co-authored-by: Kevin Su <[email protected]> Co-authored-by: Fabio M. Graetz, Ph.D <[email protected]> Co-authored-by: Fabio Grätz <[email protected]> Co-authored-by: Yicheng-Lu-llll <[email protected]> Co-authored-by: ByronHsu <[email protected]> Co-authored-by: byhsu <[email protected]> Co-authored-by: TomNam <[email protected]> Co-authored-by: tnam <[email protected]> Co-authored-by: ChungYujoyce <[email protected]> Co-authored-by: bstadlbauer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add remote-file download and delete feature.
pyflyte run --remote https://raw.githubusercontent.com/flyteorg/flytesnacks/master/cookbook/core/flyte_basics/basic_workflow.py my_wf --a 3 --b testType
Are all requirements met?
Complete description
Users can provide a remote workflow file link and Flytekit would download it.
After workflow registration, the file would be removed automatically.
Tracking Issue
NA
Follow-up issue
NA