Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Backend][Plugin] Kubeflow operators - Pytorch #338

Closed
2 of 6 tasks
kumare3 opened this issue Jun 2, 2020 · 4 comments
Closed
2 of 6 tasks

[Backend][Plugin] Kubeflow operators - Pytorch #338

kumare3 opened this issue Jun 2, 2020 · 4 comments
Milestone
0.4.0

Comments

@kumare3
Copy link
Contributor

kumare3 commented Jun 2, 2020

Why would this plugin be helpful to the Flyte community
Allow users to executed distributed pytorch tasks

Type of Plugin

  • Web Service (e.g. AWS Sagemaker, GCP DataFlow, Qubole etc...)
  • Kubernetes Operator (e.g. TfOperator, SparkOperator, FlinkK8sOperator, etc...)
  • Customized Plugin using native kubernetes constructs
  • Other

Can you help us with the implementation?
@kumare3 kumare3 added the untriaged This issues has not yet been looked at by the Maintainers label Jun 2, 2020
@kumare3 kumare3 added this to the 0.4.0 milestone Jun 2, 2020
@kumare3 kumare3 removed the untriaged This issues has not yet been looked at by the Maintainers label Jun 2, 2020
@kumare3
Copy link
Contributor Author

kumare3 commented Jun 2, 2020

flyteorg/flyteplugins#85

@kumare3 kumare3 mentioned this issue Jun 2, 2020
Merged
8 tasks
@kumare3
Copy link
Contributor Author

kumare3 commented Jun 10, 2020

This is code complete, pending documentation.

@kumare3
Copy link
Contributor Author

kumare3 commented Jun 10, 2020

@igorvalko is working on this

@kumare3
Copy link
Contributor Author

kumare3 commented Jun 16, 2020

All related commits
Creating an interface in FlyteIDL
flyteorg/flyteidl#61
Implementing the backend plugin
flyteorg/flyteplugins#85
Implementation in flytekit
flyteorg/flytekit#112
Example usage:
flyteorg/flytesnacks#11
Adding Pytorch operator to kustomize manifests
#330

A good way to test the entire Plugin in isolation is using
https://github.com/flyteorg/flytepluginexample

@kumare3 kumare3 closed this as completed Jun 16, 2020
@kumare3 kumare3 mentioned this issue Jun 16, 2020
Closed
4 tasks
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Dec 6, 2022
@kumare3
Default kubeconfig should be empty (flyteorg#338)
80c9717 
* Default kubeconfig should be empty

Signed-off-by: Ketan Umare <[email protected]>

* fixed generate

Signed-off-by: Ketan Umare <[email protected]>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Dec 20, 2022
Link athena docs from aws integrations page overview (flyteorg#338)
1ecdfd5
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Dec 20, 2022
@anrusina
chore: fix TaskType display to ensure consistency, code cleanup, add …
3c7f27c 
…mapTaskListItem component (flyteorg#338)

* chore: add component for mapped task support
* test: add test coverage for isMapTaskType and MapTaskStatusInfo
* chore: ignore *.stories.tsx files when collecting coverage
Signed-off-by: Nastya Rusina <[email protected]>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Jul 24, 2023
@hamersaw
mocking plugin state reader in maptask (flyteorg#338)
38e8a07 
Signed-off-by: Daniel Rammer <[email protected]>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Aug 21, 2023
@kumare3
Default kubeconfig should be empty (flyteorg#338)
a454295 
* Default kubeconfig should be empty

Signed-off-by: Ketan Umare <[email protected]>

* fixed generate

Signed-off-by: Ketan Umare <[email protected]>
eapolinario pushed a commit to eapolinario/flyte that referenced this issue Aug 21, 2023
@hamersaw
mocking plugin state reader in maptask (flyteorg#338)
eb522f4 
Signed-off-by: Daniel Rammer <[email protected]>
troychiu pushed a commit that referenced this issue Jul 8, 2024
@kamaleybov
Mount multiple file secrets using single init container (#338)
70ff33b 
## Overview
Follow up to #293, in which each secret mounted as file uses a separate init container. This change utilizes a single init container.

## Test Plan
1. Deploy to dogfood.
2. Create multiple secrets using unionai's `-f` flag and file.
3. Run a workflow using the secrets created in previous step.
4. Inspect pod spec.
5. Verify that secret values were read correctly.

## Rollout Plan (if applicable)
Run `managed-cluster-<env>-sync-all`.

## Upstream Changes
Should this change be upstreamed to OSS (flyteorg/flyte)? If not, please uncheck this box, which is used for auditing. Note, it is the responsibility of each developer to actually upstream their changes. See [this guide](https://unionai.atlassian.net/wiki/spaces/ENG/pages/447610883/Flyte+-+Union+Cloud+Development+Runbook/#When-are-versions-updated%3F).
- [ ] To be upstreamed to OSS

## Issue
ref COR-811

## Checklist
* [x] Added tests
* [ ] Ran a deploy dry run and shared the terraform plan
* [ ] Added logging and metrics
* [ ] Updated [dashboards](https://unionai.grafana.net/dashboards) and [alerts](https://unionai.grafana.net/alerting/list)
* [ ] Updated documentation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
0.4.0
Development

No branches or pull requests
1 participant
@kumare3