Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Plugin][Pod] Support local execution of simple pod tasks #1328

Closed
1 of 7 tasks
kumare3 opened this issue Aug 11, 2021 · 1 comment
Closed
1 of 7 tasks

[Plugin][Pod] Support local execution of simple pod tasks #1328

kumare3 opened this issue Aug 11, 2021 · 1 comment
Labels
flytekit FlyteKit Python related issue plugins Plugins related labels (backend or frontend)

Comments

@kumare3
Copy link
Contributor

kumare3 commented Aug 11, 2021

Why would this plugin be helpful to the Flyte community
K8sPods are supported as a separate entity from regular tasks in Flyte. Pods allow developers to access raw K8s features and are useful for extending infrastructure from the user pov. In many cases users of Flyte are forced to use pod tasks, because container tasks do not expose things like affinity, node pools etc. This is by design to allow maximum portability and reduce assumptions for the task author.

But, when users use pod tasks just to affinitize their workloads and not actually to run multiple containers, local execution can be extremely useful. Flytekit can allow local execution of Flyte pod tasks that do not use more than one container. In other cases it can continue to raise as error (current behavior)
This would make it simple for users to use pod tasks and test it locally.

Type of Plugin

  • Python/Java interface only plugin
  • Web Service (e.g. AWS Sagemaker, GCP DataFlow, Qubole etc...)
  • Kubernetes Operator (e.g. TfOperator, SparkOperator, FlinkK8sOperator, etc...)
  • Customized Plugin using native kubernetes constructs
  • Other

Can you help us with the implementation?

  • Yes
  • No
@kumare3 kumare3 added untriaged This issues has not yet been looked at by the Maintainers plugins Plugins related labels (backend or frontend) and removed untriaged This issues has not yet been looked at by the Maintainers labels Aug 11, 2021
@EngHabu EngHabu added the flytekit FlyteKit Python related issue label Aug 25, 2021
@wild-endeavor
Copy link
Contributor

This was updated recently and new behavior should be sufficient flyteorg/flytekit#852

neverett added a commit that referenced this issue Feb 28, 2024
* add failure node example from flyteorg/flytesnacks#1328

Signed-off-by: nikki everett <[email protected]>

* fix formatting and index toc

Signed-off-by: nikki everett <[email protected]>

* copy edits

Signed-off-by: nikki everett <[email protected]>

---------

Signed-off-by: nikki everett <[email protected]>
yubofredwang pushed a commit to yubofredwang/flyte that referenced this issue Mar 26, 2024
* add failure node example from flyteorg/flytesnacks#1328

Signed-off-by: nikki everett <[email protected]>

* fix formatting and index toc

Signed-off-by: nikki everett <[email protected]>

* copy edits

Signed-off-by: nikki everett <[email protected]>

---------

Signed-off-by: nikki everett <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flytekit FlyteKit Python related issue plugins Plugins related labels (backend or frontend)
Projects
None yet
Development

No branches or pull requests

3 participants