-
Notifications
You must be signed in to change notification settings - Fork 669
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Core Feature] FlyteAdmin/Propeller should allow offloaded workflow specs #722
Labels
Comments
kumare3
added
enhancement
New feature or request
untriaged
This issues has not yet been looked at by the Maintainers
propeller
Issues related to flyte propeller
labels
Feb 11, 2021
kumare3
removed
the
untriaged
This issues has not yet been looked at by the Maintainers
label
Mar 1, 2021
palchicz
pushed a commit
to palchicz/flyte
that referenced
this issue
Dec 23, 2021
* schema in dataclass Signed-off-by: Kevin Su <[email protected]> * Added tests Signed-off-by: Kevin Su <[email protected]> * Fixed lint Signed-off-by: Kevin Su <[email protected]> * Updated tests Signed-off-by: Kevin Su <[email protected]> * Updated tests Signed-off-by: Kevin Su <[email protected]> * Fixed lint Signed-off-by: Kevin Su <[email protected]> * updated Signed-off-by: Kevin Su <[email protected]> * updated Signed-off-by: Kevin Su <[email protected]> * updated Signed-off-by: Kevin Su <[email protected]>
Completed in #2705 |
eapolinario
pushed a commit
to eapolinario/flyte
that referenced
this issue
Dec 20, 2022
* updat image Signed-off-by: Yee Hing Tong <[email protected]> * update sha again Signed-off-by: Yee Hing Tong <[email protected]> * delete flytekitvenv Signed-off-by: Yee Hing Tong <[email protected]> * fixes Signed-off-by: Yee Hing Tong <[email protected]> * delete bad file Signed-off-by: Yee Hing Tong <[email protected]> * revert 4.02 Signed-off-by: Yee Hing Tong <[email protected]> * reverttagdeletion Signed-off-by: Yee Hing Tong <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Motivation: Why do you think this is important?
There are cases in which the spec increases to a very large size - workflowtemplate or tasktempaltes with queries are very large. Also having the spec in etcD does not really offer any benefit.
Goal: What should the final outcome look like, ideally?
When a user executes a workflow, instead of writing the entire workflow to the CRD, we could write the spec to the Blob store and just pass a reference. This would have following benefits
Describe alternatives you've considered
Spec compression. This is very complicated and debugging will be harder.
[Optional] Propose: Link/Inline OR Additional context
Instead of the inline spec - we can just have a reference to the spec in blob store - https://github.com/flyteorg/flytepropeller/blob/master/pkg/apis/flyteworkflow/v1alpha1/workflow.go#L29
The text was updated successfully, but these errors were encountered: