Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create new compiler helper method to return workflow dictionary only #737

Closed
Tomcli opened this issue Sep 22, 2021 · 0 comments · Fixed by #757
Closed

Create new compiler helper method to return workflow dictionary only #737

Tomcli opened this issue Sep 22, 2021 · 0 comments · Fixed by #757

Comments

@Tomcli
Copy link
Member

Tomcli commented Sep 22, 2021

@Tomcli

One thing comes to mind: I noticed that _write_workflow has the ability to simply return the yaml in-memory:

if package_path is None:
  return yaml_text

However, _create_and_write_workflow doesn't propagate that, as it doesn't return the _write_workflow's result. In the original KFP it wasn't a real issue, as the implementation of _create_and_write_workflow was as simple as:

workflow = self._create_workflow(
    pipeline_func,
    pipeline_name,
    pipeline_description,
    params_list,
    pipeline_conf)
self._write_workflow(workflow, package_path)
_validate_workflow(workflow)

so one could simply use _create_workflow if really needed. It's to as easy for kfp-tekton, _create_and_write_workflow being 60 lines.

Considering how compile is based on Argo KFP's version, it probably cannot be changed that way, but maybe another method that would return a yaml (or maybe a list of yamls, considering the possibility of multiple resources?) in-memory could be added? That could possibly result in an even bigger speed-up for some cases.

Originally posted by @Udiknedormin in #736 (comment)

We want to have a new method to return the workflow dictionary in Python memory to optimize the API response time when running the SDK as a service.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant