You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What steps did you take and what happened:
Generate a custom task which accepts literals as all of its parameters and at least one of these parameters matches the name of some pipeline parameter.
In the following example code:
importitertoolsfromkfpimportdsl, componentsfromkfp_tekton.tektonimportTEKTON_CUSTOM_TASK_IMAGESfromkfp_tekton.compilerimportTektonCompilerimportyamlARTIFACT_FETCHER_IMAGE_NAME="fetcher/image:latest"TEKTON_CUSTOM_TASK_IMAGES=TEKTON_CUSTOM_TASK_IMAGES.append(ARTIFACT_FETCHER_IMAGE_NAME)
_artifact_fetcher_no=0defartifact_fetcher(**artifact_paths: str):
'''A containerOp template resolving some artifacts, given their paths.'''global_artifact_fetcher_notemplate_yaml= {
'name': f'artifact-fetcher-{_artifact_fetcher_no}',
'description': 'Artifact Fetch',
'inputs': [
{'name': name, 'type': 'String'}
fornameinartifact_paths.keys()
],
'outputs': [
{'name': name, 'type': 'Artifact'}
fornameinartifact_paths.keys()
],
'implementation': {
'container': {
'image': ARTIFACT_FETCHER_IMAGE_NAME,
'command': ['sh', '-c'], # irrelevant'args': [
'--apiVersion', 'fetcher.tekton.dev/v1alpha1',
'--kind', 'FETCHER',
'--name', f'artifact_fetcher_{_artifact_fetcher_no}',
*itertools.chain(*[
(f'--{name}', {'inputValue': name})
fornameinartifact_paths.keys()
])
]
}
}
}
_artifact_fetcher_no+=1template_str=yaml.dump(template_yaml, Dumper=yaml.SafeDumper)
template=components.load_component_from_text(template_str)
op=template(**artifact_paths)
op.add_pod_annotation("valid_container", "false")
returnop@dsl.pipeline("all-literals")defall_literals(foo: str):
global_artifact_fetcher_no# no literals_artifact_fetcher_no=0op00=artifact_fetcher(bar=foo)
op01=artifact_fetcher(foo=foo)
# not all inputs are literals_artifact_fetcher_no=10op10=artifact_fetcher(foo="foo", bar=foo)
op11=artifact_fetcher(foo=foo, bar="bar")
# all inputs are literals but none of them# matches the name of any pipeline param_artifact_fetcher_no=20op20=artifact_fetcher(bar="bar")
op21=artifact_fetcher(bar="bar", buzz="buzz")
# all inputs are literals and at least one of them# matches the name of a pipeline param_artifact_fetcher_no=30op30=artifact_fetcher(foo="foo")
op31=artifact_fetcher(foo="bar") # doesn't matter what the literal isop32=artifact_fetcher(foo="foo", bar="bar")
op33=artifact_fetcher(foo="bar", bar="foo")
if__name__=='__main__':
TektonCompiler().compile(all_literals, __file__.replace('.py', '.yaml'))
Note how cases 0x, 1x and 2x are all correct, but for 3x, "foo" parameter is assigned $(params.foo) despite being passed a some literal in the DSL code. The value of literal seems to be irrelevant.
What did you expect to happen:
The 3x cases compiling just fine, to:
Additional information:
In compiler.py, I can see the following check:
iftask.get('orig_params', []): # custom task
...but no orig-param is generated for literals, so that list is empty in such cases. In such a case, it's handled as if it was a "normal" task --- and kfp-tekton seems to utilize the fact that the name matching a pipeline or loop parameter means that parameter actually origins from that pipeline or loop parameter:
This is probably because in custom-task, the name is directly controlled by the user, while for "normal" tasks --- the names are controlled by kubeflow-pipelines and the compiler can (but should it?) reason about parameters based on their names.
It seems to me it should be enough to change:
iftask.get('orig_params', []): # custom task
to:
if'orig_params'intask: # custom task
...as "normal" Tekton tasks don't have 'orig_params' key at all --- now at the moment at least. An additional parameter (e.g. 'custom_task': true), removed at the end of this if could be added too.
* add test for custom-tasks with literals
* fix orig_params check for custom tasks
* fix: style: min 2 spaces before inline comment
* fix: style: 2 empty line before function decl
/kind bug
What steps did you take and what happened:
Generate a custom task which accepts literals as all of its parameters and at least one of these parameters matches the name of some pipeline parameter.
In the following example code:
Produces the following yaml:
Note how cases 0x, 1x and 2x are all correct, but for 3x, "foo" parameter is assigned
$(params.foo)
despite being passed a some literal in the DSL code. The value of literal seems to be irrelevant.What did you expect to happen:
The 3x cases compiling just fine, to:
Additional information:
In compiler.py, I can see the following check:
...but no orig-param is generated for literals, so that list is empty in such cases. In such a case, it's handled as if it was a "normal" task --- and kfp-tekton seems to utilize the fact that the name matching a pipeline or loop parameter means that parameter actually origins from that pipeline or loop parameter:
This is probably because in custom-task, the name is directly controlled by the user, while for "normal" tasks --- the names are controlled by kubeflow-pipelines and the compiler can (but should it?) reason about parameters based on their names.
It seems to me it should be enough to change:
to:
...as "normal" Tekton tasks don't have
'orig_params'
key at all --- now at the moment at least. An additional parameter (e.g.'custom_task': true
), removed at the end of thisif
could be added too.Environment:
python --version
): 3.9.0tkn version
): irrelevantkubectl version
): irrelevant/etc/os-release
): irrelevantThe text was updated successfully, but these errors were encountered: