feat(sdk): add v2 placeholder variables

[ji-yaqi]

No description provided.

[chensun]

What is the plan for RUN_ID_PLACEHOLDER? If we want to deprecate it in v2, we shouldn't add it here.

[ji-yaqi]

Removed.

[chensun]

nit: v2 placeholder variables

Same for the PR title.

[google-oss-robot]

@ji-yaqi: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
kubeflow-pipelines-samples-v2	3c1dae7	link	true	/test kubeflow-pipelines-samples-v2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[chensun]

/lgtm
/approve

Thanks!

[google-oss-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chensun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• sdk/OWNERS [chensun]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[chensun]

/kubeflow-pipelines-samples-v2 test is failing due to some known infra issue, and this PR shouldn't affect the that tests.
So merging the PR manually.

[judahrand]

{{$.pipeline_task_name}} and {{$.pipeline_task_uuid}} don't seem to work on Vertex AI at the moment (not sure about a self hosted KFP). Is this expected? I've also tried various placeholders to try to access the Pipeline job's status/error (eg. {{$.error}}, {{$.status}}, {{$.error.code}}, etc.) and nothing seems to work. How is this templating working and how does it differ from the old Argo templating?

[ji-yaqi]

{{$.pipeline_task_name}} and {{$.pipeline_task_uuid}} don't seem to work on Vertex AI at the moment (not sure about a self hosted KFP). Is this expected? I've also tried various placeholders to try to access the Pipeline job's status/error (eg. {{$.error}}, {{$.status}}, {{$.error.code}}, etc.) and nothing seems to work. How is this templating working and how does it differ from the old Argo templating?

Hi @judahrand, the usage would be like:

@dsl.pipeline(name=f'test-pipeline-1008-{USER}')
def my_pipeline():
    print_op(msg=dsl.PIPELINE_JOB_NAME_PLACEHOLDER)
    print_op(msg=dsl.PIPELINE_JOB_RESOURCE_NAME_PLACEHOLDER) 
    print_op(msg=dsl.PIPELINE_JOB_ID_PLACEHOLDER) 
    print_op(msg=dsl.PIPELINE_TASK_NAME_PLACEHOLDER) 
    print_op(msg=dsl.PIPELINE_TASK_ID_PLACEHOLDER) 

Is this the way you were using this feature?

[judahrand]

{{$.pipeline_task_name}} and {{$.pipeline_task_uuid}} don't seem to work on Vertex AI at the moment (not sure about a self hosted KFP). Is this expected? I've also tried various placeholders to try to access the Pipeline job's status/error (eg. {{$.error}}, {{$.status}}, {{$.error.code}}, etc.) and nothing seems to work. How is this templating working and how does it differ from the old Argo templating?

Hi @judahrand, the usage would be like:

@dsl.pipeline(name=f'test-pipeline-1008-{USER}')
def my_pipeline():
    print_op(msg=dsl.PIPELINE_JOB_NAME_PLACEHOLDER)
    print_op(msg=dsl.PIPELINE_JOB_RESOURCE_NAME_PLACEHOLDER) 
    print_op(msg=dsl.PIPELINE_JOB_ID_PLACEHOLDER) 
    print_op(msg=dsl.PIPELINE_TASK_NAME_PLACEHOLDER) 
    print_op(msg=dsl.PIPELINE_TASK_ID_PLACEHOLDER) 

Is this the way you were using this feature?

Yes, that's what I was doing. The task specific placeholders did not work while the job specific ones did.

Do you have any suggestions for alternatives to {{workflow.status}}? This seems like a crucial thing to be able to access for error handling in an exit handler.

[ji-yaqi]

Hi @judahrand, I tried to reproduce this on KFP 1.8.6 and PIPELINE_TASK_NAME_PLACEHOLDER and PIPELINE_TASK_ID_PLACEHOLDER works (shown in arguments).

For {{workflow.status}}, you can use the string as is. If you think extra is needed, could you file an issue? Thanks!

[judahrand]

Perhaps this is a Vertex specific issue?