Email notification support

[fenglixa]

/kind feature

Description:
[A clear and concise description of what your proposal. What problem does it solve?]
Tekton community already has sendmail catalog: https://github.com/tektoncd/catalog/tree/master/task/sendmail, kfp-tekton community can enhance dsl to support it accordingly.

Details of the the notification requirement:

User can specify a parameter whose value you indicate at run time for the SMTP Mail server name field, Senders email address field, Recipients email address field, and Email subject field.

The Notification stage includes the following fields.

• SMTP Mail server name

The name of the server or its IP address. You can specify a parameter whose value you indicate at run time.
Senders email address
The email address that the notification is sent from.
Recipients email address
The email address that the notification is sent to. You can specify multiple email addresses, separated by a space.

• Email subject

The text that is included in the subject line of the notification.

• Attachments

Files to be sent with the notification. Specify a path name or a comma-separated list of pathnames, which must be enclosed in single-quotes or double-quotes. You can also specify an expression that resolves to a pathname or comma-separated pathnames.

• Email body

The text that is included in the body of the notification.

• Include job status in email

specifies whether you want to include available job status information in the message.

Additional information:
[Miscellaneous information that will assist in solving the issue.]

[fenglixa]

Potaintail Solution/Findings:

1. Catalog: https://github.com/tektoncd/catalog/tree/master/task/sendmail, kfp-tekton community can enhance dsl to support it accordingly.

2. Support notification after a pipeline run done pipelines#3516

3. looks like it was being implemented upstream by using CloudEvents Optionally emit events during execution tektoncd/pipeline#2082

[Tomcli]

We can make the sendmail catalog task as an kubeflow component.yaml. This way it can be used as exit handler or as a regular containerOp.

[Tomcli]

@fenglixa can you give me an example for this? Can this run at the end of the pipeline or any time when a task is finished?

[fenglixa]

Sure, the notification could be an task inside of the pipeline, as below a typical example:

I get the information that CPD platform already has notification API, so this feature should be not tricky to us currently.

[jinchihe]

I saw @Tomcli implements, that's great.

One question, user need to set env from secret when compiling as below

kfp-tekton/samples/exit-handler-email/send-email.py

Lines 29 to 38 in d5e86d7

	def env_from_secret(env_name, secret_name, secret_key):
	return k8s_client.V1EnvVar(
	name=env_name,
	value_from=k8s_client.V1EnvVarSource(
	secret_key_ref=k8s_client.V1SecretKeySelector(
	name=secret_name,
	key=secret_key
	)
	)
	)

that's a little complicated to use/load the component, since user may just want to focus on inputs of components.

Shall we wrap this in DSL, or we build a image with k8s python SDK? user just need to input secret name and we get the value in the container, no need end user to set this while compiling? I tried, seems we cannot set the env (value from secret) in component definition.

Comments? @Tomcli

[fenglixa]

/close via PR #386

[k8s-ci-robot]

@fenglixa: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[fenglixa]

/reopen since still question/issue on #366 (comment)

[fenglixa]

/reopen

[k8s-ci-robot]

@fenglixa: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Tomcli]

I saw @Tomcli implements, that's great.

One question, user need to set env from secret when compiling as below

kfp-tekton/samples/exit-handler-email/send-email.py

Lines 29 to 38 in d5e86d7

	def env_from_secret(env_name, secret_name, secret_key):
	return k8s_client.V1EnvVar(
	name=env_name,
	value_from=k8s_client.V1EnvVarSource(
	secret_key_ref=k8s_client.V1SecretKeySelector(
	name=secret_name,
	key=secret_key
	)
	)
	)

that's a little complicated to use/load the component, since user may just want to focus on inputs of components.

Shall we wrap this in DSL, or we build a image with k8s python SDK? user just need to input secret name and we get the value in the container, no need end user to set this while compiling? I tried, seems we cannot set the env (value from secret) in component definition.

Comments? @Tomcli

I can put this helper function as part of the kfp-tekton package, but users will still need to run something like containerOp().apply(kfp_tekton.k8s_helper.env_from_secret()). Is it fine for your?