Updated with scheduling capability for kfp.Client

[NikeNano]

This PR includes code to add the functionality to schedule a pipeline from the python sdk. The added code is written to be similar to the implementation of how to run a pipeline in order to keep the API togheter.

I believe this is a feature that would be great to have in Kubeflow and also allow people to use it for CI/CD/MLops where the pipelines are run on schedule.

I am new to swagger and how the API works so please feel free to point out all possible improvements!

---

This change is 

[NikeNano]

Do I understand the failed test correctly when I assume that the failures are due to connectivity

run-sample-tests-loop(4:secret):	access-gcs-using-auth-token:	ERROR: (gsutil) timed out
run-sample-tests-loop(4:secret):	access-gcs-using-auth-token:	This may be due to network connectivity issues. Please check your network settings, and the status of the service you are trying to reach.

?

[NikeNano]

friendly ping @Ark-kun and @hongye-sun :)

[NikeNano]

/test kubeflow-pipeline-e2e-test

[NikeNano]

/test kubeflow-pipeline-sample-test

[NikeNano]

/assign @hongye-sun

[NikeNano]

/assign @Ark-kun

[Ark-kun]

/retest

[IronPan]

/test kubeflow-pipeline-sample-test

[Ark-kun]

/lgtm

[NikeNano]

Thanks for the review @Ark-kun!

[Ark-kun]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Ark-kun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• sdk/OWNERS [Ark-kun]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[NikeNano]

/retest

[parthmishra]

I haven't had the opportunity to try this yet, but from a quick glance it doesn't seem like this will support scheduling a specific version of a pipeline, right? I assume it should be possible to target a version like in the UI

[NikeNano]

@parthmishra Yes and no it targets based on Pipeline Id, which is not the same as name. However you can get the id based upon the name. I usually target based upon the pipeline id I didn't actually knew about versioning. I do versioning based on Github SHA (hash) which i include in the name. I can check it out and might be able to update with a new PR. How do you use the versioning?

[parthmishra]

@NikeNano I was talking about targeting a Pipeline Version resource. You could probably do it by creating an additional resource reference e.g.:

if version_id:
    pipline_yaml = client._pipelines_api.get_pipeline_version_template(version_id).template
    pipeline_obj = yaml.safe_load(pipeline_yaml)
    pipeline_json_string = json.loads(pipeline_obj)
    version_ref = kfp_server_api.models.ApiResourceReference(
        key=models.ApiResourceKey(
            type=models.ApiResourceType.PIPELINE_VERSION,
            id=version_id,
        ),
        relationship=models.ApiRelationship.CREATOR,
    )

One thing to note would be that the pipeline version would need to exist and therefore supplying a package wouldn't make sense. Perhaps it might be a good idea to create separate methods similar to create_run_from_func and create_run_from_package?

Edit: Also, I noticed that the ApiTrigger object isn't being created, e.g:

trigger = kfp_server_api.models.ApiTrigger(
    cron_schedule=api_cron_schedule.ApiCronSchedule(cron=cron_schedule),
    periodic_schedule=periodic_schedule
)

Does it work without it?

[omarzouk]

I agree with @parthmishra , the function did not work when I tried to use it for my CI/CD setup. I created this PR: 3490 to address those issues.

[NikeNano]

Ohh sorry I missed this completely! Cool that you did a update @omarzouk