feat(sdk/backend): enable parameterization of container images

[HumairAK]

Resolves #11391

Description of your changes:
This change allows component base images to be parameterized using runtime pipeline parameters. The container images can be specified within an @pipeline decorated function, and takes precedence over the @component(base_image=..) argument.

This change also adds logic to resolve these runtime parameters in the argo driver logic. It also includes resolution steps for resolving the accelerator type which functions the same way but was missing the resolution logic. The resolution logic is a generic workaround solution for any run time pod spec input parameters that cannot be resolved because they cannot be added dynamically in the argo pod spec container template.

Checklist:

• You have signed off your commits
• The title for your pull request (PR) should follow our title convention. Learn more about the pull request title convention used in this repository.

[jgarciao]

One question: if the user sets the image with set_container_image, is the cache key using that setting?

[HumairAK]

One question: if the user sets the image with set_container_image, is the cache key using that setting?

this is a good point, I think we may want to resolve the container image before creating the fingerprint

[javali-google]

Nit: Is the documentation updated as part of different PR?

[HumairAK]

this is a good point, I think we may want to resolve the container image before creating the fingerprint

Actually we don't need to cover this as it's already covered via the inputs field. Since fingerprint is also generated based on input, every image change via runtime parameter input will result in a new cache, which is what we want.

Example:

For this sample pipeline:

@dsl.component(base_image="docker.io/python:3.9.17")
def empty_component(image: str, otherinput: str):
    print("image=" + image + ", otherinput=" + otherinput)

@dsl.pipeline()
def pipeline_accel(img: str, otherinput: str):
    task = empty_component(image=img, otherinput=otherinput)
    task.set_container_image(img)

We have:

[HumairAK]

@javali-google I belive the docs are auto generated and served via readthedocs

@chensun can confirm

my expectation is the docstring here will be auto surfaced like the other task fields here

[jgarciao]

Actually we don't need to cover this as it's already covered via the inputs field. Since fingerprint is also generated based on input, every image change via runtime parameter input will result in a new cache, which is what we want ...

Thanks for the detailed analysis, @HumairAK !

[jgarciao]

lgtm

[HumairAK]

cc @chensun PTAL when you get the chance it would be good to be able to get this in for 2.11

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jgarciao, leseb
Once this PR has been reviewed and has the lgtm label, please ask for approval from humairak. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• backend/OWNERS
• sdk/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment