[feature request] yurtadm join option to specify secret for private registry

[fujitatomoya]

What would you like to be added:

--registry-secret option to specify secret to be filled in /etc/kubernetes/manifests/yurthub.yaml so that Kuberenetes can pull the image from private registry with this secret.

yurtadm join --yurthub-image "some.cloud.private.registry.com/openyurt/yurthub:latest" --registry-secret "secrettobewritteninmanifest"

Why is this needed:

It is likely that user want to guarantee all the container images controlled by them self for production phase from their own registry.
Currently we can only specify the registry but secret. This does not allow the user to pull the image from secured private registry.

others
/kind feature

[fujitatomoya]

Implementation Consideration

• extend addJoinConfigFlags with --registry-secret, in default None.
• YurtJoinData interface to be extended with RegistrySecret.
• If the secret is specified with yurtadm join --registry-secret, fill that into static yaml file yurthub.yaml
• (the rest will be taken care by kubernetes to pull the image from private registry with specified secret)

[fujitatomoya]

off topic from this feature request, but yurthub:latest could be the reason for possible problem once image is pushed or updated w/o notice... I saw these issues many times.

[YTGhost]

Implementation Consideration

• extend addJoinConfigFlags with --registry-secret, in default None.
• YurtJoinData interface to be extended with RegistrySecret.
• If the secret is specified with yurtadm join --registry-secret, fill that into static yaml file yurthub.yaml
• (the rest will be taken care by kubernetes to pull the image from private registry with specified secret)

I think this requirement is necessary, we need to add a --registry-secret flag and fill the value into imagePullSecrets in yurthub.yaml.

[fujitatomoya]

yeah, i mean it is not hard-requirement right away, but better to have this option.

[YTGhost]

off topic from this feature request, but yurthub:latest could be the reason for possible problem once image is pushed or updated w/o notice... I saw these issues many times.

Maybe we should set the default value image version of yurthubImage to follow the yurtadm version? WDYT @rambohe-ch

[rambohe-ch]

@fujitatomoya @YTGhost As we have discussed in the issue #1079 , a new CRD named StaticPod will be added, and yurtadm join will use this CR template to install YurtHub static pod instead of hard code of YurtHub in yurtadm, the improvement of yurtadm is covered in the issue #1080

The above features will be released in OpenYurt v1.3, and based on these features, users can define the YurtHub StaticPod template as their requirements. i think these features can satisfy the requirements of @fujitatomoya .

[fujitatomoya]

@rambohe-ch thanks for the information, that looks good to me. I would do review to see the details, appreciate the contribution and feature enhancement!

[fujitatomoya]

@rambohe-ch

having StaticPod template makes sense to me, i believe that it should be the way it it to align kubernetes CRD.

AFAIU, user still needs to issue yurtadm join after this feature is integrated, right? in that case, the question is how user can specify the secret? besides, StaticPod spec does not allow to specify the secret? (https://kubernetes.io/docs/tasks/configure-pod-container/static-pod/)

Note: The spec of a static Pod cannot refer to other API objects (e.g., ServiceAccount, ConfigMap, Secret, etc).

[rambohe-ch]

@rambohe-ch

having StaticPod template makes sense to me, i believe that it should be the way it it to align kubernetes CRD.

AFAIU, user still needs to issue yurtadm join after this feature is integrated, right? in that case, the question is how user can specify the secret? besides, StaticPod spec does not allow to specify the secret? (https://kubernetes.io/docs/tasks/configure-pod-container/static-pod/)

Note: The spec of a static Pod cannot refer to other API objects (e.g., ServiceAccount, ConfigMap, Secret, etc).

@fujitatomoya sorry, my response had confused you, and i had made a mistake that static pod can not support imagePullSecrets field.

but i think yurtadm join can not manage to help static pod to support imagePullSecrets because this secret can not be passed through to kubelet component.

and for private registry, i suggested that user should configure the runtime to support it. like $HOME/.docker/config.json for docker runtime.

[fujitatomoya]

but i think yurtadm join can not manage to help static pod to support imagePullSecrets because this secret can not be passed through to kubelet component.

yeah, agree. I believe that StaticPods are meant to be controlled by only kubelet but high level controller of kubernetes. that is said that it cannot bind or control the secret from k8s perspective.

and for private registry, i suggested that user should configure the runtime to support it. like $HOME/.docker/config.json for docker runtime.

it is doable, setting the appropriate secret for private registry to container runtime address this issue. i think this needs to be well-documented since user might meet the same situation like where to set the secret?. and probably passing secret via command line is not really good idea for security?

[rambohe-ch]

but i think yurtadm join can not manage to help static pod to support imagePullSecrets because this secret can not be passed through to kubelet component.

yeah, agree. I believe that StaticPods are meant to be controlled by only kubelet but high level controller of kubernetes. that is said that it cannot bind or control the secret from k8s perspective.

and for private registry, i suggested that user should configure the runtime to support it. like $HOME/.docker/config.json for docker runtime.

it is doable, setting the appropriate secret for private registry to container runtime address this issue. i think this needs to be well-documented since user might meet the same situation like where to set the secret?. and probably passing secret via command line is not really good idea for security?

@fujitatomoya I will add the solution of using private registry for YurtHub component(static pod) in faq doc.
and i agree that user should configure the private registry for runtime manually.

/assign @rambohe-ch

[rambohe-ch]

@fujitatomoya I have updated yurthub faq doc, and added a item: 【12. How to use the yurthub image of the private image registry】
https://openyurt.io/docs/faq/yurthub/

@fujitatomoya please have a check whether your problem has been solved or not, if so, please close this issue.

[fujitatomoya]

12. How to use the yurthub image of the private image registry makes sense to me, thanks!