volcano-admission-init error is reported when the volcano is uninstalled and reinstalled or the helm is used to upgrade the volcano

[wangyang0616]

What happened:
After the Volcano is installed, if you uninstall and reinstall the Volcano or use the helm upgrade tool to upgrade the Volcano, the volcano-admission-init-k6kct fails. The symptoms are as follows:

volcano-system   volcano-admission-66775d9c56-n2hkr         1/1     Running   0          19s
volcano-system   volcano-admission-init-7pg2h               0/1     Error     0          40s
volcano-system   volcano-admission-init-k6kct               0/1     Error     0          52s
volcano-system   volcano-admission-init-qj4d7               0/1     Error     0          44s
volcano-system   volcano-admission-init-x8ck2               0/1     Error     0          48s
volcano-system   volcano-controllers-dd4b9d857-2scwk        1/1     Running   0          19s
volcano-system   volcano-scheduler-5bd68f8f8-l6knh          1/1     Running   0          19s

Error log information of volcano-admission-init：

creating certs in dir /tmp 
Certificate request self-signature ok
subject=CN = volcano-admission-service.volcano-system.svc
error: failed to create secret secrets "volcano-admission-secret" already exists

Helm upgrade error information：

Error: cannot patch "volcano-admission-init" with kind Job: Job.batch "volcano-admission-init" is invalid: spec.template: Invalid value: core.PodTemplateSpec{ObjectMeta:v1.ObjectMeta{Name:"", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"controller-uid":"d83dd0f8-c23b-4987-8aa9-9a05900e4a71", "job-name":"volcano-admission-init"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:core.PodSpec{Volumes:[]core.Volume(nil), InitContainers:[]core.Container(nil), Containers:[]core.Container{core.Container{Name:"main", Image:"volcanosh/vc-webhook-manager:v1.6.0", Command:[]string{"./gen-admission-secret.sh", "--service", "volcano-admission-service", "--namespace", "volcano-system", "--secret", "volcano-admission-secret"}, Args:[]string(nil), WorkingDir:"", Ports:[]core.ContainerPort(nil), EnvFrom:[]core.EnvFromSource(nil), Env:[]core.EnvVar(nil), Resources:core.ResourceRequirements{Limits:core.ResourceList(nil), Requests:core.ResourceList(nil)}, VolumeMounts:[]core.VolumeMount(nil), VolumeDevices:[]core.VolumeDevice(nil), LivenessProbe:(*core.Probe)(nil), ReadinessProbe:(*core.Probe)(nil), StartupProbe:(*core.Probe)(nil), Lifecycle:(*core.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"IfNotPresent", SecurityContext:(*core.SecurityContext)(nil), Stdin:false, StdinOnce:false, TTY:false}}, EphemeralContainers:[]core.EphemeralContainer(nil), RestartPolicy:"Never", TerminationGracePeriodSeconds:(*int64)(0xc018843b70), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"ClusterFirst", NodeSelector:map[string]string(nil), ServiceAccountName:"volcano-admission", AutomountServiceAccountToken:(*bool)(nil), NodeName:"", SecurityContext:(*core.PodSecurityContext)(0xc004395680), ImagePullSecrets:[]core.LocalObjectReference(nil), Hostname:"", Subdomain:"", SetHostnameAsFQDN:(*bool)(nil), Affinity:(*core.Affinity)(nil), SchedulerName:"default-scheduler", Tolerations:[]core.Toleration(nil), HostAliases:[]core.HostAlias(nil), PriorityClassName:"system-cluster-critical", Priority:(*int32)(nil), PreemptionPolicy:(*core.PreemptionPolicy)(nil), DNSConfig:(*core.PodDNSConfig)(nil), ReadinessGates:[]core.PodReadinessGate(nil), RuntimeClassName:(*string)(nil), Overhead:core.ResourceList(nil), EnableServiceLinks:(*bool)(nil), TopologySpreadConstraints:[]core.TopologySpreadConstraint(nil), OS:(*core.PodOS)(nil)}}: field is immutable

During Volcano installation, a secret is created using a job. When Volcano is uninstalled, the original secret is not cleared. When Volcano is installed or upgraded again, an error is reported when volcano-admission-init creates a secret with the same name.

volcano/installer/dockerfile/webhook-manager/gen-admission-secret.sh

Lines 93 to 100 in ff3ff9e

	function createSecret() {
	# create the secret with CA cert and server cert/key
	kubectl create secret generic ${SECRET} \
	--from-file=tls.key=${CERTDIR}/server.key \
	--from-file=tls.crt=${CERTDIR}/server.crt \
	--from-file=ca.crt=${CERTDIR}/ca.crt \
	-n ${NAMESPACE}
	}

What you expected to happen:
When the volcano-admission-init command is used to create a secret, if the secret with the same name already exists, the system does not create a secret and returns a success message.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

• Volcano Version: master, v1.6.0, v1.7.0
• Kubernetes version (use kubectl version):
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release):
• Kernel (e.g. uname -a):
• Install tools:
• Others:

[wangyang0616]

/assign

[hwdef]

I think this secret should also be deleted when uninstalling the volcano

[xiao-jay]

I agree.

[xiao-jay]

@wangyang0616 Could you please tell me how to uninstall volcano?

[hwdef]

The secret will be deleted when the volcano is uninstalled, because the volcano-system namespace is deleted when the volcano is uninstalled.
this issue may be caused by the redeployment of the volcano-admission-init job for some reasons

[hwdef]

@wangyang0616 Could you please tell me how to uninstall volcano?

kubectl delete -f https://raw.githubusercontent.com/volcano-sh/volcano/master/installer/volcano-development.yaml