'local-path-provisioner-role' ClusterRole missing from k3s-v0.10.0 installation

[ntx-ben]

Version:
v0.10.0

Describe the bug
Doing a fresh install (--docker) of k3s v0.10.0 results in 'local-path-provisioner' pod failing to load due to missing 'local-path-provisioner-role':

ERROR: logging before flag.Parse: E1025 20:21:52.836721 1 leaderelection.go:252] error retrieving resource lock kube-system/rancher.io-local-path: endpoints "rancher.io-local-path" is forbidden: User "system:serviceaccoun ││ t:kube-system:local-path-provisioner-service-account" cannot get resource "endpoints" in API group "" in the namespace "kube-system": RBAC: clusterrole.rbac.authorization.k8s.io "local-path-provisioner-role" not found

Getting all ClusterRoles in 'kube-system':

kubectl --kubeconfig rancher-k3s.yaml get clusterroles -n kube-system
NAME AGE
cluster-admin 11m
system:discovery 11m
system:basic-user 11m
system:public-info-viewer 11m
system:aggregate-to-admin 11m
system:aggregate-to-edit 11m
system:aggregate-to-view 11m
system:heapster 11m
system:node 11m
system:node-problem-detector 11m
system:kubelet-api-admin 11m
system:node-bootstrapper 11m
system:auth-delegator 11m
system:kube-aggregator 11m
system:kube-controller-manager 11m
system:kube-dns 11m
system:persistent-volume-provisioner 11m
system:csi-external-attacher 11m
system:certificates.k8s.io:certificatesigningrequests:nodeclient 11m
system:certificates.k8s.io:certificatesigningrequests:selfnodeclient 11m
system:volume-scheduler 11m
system:node-proxier 11m
system:kube-scheduler 11m
system:csi-external-provisioner 11m
system:controller:attachdetach-controller 11m
system:controller:clusterrole-aggregation-controller 11m
system:controller:cronjob-controller 11m
system:controller:daemon-set-controller 11m
system:controller:deployment-controller 11m
system:controller:disruption-controller 11m
system:controller:endpoint-controller 11m
system:controller:expand-controller 11m
system:controller:generic-garbage-collector 11m
system:controller:horizontal-pod-autoscaler 11m
system:controller:job-controller 11m
system:controller:namespace-controller 11m
system:controller:node-controller 11m
system:controller:persistent-volume-binder 11m
system:controller:pod-garbage-collector 11m
system:controller:replicaset-controller 11m
system:controller:replication-controller 11m
system:controller:resourcequota-controller 11m
system:controller:route-controller 11m
system:controller:service-account-controller 11m
system:controller:service-controller 11m
system:controller:statefulset-controller 11m
system:controller:ttl-controller 11m
system:controller:certificate-controller 11m
system:controller:pvc-protection-controller 11m
system:controller:pv-protection-controller 11m
system:coredns 11m
cloud-controller-manager 11m
nginx-ingress-clusterrole 11m
kubernetes-dashboard 11m
system:aggregated-metrics-reader 11m
system:metrics-server 11m
view 11m
edit 11m
admin 11m

To Reproduce
Install k3s v0.10.0 and get status of 'local-path-provisoner' pod in 'kube-system' namespace.

Expected behavior
'local-path-provisoner' pod loading successfully.

Actual behavior
'local-path-provisoner' pod not loading successfully.

[matlec]

Could this be related to my bugfix in wrangler? The manifest file delivered with k3s still includes namespaces for cluster-scoped objects. If the wrangler fix has not been pulled in, the issue you describe may still be present.

[ntx-ben]

Thanks for the feedback.

Not sure which deployment file is being used, but looking at the one bundled with k3s, the 'local-path-provisioner-role' is scoped to the 'kube-system'.

[matlec]

You could try to remove the namespace from the ClusterRole and restart k3s.

[ntx-ben]

Yes that did the trick.

[erikwilson]

Thanks for the info @matlec, I was able to duplicate by re-installing/modifying the k3s service to run with the --default-local-storage-path flag, which triggers the issue. Fixed the issue in the manifest by removing the namespace for the clusterrole, will be released with v0.10.1 here shortly.

[ShylajaDevadiga]

Related issue #884
With v0.10.1 updating to use --default-local-storage-path /tmp/k3s resulted in the pvc and 'local-path-provisoner' pod loading successfully.

Closing issue. Please feel free to re-open if you see any fails. Thanks.