Refactor bsc#1155810

[jenting]

Why is this PR needed?

Address the comments of this PR

• Kubelet use cluster-wide root CA, not per-node (bsc#1155810) #832 (comment)
• Kubelet use cluster-wide root CA, not per-node (bsc#1155810) #832 (comment)
• Kubelet use cluster-wide root CA, not per-node (bsc#1155810) #832 (comment)
• Kubelet use cluster-wide root CA, not per-node (bsc#1155810) #832 (comment)

What does this PR do?

1. Moves create root CA certificate and key to kubernetes package.
2. Splits state into two: upload ca cert, generate server cert/key and upload.

Anything else a reviewer needs to know?

N/A

Info for QA

Please double verify on all the platforms: OpenStack, VMWare, Bare Metal.

Related info

#832

Status BEFORE applying the patch

N/A

Status AFTER applying the patch

N/A

Docs

N/A

Merge restrictions

(Please do not edit this)

We are in v4-maintenance phase, so we will restrict what can be merged to prevent unexpected surprises:

What can be merged (merge criteria):
    2 approvals:
        1 developer: code is fine
        1 QA: QA is fine
    there is a PR for updating documentation (or a statement that this is not needed)

Signed-off-by: JenTing Hsiao [email protected]

[ereslibre]

Thank you @jenting, two minor comments and it's a LGTM.

[maximenoel8]

QA information :

Building skuba :

• git pull on pull request 853
• make ( for devel env )
  I'm using devel to be able to upgrade kubelet from 1.15.2 to 1.16.2

Testing environment :

• 1 lb / 1 master / 1 node on vmware

Cluster setting :

skuba cluster init --control-plane <IP_LB> --strict-capability-defaults --kubernetes-version 1.15.2 cluster

I'm using the new option --kubernetes-version to be able to upgrade once the cluster is deployed

Scenario:

I want to upgrade my kubelet version and crio version ( kubelet: 1.15.2 -> 1.16.2 / crio: 1.15.0 -> 1.16.0) using the upgrade method.

Before :

• Checking if the cluster need an upgrade

Command : skuba cluster upgrade plan
Result :

** This is an UNTAGGED version and NOT intended for production usage. **
Current Kubernetes cluster version: 1.15.2
Latest Kubernetes version: 1.16.2

Upgrade path to update from 1.15.2 to 1.16.2:
 - 1.15.2 -> 1.16.2

Addons for next cluster version 1.16.2 are already up to date.

There is no need to run 'skuba addon upgrade apply' after you have completed the platform upgrade.

• Checking master01 need an upgrade
  Command : skuba node upgrade plan master01
  Result :

** This is an UNTAGGED version and NOT intended for production usage. **
Current Kubernetes cluster version: 1.15.2
Latest Kubernetes version: 1.16.2

Current Node version: 1.15.2

Component versions in master01
  - apiserver: 1.15.2 -> 1.16.2
  - controller-manager: 1.15.2 -> 1.16.2
  - scheduler: 1.15.2 -> 1.16.2
  - etcd: 3.3.11 -> 3.3.15
  - kubelet: 1.15.2 -> 1.16.2
  - cri-o: 1.15.0 -> 1.16.0

• Checking worker01 need an upgrade

Test:

Part 1 - Upgrade master01

• Upgrade master01
• Check master01 DON'T need an upgrade
• Check current version for kubelet is 1.16.2 with skuba command
• Check current version for crio is 1.16.0 with skuba command
• Check current version for kubelet is 1.16.2 directly on the node
• Check current version for crio is 1.16.0 directly on the node
• Check skuba cluster upgrade plan command return (*)
  Command : skuba cluster upgrade plan
  Result :

** This is an UNTAGGED version and NOT intended for production usage. **
Current Kubernetes cluster version: 1.16.2
Latest Kubernetes version: 1.16.2

Congratulations! You are already at the latest version available

Part 2 - Upgrade worker01

• Upgrade worker01
• Check worker01 DON'T need an upgrade
• Check current version for kubelet is 1.16.2 with skuba command
• Check current version for crio is 1.16.0 with skuba command
• Check current version for kubelet is 1.16.2 directly on the node
• Check current version for crio is 1.16.0 directly on the node
• Check skuba cluster upgrade plan command return

Comments

The test is PASS for me. I was able to correctly upgrade my cluster.
The only things that doesn't seem correct is the step Check skuba cluster upgrade plan command return in part 1 (*)
Command : skuba cluster upgrade plan
Result :

** This is an UNTAGGED version and NOT intended for production usage. **
Current Kubernetes cluster version: 1.16.2
Latest Kubernetes version: 1.16.2

Congratulations! You are already at the latest version available

The worker is not upgraded yet but the cluster upgrade plan doesn't show me that. It can be confusing during an upgrade.

[jenting]

Thanks for your testing @maximenoel8.

The test steps test the CaaS platform upgrade but it does not contain checking the bug symptom that all the master and worker node kubelet server certificates signed by the same CA certificate.

You could follow the PR #832 test steps (Status BEFORE/AFTER applying the patch), or check all nodes kubelet server certificates are signed by the same CA certificate.

# ssh into the node
ssh sles@<ip-address>
# check certificate
openssl s_client -connect localhost:10250 -CAfile /var/lib/kubelet/pki/kubelet-ca.crt <<< "Q"

[ereslibre]

LGTM, thanks @jenting.

[maximenoel8]

Check the certificate by using the command :

openssl s_client -connect localhost:10250 -CAfile kubelet-ca.crt <<< "Q" 

on each node.

CN is correctly equal to kubelet-ca and I can identify myself by using kubelet-ca.crt to 10250