Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

set golang's secure cipher suites as etcd's cipher suites #4253

Merged
merged 1 commit into from
Nov 20, 2023
Merged

set golang's secure cipher suites as etcd's cipher suites #4253

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions artifacts/deploy/karmada-etcd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,10 @@ spec:
- --trusted-ca-file=/etc/karmada/pki/etcd-ca.crt
- --data-dir=/var/lib/etcd
- --snapshot-count=10000
# Setting Golang's secure cipher suites as etcd's cipher suites.
# They are obtained by the return value of the function CipherSuites() under the go/src/crypto/tls/cipher_suites.go package.
# Consistent with the Preferred values of k8s’s default cipher suites.
- --cipher-suites=TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be great to have a comment to describe where these suites come from.

Copy link
Member

@yanfeng1992 yanfeng1992 Nov 17, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be great to have a comment to describe where these suites come from.

+1

Other parts, lgtm

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Referring to go/src/crypto/tls/cipher_suites.go, golang supports 17 security algorithms.

On this basis, k8s adds map mapping to two of the security algorithms. Finally, these 19 cipher suites were formed.

karmada/vendor/k8s.io/component-base/cli/flag/ciphersuites_flag.go

Lines 33 to 44 in 98e655f

func init() {
for _, suite := range tls.CipherSuites() {
ciphers[suite.Name] = suite.ID
}
// keep legacy names for backward compatibility
ciphers["TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305"] = tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
ciphers["TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305"] = tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
for _, suite := range tls.InsecureCipherSuites() {
insecureCiphers[suite.Name] = suite.ID
}
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean comment in the code, so that when people read the code they can get where these cipher suites come from.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it~, I'll comment in the code right away

volumes:
- hostPath:
path: /var/lib/karmada-etcd
Expand Down
4 changes: 4 additions & 0 deletions charts/karmada/templates/etcd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,10 @@ spec:
- --key-file=/etc/kubernetes/pki/etcd/karmada.key
- --trusted-ca-file=/etc/kubernetes/pki/etcd/server-ca.crt
- --data-dir=/var/lib/etcd
# Setting Golang's secure cipher suites as etcd's cipher suites.
# They are obtained by the return value of the function CipherSuites() under the go/src/crypto/tls/cipher_suites.go package.
# Consistent with the Preferred values of k8s’s default cipher suites.
- --cipher-suites=TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
volumes:
- name: etcd-cert
secret:
Expand Down
11 changes: 10 additions & 1 deletion operator/pkg/controlplane/etcd/etcd.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import (
kuberuntime "k8s.io/apimachinery/pkg/runtime"
clientset "k8s.io/client-go/kubernetes"
clientsetscheme "k8s.io/client-go/kubernetes/scheme"
"k8s.io/component-base/cli/flag"

operatorv1alpha1 "github.com/karmada-io/karmada/operator/pkg/apis/operator/v1alpha1"
"github.com/karmada-io/karmada/operator/pkg/constants"
Expand Down Expand Up @@ -47,7 +48,7 @@ func installKarmadaEtcd(client clientset.Interface, name, namespace string, cfg
etcdStatefulSetBytes, err := util.ParseTemplate(KarmadaEtcdStatefulSet, struct {
StatefulSetName, Namespace, Image, EtcdClientService string
CertsSecretName, EtcdPeerServiceName string
InitialCluster, EtcdDataVolumeName string
InitialCluster, EtcdDataVolumeName, EtcdCipherSuites string
Replicas, EtcdListenClientPort, EtcdListenPeerPort int32
}{
StatefulSetName: util.KarmadaEtcdName(name),
Expand All @@ -58,6 +59,7 @@ func installKarmadaEtcd(client clientset.Interface, name, namespace string, cfg
EtcdPeerServiceName: util.KarmadaEtcdName(name),
EtcdDataVolumeName: constants.EtcdDataVolumeName,
InitialCluster: strings.Join(initialClusters, ","),
EtcdCipherSuites: genEtcdCipherSuites(),
Replicas: *cfg.Replicas,
EtcdListenClientPort: constants.EtcdListenClientPort,
EtcdListenPeerPort: constants.EtcdListenPeerPort,
Expand Down Expand Up @@ -127,3 +129,10 @@ func createEtcdService(client clientset.Interface, name, namespace string) error

return nil
}

// Setting Golang's secure cipher suites as etcd's cipher suites.
// They are obtained by the return value of the function CipherSuites() under the go/src/crypto/tls/cipher_suites.go package.
// Consistent with the Preferred values of k8s’s default cipher suites.
func genEtcdCipherSuites() string {
return strings.Join(flag.PreferredTLSCipherNames(), ",")
}
3 changes: 2 additions & 1 deletion operator/pkg/controlplane/etcd/mainfests.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,8 @@ spec:
- --key-file=/etc/karmada/pki/etcd/etcd-server.key
- --data-dir=/var/lib/etcd
- --snapshot-count=10000
- --log-level=debug
- --log-level=debug=
Copy link
Member

@yanfeng1992 yanfeng1992 Nov 20, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does add = here have any effect? @zhzhuang-zju

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for finding this problem. I fixed it in PR #4286 . Can you help with review?

- --cipher-suites={{ .EtcdCipherSuites }}
env:
- name: KARMADA_ETCD_NAME
valueFrom:
Expand Down
17 changes: 15 additions & 2 deletions pkg/karmadactl/cmdinit/kubernetes/statefulset.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/component-base/cli/flag"
"k8s.io/utils/pointer"

"github.com/karmada-io/karmada/pkg/karmadactl/cmdinit/options"
Expand Down Expand Up @@ -35,8 +36,9 @@ const (

var (
// appLabels remove via Labels karmada StatefulSet Deployment
appLabels = map[string]string{"karmada.io/bootstrapping": "app-defaults"}
etcdLabels = map[string]string{"app": etcdStatefulSetAndServiceName}
appLabels = map[string]string{"karmada.io/bootstrapping": "app-defaults"}
etcdLabels = map[string]string{"app": etcdStatefulSetAndServiceName}
etcdCipherSuites = genEtcdCipherSuites()
)

func (i *CommandInitOption) etcdVolume() (*[]corev1.Volume, *corev1.PersistentVolumeClaim) {
Expand Down Expand Up @@ -141,6 +143,7 @@ listen-client-urls: https://${%s}:%v,http://127.0.0.1:%v
initial-advertise-peer-urls: http://${%s}:%v
advertise-client-urls: https://${%s}.%s.%s.svc.%s:%v
data-dir: %s
cipher-suites: %s

`,
etcdContainerConfigDataMountPath, etcdConfigName,
Expand All @@ -159,6 +162,7 @@ data-dir: %s
i.Namespace, i.HostClusterDomain,
etcdContainerClientPort,
etcdContainerDataVolumeMountPath,
etcdCipherSuites,
),
}

Expand Down Expand Up @@ -350,3 +354,12 @@ func (i *CommandInitOption) makeETCDStatefulSet() *appsv1.StatefulSet {

return etcd
}

// Setting Golang's secure cipher suites as etcd's cipher suites.
// They are obtained by the return value of the function CipherSuites() under the go/src/crypto/tls/cipher_suites.go package.
// Consistent with the Preferred values of k8s’s default cipher suites.
func genEtcdCipherSuites() string {
cipherSuites := strings.Join(flag.PreferredTLSCipherNames(), "\",\"")
cipherSuites = "[\"" + cipherSuites + "\"]"
return cipherSuites
}
Loading