Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop storing stateless kubernetes keystores #21880

Merged
merged 8 commits into from
Oct 19, 2020
Merged

Stop storing stateless kubernetes keystores #21880

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
e5abbda
Protect keystore map with mutex
ChrsMark Oct 16, 2020
ef269ab
Add changelog
ChrsMark Oct 16, 2020
21d61b1
lock once on read/write level
ChrsMark Oct 16, 2020
c811fa8
Make use of LoadOrStore
ChrsMark Oct 16, 2020
f0a8cd0
review comments
ChrsMark Oct 19, 2020
8af93be
fix changelog
ChrsMark Oct 19, 2020
3a03732
fix leftover comment
ChrsMark Oct 19, 2020
69a42c6
Update CHANGELOG.next.asciidoc
ChrsMark Oct 19, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -373,6 +373,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Fix retrieving resources by ID for the azure module. {pull}21711[21711] {issue}21707[21707]
- Use timestamp from CloudWatch API when creating events. {pull}21498[21498]
- Report the correct windows events for system/filesystem {pull}21758[21758]
- Protect kubernetes keystore map with mutex {pull}21880[21880]

*Packetbeat*

Expand Down
15 changes: 7 additions & 8 deletions libbeat/common/kubernetes/k8skeystore/kubernetes_keystore.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ package k8skeystore
import (
"context"
"strings"
"sync"

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
k8s "k8s.io/client-go/kubernetes"
Expand All @@ -30,12 +31,13 @@ import (
"github.com/elastic/beats/v7/libbeat/logp"
)

type KubernetesKeystores map[string]keystore.Keystore
//type KubernetesKeystores map[string]keystore.Keystore
ChrsMark marked this conversation as resolved.
Show resolved Hide resolved
type KubernetesKeystores sync.Map
ChrsMark marked this conversation as resolved.
Show resolved Hide resolved

// KubernetesKeystoresRegistry holds KubernetesKeystores for known namespaces. Once a Keystore for one k8s namespace
// is initialized it will be reused every time it is needed.
type KubernetesKeystoresRegistry struct {
kubernetesKeystores KubernetesKeystores
kubernetesKeystores sync.Map
ChrsMark marked this conversation as resolved.
Show resolved Hide resolved
logger *logp.Logger
client k8s.Interface
}
Expand All @@ -56,7 +58,7 @@ func Factoryk8s(keystoreNamespace string, ks8client k8s.Interface, logger *logp.
// NewKubernetesKeystoresRegistry initializes a KubernetesKeystoresRegistry
func NewKubernetesKeystoresRegistry(logger *logp.Logger, client k8s.Interface) keystore.Provider {
return &KubernetesKeystoresRegistry{
kubernetesKeystores: KubernetesKeystores{},
kubernetesKeystores: sync.Map{},
logger: logger,
client: client,
}
Expand All @@ -76,12 +78,9 @@ func (kr *KubernetesKeystoresRegistry) GetKeystore(event bus.Event) keystore.Key
}
if namespace != "" {
// either retrieve already stored keystore or create a new one for the namespace
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is not needed anymore.

Suggested change
// either retrieve already stored keystore or create a new one for the namespace

if storedKeystore, ok := kr.kubernetesKeystores[namespace]; ok {
return storedKeystore
}
k8sKeystore, _ := Factoryk8s(namespace, kr.client, kr.logger)
kr.kubernetesKeystores["namespace"] = k8sKeystore
return k8sKeystore
storedKeystore, _ := kr.kubernetesKeystores.LoadOrStore(namespace, k8sKeystore)
return storedKeystore.(keystore.Keystore)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This way we are instantiating a new keystore on every call, this is why I mentioned that we could need to store an object that creates the keystore lazily.

But, I have been looking at the implementation of the k8s keystore and it actually doesn't keep any state at all, so maybe we can return a new one each time and don't keep a registry?

Suggested change
storedKeystore, _ := kr.kubernetesKeystores.LoadOrStore(namespace, k8sKeystore)
return storedKeystore.(keystore.Keystore)
return k8sKeystore

}
kr.logger.Debugf("Cannot retrieve kubernetes namespace from event: %s", event)
return nil
Expand Down