Constructing many clients error

[arukaen]

Hello all, recently we've enabled aws-iam-authenticator and I'm seeing an error message whenever I run anything that uses --all-namespaces (i.e. kubectl --context cluster get pods --all-namespaces).

The error message in question is:

W0407 12:20:16.744257   32782 exec.go:203] constructing many client instances from the same exec auth config can cause performance problems during cert rotation and can exhaust available network connections; 1001 clients constructed calling "aws-iam-authenticator"

kubectl version v1.18.0
aws-iam-auth version: {"Version":"v0.5.0","Commit":"1cfe2a90f68381eacd7b6dcfa2bf689e76eb8b4b"}
OSX: 10.14.6

[micahhausler]

This is actually a warning message, and is mainly targeted for resource usage when using client certificate auth. Kubectl/client-go will only invoke aws-iam-authenticator once and reuse the token for multiple connections.

[arukaen]

Is there a way to turn off this message?

[micahhausler]

You might be able to set --stderrthreshold=ERROR (see klog), but if that doesn't work you'd have to file an issue on kubernetes.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[max-rocket-internet]

I'm getting about 2000 lines of this error also when running kubectl today. There is no --stderrthreshold option for aws-iam-authenticator. Is there any other fix?

[max-rocket-internet]

My errors looked like this:

W0902 17:12:19.796919   25579 exec.go:203] constructing many client instances from the same exec auth config can cause performance problems during cert rotation and can exhaust available network connections; 1384 clients constructed calling "aws-iam-authenticator"

So I just filtered it out with grep, but this is quite hacky:

$ kubectl describe pods 2>&1 | grep -v W0902

[sysadmiral]

This warning appears continuously when running https://github.com/vmware-tanzu/octant

[dunjoye4real]

@sysadmiral I am getting the same warning using octant. Know of any work around?

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[HeavensRegent]

/reopen
This is still an issue Nothing has been resolved.

[k8s-ci-robot]

@HeavensRegent: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen
This is still an issue Nothing has been resolved.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[philoserf]

related kubernetes/kubernetes#91913

[ajcann]

Also getting this and commented on #91913 with more information.

[mooreniemi]

Seeing this (slightly different message so adding for Google) when using aws to load data from S3 in initContainer:

connections; 1377 clients constructed calling "aws"
W1218 16:36:11.240927 1982077 exec.go:271] constructing many client instances from the same exec auth config can cause performance problems during cert rotation and can exhaust available network
connections; 1378 clients constructed calling "aws"
W1218 16:36:11.240941 1982077 exec.go:271] constructing many client instances from the same exec auth config can cause performance problems during cert rotation and can exhaust available network
connections; 1379 clients constructed calling "aws"

[irl-segfault]

I get this when using octant, just FYI. Wondering if there's a way to force reuse of the same client.

[TBBle]

Per kubernetes/kubernetes#91913 (comment) it should be resolved by updating the version of k8s.io/client-go being used, which was done for aws-iam-authenticator in #398, released in v0.5.4.

Note that this is a high hit in a search for this error message, but other tools use k8s.io/client-go (or just call-out to kubectl) as well, so make sure you're looking at the right project.