Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: create serviceaccount token for v1.24 clusters #9546

Merged
merged 2 commits into from
Jun 8, 2022

Conversation

danielhelfand
Copy link
Contributor

@danielhelfand danielhelfand commented May 31, 2022

Closes #9422

This pull request allows users to add v1.24 Kubernetes clusters to be managed by Argo CD by creating a service account secret with a bearer token. In previous versions of Kubernetes, this secret was created when serviceaccounts were created, but was removed in order to discourage the use of long lived tokens.

Two approaches were explored to address this:

  1. Create the secret for serviceaccounts via Argo CD UX experiences
  2. Use the TokenRequest API provided by Kubernetes to request tokens with configurable expiration

The decision was made to go with option 1 above since using the tokenrequest api would require moving the InstallClusterManagerRBAC func behind an API endpoint. This would need to be done in order to persist a TokenManager that would be injected into the cluster server upon start up to manage tokens created by the token request api.

There may need to be a longer term discussion on adding this new endpoint/token manager to the cluster server so the current proposal would keep the use of secrets for storing bearer tokens for service accounts.

Due to k3s not having a stable 1.24 release available, testing for 1.24 clusters was conducted manually using kind.

Note on DCO:

If the DCO action in the integration test fails, one or more of your commits are not signed off. Please click on the Details link next to the DCO action for instructions on how to resolve this.

Checklist:

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
  • The title of the PR states what changed and the related issues number (used for the release note).
  • I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
  • I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
  • Does this PR require documentation updates?
  • I've updated documentation as required by this PR.
  • Optional. My organization is added to USERS.md.
  • I have signed off all my commits as required by DCO
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My build is green (troubleshooting builds).

@codecov
Copy link

codecov bot commented May 31, 2022

Codecov Report

Merging #9546 (4e1db7d) into master (68c7e7d) will increase coverage by 0.10%.
The diff coverage is 70.00%.

@@            Coverage Diff             @@
##           master    #9546      +/-   ##
==========================================
+ Coverage   45.79%   45.89%   +0.10%     
==========================================
  Files         222      222              
  Lines       26377    26458      +81     
==========================================
+ Hits        12079    12143      +64     
- Misses      12650    12658       +8     
- Partials     1648     1657       +9     
Impacted Files Coverage Δ
cmd/argocd/commands/admin/cluster.go 0.00% <0.00%> (ø)
cmd/argocd/commands/cluster.go 9.17% <0.00%> (ø)
util/clusterauth/clusterauth.go 68.32% <73.13%> (+1.65%) ⬆️
...licationset/generators/generator_spec_processor.go 51.66% <0.00%> (-5.91%) ⬇️
util/helm/helm.go 41.89% <0.00%> (-0.78%) ⬇️
util/io/path/resolved.go 76.92% <0.00%> (ø)
util/settings/settings.go 48.16% <0.00%> (ø)
...is/applicationset/v1alpha1/applicationset_types.go 34.69% <0.00%> (ø)
applicationset/generators/matrix.go 72.60% <0.00%> (+0.38%) ⬆️
applicationset/generators/merge.go 60.60% <0.00%> (+0.40%) ⬆️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 68c7e7d...4e1db7d. Read the comment docs.

@rishabh625
Copy link
Contributor

Hey @danielhelfand , Thanks for the awesome work

@rishabh625 rishabh625 added the cherry-pick/2.4 Candidate for cherry picking into the 2.4 release branch label May 31, 2022
@danielhelfand danielhelfand force-pushed the 9422 branch 14 times, most recently from 422080a to 3e2b3c1 Compare June 2, 2022 04:12
Copy link
Member

@crenshaw-dev crenshaw-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the nitpicks. :-)

I think the only substantial things are:

  1. timeouts for requests
  2. using patch instead of update for the sa

util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved
util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved
util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved
util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved
util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved
util/clusterauth/clusterauth.go Show resolved Hide resolved
util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved
util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved
util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved
util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved
@danielhelfand danielhelfand force-pushed the 9422 branch 2 times, most recently from 417dd70 to 97d3061 Compare June 4, 2022 17:45
util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved
util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved
util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved
util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved
util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved

if len(serviceAccount.Secrets) != 0 {
for _, s := range serviceAccount.Secrets {
existingSecret, err := clientset.CoreV1().Secrets(ns).Get(context.Background(), s.Name, metav1.GetOptions{})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about an outside-the-loop context with a 30s timeout and an inside-the-loop context with a 10s timeout?

If for some absurd reason there are 100 secrets, the outer context will keep us from waiting forever.

If there's high latency, inner context will make sure we give a nice amount of time to get the secret.

If there's high latency and 3+ secrets and the target secret isn't in the first 3... well that's just a bummer.

Type: corev1.SecretTypeServiceAccountToken,
}

secret, err = clientset.CoreV1().Secrets(ns).Create(context.Background(), secret, metav1.CreateOptions{})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How bout the ol' 10s. Maybe we need a constant. :-P

util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved
util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved
util/clusterauth/clusterauth.go Outdated Show resolved Hide resolved
@danielhelfand danielhelfand force-pushed the 9422 branch 2 times, most recently from e86cc95 to 88032a9 Compare June 5, 2022 16:06
Signed-off-by: Daniel Helfand <[email protected]>
Copy link
Collaborator

@leoluz leoluz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
Thank you Daniel!

Copy link
Member

@crenshaw-dev crenshaw-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested adding a 1.24 docker-desktop cluster from a 1.20 minikube cluster. lgtm. Thanks so much!

@crenshaw-dev crenshaw-dev enabled auto-merge (squash) June 8, 2022 15:20
@crenshaw-dev crenshaw-dev merged commit 67bc5ee into argoproj:master Jun 8, 2022
crenshaw-dev pushed a commit that referenced this pull request Jun 8, 2022
* fix: create serviceaccount token for v1.24 clusters

Signed-off-by: Daniel Helfand <[email protected]>

* change create to get in err

Signed-off-by: Daniel Helfand <[email protected]>
@crenshaw-dev
Copy link
Member

Cherry-picked onto 2.4.

@rishabh625
Copy link
Contributor

Hi @crenshaw-dev will this be backported to 2.3 too?

@crenshaw-dev
Copy link
Member

Good question... I'm not opposed to it, if anyone needs it. Want to ask in #argo-contributors on CNCF Slack?

@rishabh625
Copy link
Contributor

Yeah sure, we'll need it

@crenshaw-dev crenshaw-dev added the cherry-pick/2.3 Candidate for cherry picking into the 2.3 release branch label Jul 13, 2022
crenshaw-dev pushed a commit that referenced this pull request Jul 13, 2022
* fix: create serviceaccount token for v1.24 clusters

Signed-off-by: Daniel Helfand <[email protected]>

* change create to get in err

Signed-off-by: Daniel Helfand <[email protected]>
@crenshaw-dev
Copy link
Member

Cherry-picked onto release-2.3 for 2.3.7.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cherry-pick/2.3 Candidate for cherry picking into the 2.3 release branch cherry-pick/2.4 Candidate for cherry picking into the 2.4 release branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unable to add 1.24.0 Kubernetes cluster
4 participants