Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VAULT-15547 First pass at agent/proxy decoupling #20548

Merged
merged 15 commits into from
May 17, 2023

Conversation

VioletHynes
Copy link
Contributor

@VioletHynes VioletHynes commented May 8, 2023

Sorry this is such a big PR. It didn't make sense to piecemeal some of this stuff and I wouldn't have felt good adding untested code (e.g. if I saved testing for later PR). I've still made an effort to keep this relatively contained, with more to come later for hopefully easier and better review.

Most of the code changes in this PR specific to the proxy command is copied from Agent, with template-related bits deleted. Hopefully, a lot of the code is familiar and therefore less daunting to review.

While the diff is large, a lot of it is just moving some of the shared things from the agent folder to a new, shared folder, agentproxyshared. While still large, the less-large salient subset of these changes are contained within the following files:

  • command/proxy.go
  • command/proxy_test.go
  • command/proxy/*

There's still more to come, after this PR, but I wanted to keep this functional, but as small as possible while remaining fully functional (since it's still really big). What's coming in later PRs:

  • I want to refactor the shared code even more, to have less duplication
  • Warnings for an Agent that's using the API proxy
  • There's still more to be added on the test side for proxy, I just started with a simple set that would validate my changes
  • Tests for the user agent functions
  • Docs will be a separate PR/Jira that I can work on in parallel to this review

@VioletHynes VioletHynes added this to the 1.14 milestone May 8, 2023
@VioletHynes VioletHynes marked this pull request as ready for review May 12, 2023 13:08
@VioletHynes VioletHynes requested a review from a team as a code owner May 12, 2023 13:08

// Internal-only flags to follow.
//
// Why hello there little source code reader! Welcome to the Vault source
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello to you too!

Copy link
Contributor

@marcboudreau marcboudreau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For comments structured as a question, it could be explained away or maybe I wasn't sure enough to explicitly call for a change. I remember leaving a couple of suggestions to use testing.T.Setenv() instead of os.Setenv. Those are just a suggestion to consider, I'll approve one way or the other on that.

command/proxy.go Outdated

serverHealth, err := client.Sys().Health()
if err == nil {
// We don't exit on error here, as this is not worth stopping Proxy over
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is confusing to me. The use of on error here makes me think that it relates to the check on the above line, but then that doesn't make any sense. Should the comment read something like We don't exit if the versions don't match...?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I can see how this can be confusing. Let me reword, and thank you for the suggestion! I'll steal your wording :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated this, hopefully it's a bit clearer!

command/proxy.go Show resolved Hide resolved
command/proxy.go Show resolved Hide resolved
command/proxy.go Show resolved Hide resolved
command/proxy_test.go Show resolved Hide resolved

// Unset the environment variable so that proxy picks up the right test
// cluster address
defer os.Setenv(api.EnvVaultAddress, os.Getenv(api.EnvVaultAddress))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you considered using https://pkg.go.dev/testing#T.Setenv here instead? It essentially does the same thing but in a single tidy function call.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure it does do the same? What we're doing here is deferring setting the environment variable until after the test has finished, essentially so that the proxy command doesn't pick up the environment variable, but so that we set it back afterwards so we don't mess with the CI environment, for example.

I think that t.Setenv does a set then a defer unset, which is the opposite of what we'd need

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation indicates that it does a set of the environment variable and once the test is finished, it restores the environment variable to the previous value. What I was thinking of was something like t.Setenv(api.EnvVaultAddress, ""). So that during the test execution, the environment variable would be set to an empty string. But like I said, that's merely a suggestion, no need to pursue this thread.

command/proxy_test.go Show resolved Hide resolved
command/proxy_test.go Outdated Show resolved Hide resolved
command/proxy_test.go Outdated Show resolved Hide resolved
Copy link
Contributor

@marcboudreau marcboudreau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For comments structured as a question, it could be explained away or maybe I wasn't sure enough to explicitly call for a change. I remember leaving a couple of suggestions to use testing.T.Setenv() instead of os.Setenv. Those are just a suggestion to consider, I'll approve one way or the other on that.

@VioletHynes VioletHynes requested a review from marcboudreau May 15, 2023 18:49
Copy link
Contributor

@marcboudreau marcboudreau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

Copy link

@peteski22 peteski22 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some comments which you can take or leave. 😄

command/agent.go Outdated

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whilst a lot of the code isn't new, it might be nice to unify the casing for the import aliases while you're working in here (ctconfig, token_file, agentConfig). I think a few files have the varying casing issue.

command/agentproxyshared/auth/auth.go Outdated Show resolved Hide resolved
command/agentproxyshared/auth/auth.go Outdated Show resolved Hide resolved
command/agentproxyshared/auth/auth.go Outdated Show resolved Hide resolved
command/proxy_test.go Show resolved Hide resolved
command/proxy_test.go Outdated Show resolved Hide resolved
Comment on lines +294 to +295
roleIDPath := makeTempFile(t, "role_id.txt", roleID+"\n")
secretIDPath := makeTempFile(t, "secret_id.txt", secretID+"\n")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

populateTempFile does the same thing as makeTempFile but uses t.TempDir I think I added it rather than changing all the existing calls to makeTempFile.

Now I think I should have just changed them all or changed makeTempDir to call populateTempFile until I got the courage to refactor. 😞

@peteski22
Copy link

One thing I didn't catch was if this was implemented yet:

with warnings in the logs during these releases if vault agent is using API proxy at all

If someone uses vault agent with API proxy config is there a warning generated now due to this PR?

@VioletHynes
Copy link
Contributor Author

If someone uses vault agent with API proxy config is there a warning generated now due to this PR?

Not yet! That'll be coming soon. I mentioned in the description that this was pending. I wanted to make sure the message had scrutiny, and it's easy to miss things or gloss over them when they're in a larger change request.

@peteski22
Copy link

I mentioned in the description that this was pending

  • Warnings for an Agent that's using the API proxy

Yes you totally did, sorry!

@VioletHynes VioletHynes merged commit 6b4b0f7 into main May 17, 2023
@VioletHynes VioletHynes deleted the violethynes/VAULT-15547 branch May 17, 2023 13:39
@@ -0,0 +1,3 @@
```release-note:feature
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@VioletHynes could you please update this changelog file with the format for announcing a new feature? https://hashicorp.atlassian.net/wiki/spaces/VAULT/pages/1311244491/Changelog+Process#New-and-Major-Features

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do! Good catch, sorry for missing this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants