Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Emit a warning when env loading bug is being relied on #4730

Merged
merged 1 commit into from
Jul 28, 2022

Conversation

KnVerey
Copy link
Contributor

@KnVerey KnVerey commented Jul 28, 2022

Problem

It recently came to our attention that the configMapGenerator has a bug where it will load values from the environment when a line in an env file has a key but no value. This bug dates back to the origins of Kustomize, when some of the original code was copied out of kubectl, and this undesirable behaviour (for kustomize, not kubectl) went unnoticed in the adaptation.

This behaviour is a clear violation of one of Kustomize's core principles, as documented in our Eschewed Features list:

image

Very unfortunately, someone not only discovered this bug, but made a PR to document it as a feature on kubernetes.io: kubernetes/website#30348. Unfortunately no Kustomize maintainers were tagged on the PR, and it has been live since January 2022. 😞

As stated in Kustomize documentation screenshot above, any need for environment reading should be handled as part of a pre-build workflow. We can consider adding a kustomize edit subcommand for this specifically if there is demand from those currently relying on the bug. For example, it could look like kustomize edit add configmap my-configmap --from-env-file=env/path.env --from-env=ONE,TWO.

Proposal

I don't think there's a nice way to unit test the printing to sdterr like this, barring making invasive changes to the kv loader method signatures (lmk if you have an idea for something nice). Here are some manual tests:

image

/cc @natasha41575 @annasong20

@k8s-ci-robot
Copy link
Contributor

@KnVerey: GitHub didn't allow me to request PR reviews from the following users: annasong20.

Note that only kubernetes-sigs members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

Problem

It recently came to our attention that the configMapGenerator has a bug where it will load values from the environment when a line in an env file has a key but no value. This bug dates back to the origins of Kustomize, when some of the original code was copied out of kubectl, and this undesirable behaviour (for kustomize, not kubectl) went unnoticed in the adaptation.

This behaviour is a clear violation of one of Kustomize's core principles, as documented in our Eschewed Features list:

image

Very unfortunately, someone not only discovered this bug, but made a PR to document it as a feature on kubernetes.io: kubernetes/website#30348. Unfortunately no Kustomize maintainers were tagged on the PR, and it has been live since January 2022. 😞

As stated in Kustomize documentation screenshot above, any need for environment reading should be handled as part of a pre-build workflow. We can consider adding a kustomize edit subcommand for this specifically if there is demand from those currently relying on the bug. For example, it could look like kustomize edit add configmap my-configmap --from-env-file=env/path.env --from-env=ONE,TWO.

Proposal

  • Immediately revert the docs PR incorrectly describing this bug as a feature
  • Start emitting a warning to those relying on this bug (because it has existed for so long, and was in the docs for 7 months) -- That's this PR
  • Patch the bug in the next major version of Kustomize, i.e. v5

I don't think there's a nice way to unit test the printing to sdterr like this, barring making invasive changes to the kv loader method signatures (lmk if you have an idea for something nice). Here are some manual tests:

image

/cc @natasha41575 @annasong20

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jul 28, 2022
@netlify
Copy link

netlify bot commented Jul 28, 2022

Deploy Preview for kubernetes-sigs-kustomize canceled.

Name Link
🔨 Latest commit 2a3b645
🔍 Latest deploy log https://app.netlify.com/sites/kubernetes-sigs-kustomize/deploys/62e30955e47c040009d9361b

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: KnVerey

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jul 28, 2022
@natasha41575
Copy link
Contributor

/lgtm

I can retag after you fix the lint error

@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 28, 2022
@KnVerey
Copy link
Contributor Author

KnVerey commented Jul 28, 2022

I can retag after you fix the lint error

bah, line length! 🤦‍♀️ Fixed, and I re-ran my manual tests:

image

@natasha41575
Copy link
Contributor

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 28, 2022
@k8s-ci-robot k8s-ci-robot merged commit 30843e2 into kubernetes-sigs:master Jul 28, 2022
@KnVerey KnVerey deleted the env_bug_warning branch July 28, 2022 22:51
@sftim
Copy link

sftim commented Aug 3, 2022

BTW, kubectl emits warnings in yellow; see https://kubernetes.io/blog/2020/09/03/warnings/

Consider copying that style.

@annasong20
Copy link
Contributor

I don't know the difference between os.Stderr and log, but I recently learned that you can use log.SetOutput to specify the output location of log functions. This made it really easy for me to check logging in tests.

@KnVerey
Copy link
Contributor Author

KnVerey commented Oct 11, 2022

This PR does not change behaviour; it simply emits a warning. The provided samples build fine for me with the latest release of Kustomize, and do not hit the warning case in this PR. If you're able to reproduce, please open a new issue and include the Kustomize version and error message.

sknat added a commit to projectcalico/vpp-dataplane that referenced this pull request Jan 4, 2023
This patch changes the kustomize.sh logic, due to an evolution
in the kustomize CLI [0] where envvars will not be loaded automatically
anymore.

Also this adapats simplifies the logic using a single overridden configmap

[0] kubernetes-sigs/kustomize#4730

Signed-off-by: Nathan Skrzypczak <[email protected]>
sknat added a commit to projectcalico/vpp-dataplane that referenced this pull request Jan 5, 2023
This patch changes the kustomize.sh logic, due to an evolution
in the kustomize CLI [0] where envvars will not be loaded automatically
anymore.

Also this adapats simplifies the logic using a single overridden configmap

[0] kubernetes-sigs/kustomize#4730

Signed-off-by: Nathan Skrzypczak <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants