Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't write logs that contain environment variables #13877

Merged
merged 1 commit into from
Mar 29, 2022

Conversation

spowelljr
Copy link
Member

Fixes #13527

The debug logging is very verbose and can sometimes contains logs that output every environment variable on the users computer. This in itself is not an issue, but if the user is to upload their logs to GitHub for debugging purposes they could have secrets or other sensitive tokens in their environment variables which shouldn't be exposed to the public.

This fix skips writing logs that contain all the environment variables to protect the user.

Before:

I0329 11:16:37.021218   19169 main.go:130] libmachine: (minikube) DBG | 2022/03/29 11:16:37 DEBUG: hyperkit: Redirecting stdout/stderr to logger
I0329 11:16:37.021348   19169 main.go:130] libmachine: (minikube) DBG | 2022/03/29 11:16:37 DEBUG: hyperkit: Starting &exec.Cmd{Path:"/usr/local/bin/hyperkit", Args:[]string{"/usr/local/bin/hyperkit", "-A", "-u", "-F", "/Users/powellsteven/.minikube/machines/minikube/hyperkit.pid", "-c", "2", "-m", "4000M", "-s", "0:0,hostbridge", "-s", "31,lpc", "-s", "1:0,virtio-net", "-U", "5ec75e06-af8c-11ec-b2f2-367dda37b18a", "-s", "2:0,virtio-blk,/Users/powellsteven/.minikube/machines/minikube/minikube.rawdisk", "-s", "3,ahci-cd,/Users/powellsteven/.minikube/machines/minikube/boot2docker.iso", "-s", "4,virtio-rnd", "-l", "com1,autopty=/Users/powellsteven/.minikube/machines/minikube/tty,log=/Users/powellsteven/.minikube/machines/minikube/console-ring", "-f", "kexec,/Users/powellsteven/.minikube/machines/minikube/bzimage,/Users/powellsteven/.minikube/machines/minikube/initrd,earlyprintk=serial loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=minikube"}, Env:[]string{"SENSITIVE_TOKEN=SECRET_VALUE"}, Dir:"", Stdin:io.Reader(nil), Stdout:(*os.File)(0xc000288688), Stderr:(*os.File)(0xc000288698), ExtraFiles:[]*os.File(nil), SysProcAttr:(*syscall.SysProcAttr)(nil), Process:(*os.Process)(nil), ProcessState:(*os.ProcessState)(nil), ctx:context.Context(nil), lookPathErr:error(nil), finished:false, childFiles:[]*os.File(nil), closeAfterStart:[]io.Closer{(*os.File)(0xc000288688), (*os.File)(0xc000288698)}, closeAfterWait:[]io.Closer{(*os.File)(0xc000288680), (*os.File)(0xc000288690)}, goroutine:[]func() error(nil), errch:(chan error)(nil), waitDone:(chan struct {})(nil)}
I0329 11:16:37.022714   19169 main.go:130] libmachine: (minikube) DBG | 2022/03/29 11:16:37 DEBUG: hyperkit: Pid is 19185
I0329 11:16:37.023190   19169 main.go:130] libmachine: (minikube) DBG | Attempt 0
I0329 11:16:37.023200   19169 main.go:130] libmachine: (minikube) DBG | exe=/Users/powellsteven/.minikube/bin/docker-machine-driver-hyperkit uid=0
I0329 11:16:37.023306   19169 main.go:130] libmachine: (minikube) DBG | hyperkit pid from json: 19185
I0329 11:16:37.023657   19169 main.go:130] libmachine: (minikube) DBG | 2022/03/29 11:16:37 DEBUG: hyperkit: Waiting for &exec.Cmd{Path:"/usr/local/bin/hyperkit", Args:[]string{"/usr/local/bin/hyperkit", "-A", "-u", "-F", "/Users/powellsteven/.minikube/machines/minikube/hyperkit.pid", "-c", "2", "-m", "4000M", "-s", "0:0,hostbridge", "-s", "31,lpc", "-s", "1:0,virtio-net", "-U", "5ec75e06-af8c-11ec-b2f2-367dda37b18a", "-s", "2:0,virtio-blk,/Users/powellsteven/.minikube/machines/minikube/minikube.rawdisk", "-s", "3,ahci-cd,/Users/powellsteven/.minikube/machines/minikube/boot2docker.iso", "-s", "4,virtio-rnd", "-l", "com1,autopty=/Users/powellsteven/.minikube/machines/minikube/tty,log=/Users/powellsteven/.minikube/machines/minikube/console-ring", "-f", "kexec,/Users/powellsteven/.minikube/machines/minikube/bzimage,/Users/powellsteven/.minikube/machines/minikube/initrd,earlyprintk=serial loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=minikube"}, Env:[]string{"SENSITIVE_TOKEN=SECRET_VALUE"}, Dir:"", Stdin:io.Reader(nil), Stdout:(*os.File)(0xc000288688), Stderr:(*os.File)(0xc000288698), ExtraFiles:[]*os.File(nil), SysProcAttr:(*syscall.SysProcAttr)(nil), Process:(*os.Process)(0xc0002831d0), ProcessState:(*os.ProcessState)(nil), ctx:context.Context(nil), lookPathErr:error(nil), finished:false, childFiles:[]*os.File{(*os.File)(0xc0002886a0), (*os.File)(0xc000288688), (*os.File)(0xc000288698)}, closeAfterStart:[]io.Closer{(*os.File)(0xc000288688), (*os.File)(0xc000288698), (*os.File)(0xc0002886a0)}, closeAfterWait:[]io.Closer{(*os.File)(0xc000288680), (*os.File)(0xc000288690)}, goroutine:[]func() error(nil), errch:(chan error)(nil), waitDone:(chan struct {})(nil)}
I0329 11:16:37.026272   19169 main.go:130] libmachine: (minikube) DBG | Searching for 66:8c:a6:92:1a:c2 in /var/db/dhcpd_leases ...

After:

I0329 11:05:04.445624   17739 main.go:134] libmachine: (minikube) DBG | 2022/03/29 11:05:04 DEBUG: hyperkit: Redirecting stdout/stderr to logger
I0329 11:05:04.447764   17739 main.go:134] libmachine: (minikube) DBG | 2022/03/29 11:05:04 DEBUG: hyperkit: Pid is 17771
I0329 11:05:04.448398   17739 main.go:134] libmachine: (minikube) DBG | Attempt 0
I0329 11:05:04.448410   17739 main.go:134] libmachine: (minikube) DBG | exe=/Users/powellsteven/.minikube/bin/docker-machine-driver-hyperkit uid=0
I0329 11:05:04.448537   17739 main.go:134] libmachine: (minikube) DBG | hyperkit pid from json: 17771
I0329 11:05:04.450830   17739 main.go:134] libmachine: (minikube) DBG | Searching for b2:72:84:af:33:d1 in /var/db/dhcpd_leases ...

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Mar 29, 2022
@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 29, 2022
@spowelljr
Copy link
Member Author

/ok-to-test

@k8s-ci-robot k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Mar 29, 2022
Copy link
Collaborator

@sharifelgamal sharifelgamal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sharifelgamal, spowelljr

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [sharifelgamal,spowelljr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@minikube-pr-bot
Copy link

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 13877) |
+----------------+----------+---------------------+
| minikube start | 54.0s    | 53.7s               |
| enable ingress | 29.1s    | 30.0s               |
+----------------+----------+---------------------+

Times for minikube start: 54.9s 53.0s 52.9s 54.2s 55.2s
Times for minikube (PR 13877) start: 53.8s 53.2s 54.8s 52.5s 54.1s

Times for minikube ingress: 29.1s 30.1s 26.2s 30.1s 30.2s
Times for minikube (PR 13877) ingress: 29.7s 30.1s 30.1s 31.7s 28.6s

docker driver with docker runtime
error collecting results for docker driver: timing run 0 with minikube: timing cmd: [out/minikube addons enable ingress]: waiting for minikube: exit status 10
docker driver with containerd runtime
error collecting results for docker driver: timing run 0 with minikube: timing cmd: [out/minikube addons enable ingress]: waiting for minikube: exit status 10

@minikube-pr-bot
Copy link

These are the flake rates of all failed tests.

Environment Failed Tests Flake Rate (%)
Docker_macOS TestNetworkPlugins/group/auto/DNS (gopogh) 0.88 (chart)
Docker_Linux TestAddons/parallel/Registry (gopogh) 1.23 (chart)
Docker_Linux TestFunctional/parallel/PersistentVolumeClaim (gopogh) 1.23 (chart)
Docker_Linux TestMultiNode/serial/DeployApp2Nodes (gopogh) 1.23 (chart)
Docker_Linux TestMultiNode/serial/PingHostFrom2Pods (gopogh) 1.23 (chart)
Docker_Linux TestFunctional/parallel/DashboardCmd (gopogh) 1.84 (chart)
Docker_Linux TestNetworkPlugins/group/auto/DNS (gopogh) 7.36 (chart)
Docker_Windows TestStartStop/group/newest-cni/serial/SecondStart (gopogh) 8.99 (chart)
Docker_macOS TestNetworkPlugins/group/false/DNS (gopogh) 18.42 (chart)
Docker_macOS TestNetworkPlugins/group/bridge/Start (gopogh) 27.97 (chart)
Docker_macOS TestNetworkPlugins/group/kindnet/DNS (gopogh) 28.81 (chart)
Docker_Linux_containerd TestStartStop/group/old-k8s-version/serial/FirstStart (gopogh) 29.37 (chart)
Docker_Linux_containerd TestStartStop/group/old-k8s-version/serial/SecondStart (gopogh) 29.37 (chart)
Docker_Linux_containerd TestStartStop/group/old-k8s-version/serial/DeployApp (gopogh) 29.92 (chart)
Docker_Linux_containerd TestStartStop/group/old-k8s-version/serial/UserAppExistsAfterStop (gopogh) 30.47 (chart)
Docker_Linux_containerd TestStartStop/group/no-preload/serial/DeployApp (gopogh) 30.71 (chart)
Docker_Linux_containerd TestStartStop/group/no-preload/serial/FirstStart (gopogh) 30.71 (chart)
Docker_Linux_containerd TestStartStop/group/no-preload/serial/SecondStart (gopogh) 32.28 (chart)
Docker_Linux TestNetworkPlugins/group/false/DNS (gopogh) 32.52 (chart)
Docker_Linux_containerd TestStartStop/group/no-preload/serial/UserAppExistsAfterStop (gopogh) 32.81 (chart)
Docker_macOS TestNetworkPlugins/group/calico/DNS (gopogh) 34.00 (chart)
Docker_Linux TestNetworkPlugins/group/kindnet/Start (gopogh) 43.56 (chart)
Hyper-V_Windows TestSkaffold (gopogh) 45.89 (chart)
Docker_Windows TestSkaffold (gopogh) 47.76 (chart)
Docker_Windows TestNoKubernetes/serial/Start (gopogh) 47.87 (chart)
Docker_macOS TestNetworkPlugins/group/kubenet/Start (gopogh) 48.25 (chart)
Docker_macOS TestStartStop/group/default-k8s-different-port/serial/SecondStart (gopogh) 49.06 (chart)
Docker_Linux_containerd TestNetworkPlugins/group/kindnet/Start (gopogh) 49.19 (chart)
Docker_Linux_containerd TestStartStop/group/default-k8s-different-port/serial/DeployApp (gopogh) 53.97 (chart)
Docker_Linux_containerd TestStartStop/group/default-k8s-different-port/serial/FirstStart (gopogh) 53.97 (chart)
More tests... Continued...

Too many tests failed - See test logs for more details.

To see the flake rates of all tests by environment, click here.

@spowelljr spowelljr merged commit 43c11c7 into kubernetes:master Mar 29, 2022
@spowelljr spowelljr deleted the filterOutEnvs branch March 29, 2022 21:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"minikube logs" exposes secrets in env
4 participants