Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use nssm for kubelet log rotation on Windows #762

Merged
merged 1 commit into from
Jan 7, 2022

Conversation

jsturtevant
Copy link
Contributor

@jsturtevant jsturtevant commented Dec 14, 2021

What this PR does / why we need it:

When installing kubelet via nssm we still used kubelets built in logging. The built in logger logs duplicated data two three different files streams which isn't useful since it duplicates data and the error/warning files don't have enough context in them.

Since we already use nssm we can use the built in log redirection and logging will be much simpler to reason about and contain less information. The log files will still be in the same location.

This also turns on kubelet to Automatic so it will boot automatically on restart (#554)

Which issue(s) this PR fixes (optional, in fixes #(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged): Fixes #554

Additional context
Add any other context for the reviewers

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Dec 14, 2021
@jsturtevant jsturtevant changed the title Use nssm for log rotation Use nssm for kubelet log rotation on Windows Dec 14, 2021
@k8s-ci-robot k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Dec 14, 2021
@jsturtevant
Copy link
Contributor Author

/sig windows
/assign @perithompson

@k8s-ci-robot k8s-ci-robot added sig/windows Categorizes an issue or PR as relevant to SIG Windows. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Dec 14, 2021
@k8s-ci-robot k8s-ci-robot removed the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 14, 2021
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Dec 14, 2021
@jsturtevant
Copy link
Contributor Author

hmm the pip install failed here too, we may need an updated pip in ci?

�[0;32m    sig-ubuntu-1804-gen2: fatal: [default]: FAILED! => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "azure-cli"], "msg": "stdout: Collecting azure-cli\n  Downloading https://files.pythonhosted.org/packages/ba/5a

/retest

@jsturtevant
Copy link
Contributor Author

failure was ubuntu:

�[0;32m    sig-ubuntu-2004-gen2: fatal: [default]: FAILED! => {"changed": false, "cmd": "/usr/bin/apt-key add -", "msg": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: can't connect to the agent: IPC connect call failed", "rc": 2, "stderr": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: can't connect to the agent: IPC connect call failed\n", "stderr_lines": ["Warning: apt-key output should not be parsed (stdout is not a terminal)", "gpg: can't connect to the agent: IPC connect call failed"], "stdout": "", "stdout_lines": []}�[0m

Windows passed: 1;32m==> sig-windows-2022-containerd: Goss validate ran successfully�[0m

/retest

@jsturtevant
Copy link
Contributor Author

flake with prow reading the Azure secrets

/retest

@jsturtevant
Copy link
Contributor Author

packer crashed:

�[1;31mBuild 'sig-windows-2019' errored after 29 minutes 17 seconds: unexpected EOF�[0m

==> Wait completed after 29 minutes 17 seconds

==> Some builds didn't complete successfully and had errors:
--> sig-windows-2019: unexpected EOF

==> Builds finished but no artifacts were created.
panic: runtime error: invalid memory address or nil pointer dereference
2021/12/15 19:56:10 packer-builder-azure-arm plugin: [signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0x1a933ce]
2021/12/15 19:56:10 packer-builder-azure-arm plugin: 
2021/12/15 19:56:10 packer-builder-azure-arm plugin: goroutine 42 [running]:
2021/12/15 19:56:10 packer-builder-azure-arm plugin: github.com/Azure/go-autorest/autorest/azure.(*Future).WaitForCompletionRef(0xc000788cc0, 0x546d9c0, 0xc0008740c0, 0x53f6420, 0xc000e74f80, 0x540e140, 0xc000dbcc90, 0xc000e83ac0, 0xc0012646e0, 0xdf8475800, ...)
2021/12/15 19:56:10 packer-builder-azure-arm plugin: 	/home/circleci/project/packer/vendor/github.com/Azure/go-autorest/autorest/azure/async.go:174 +0x54e

I don't see anything in this pr that would cause this will run again then investigate if it happens again...

/retest

@jsturtevant
Copy link
Contributor Author

created issue for Ubuntu pip error: #766

/retest

@perithompson
Copy link
Contributor

ah this looks much better, ive want to have that auto for a while but the logs always stopped me, thank @jsturtevant

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 6, 2022
application: '%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe'
arguments: '-ExecutionPolicy Bypass -NoProfile {{ kubernetes_install_path }}\StartKubelet.ps1'
app_rotate_bytes: 10485760
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be configurable and defaulted to this value if not passed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can improve it on another PR..

@knabben
Copy link
Member

knabben commented Jan 6, 2022

/lgtm

@jsturtevant
Copy link
Contributor Author

/assign @codenrhoden @CecileRobertMichon

Copy link
Contributor

@CecileRobertMichon CecileRobertMichon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CecileRobertMichon, jsturtevant

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 7, 2022
@k8s-ci-robot k8s-ci-robot merged commit 4eea397 into kubernetes-sigs:master Jan 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. sig/windows Categorizes an issue or PR as relevant to SIG Windows. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve the way kubelet restarts for windows
6 participants