Skip to content
This repository has been archived by the owner on Oct 22, 2024. It is now read-only.

Commit

Permalink
CI: no logging by default
Browse files Browse the repository at this point in the history
The recently added increased logging made the resulting
log files rather large, which is a problem when downloading
them and exhausted available space in the Jenkins master.

Now that extra logging is turned off by default respectively
done less frequently (top and df run once a minute). To enable
logging, edit the pipeline in a PR or in the Jenkins UI.
  • Loading branch information
pohly committed Oct 19, 2019
1 parent 484ee99 commit 272df2d
Showing 1 changed file with 26 additions and 5 deletions.
31 changes: 26 additions & 5 deletions Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,27 @@ pipeline {
}

environment {
/*
Change this into "true" to enable capturing the journalctl
output of the build host and each VM, either by editing the
Jenkinsfile in a PR or by logging into Jenkins and editing
the pipeline before running it again.
*/
LOGGING_JOURNALCTL = "false"

/*
Delay in seconds between dumping system statistics.
*/
LOGGING_SAMPLING_DELAY = "60"

/*
Pod names in the kube-system namespace for which
log output is to be captured. Empty by default,
valid values:
etcd kube-apiserver kube-controller-manager kube-scheduler
*/
LOGGING_PODS = ""

/*
For each major Kubernetes release we need one version of Clear Linux
which had that release. Installing different Kubernetes releases
Expand Down Expand Up @@ -360,8 +381,8 @@ void TestInVM(deviceMode, deploymentMode, distro, distroVersion, kubernetesVersi
*/
sh " \
mkdir -p build/reports && \
sudo journalctl -f & \
( set +x; while true; do sleep 30; top -b -n 1 -w 120 | head -n 20; df -h; done ) & \
if ${env.LOGGING_JOURNALCTL}; then sudo journalctl -f; fi & \
( set +x; while true; do sleep ${env.LOGGING_SAMPLING_DELAY}; top -b -n 1 -w 120 | head -n 20; df -h; done ) & \
docker run --rm \
--privileged=true \
-e CLUSTER=clear \
Expand All @@ -384,16 +405,16 @@ void TestInVM(deviceMode, deploymentMode, distro, distroVersion, kubernetesVersi
swupd bundle-add openssh-server && \
make start && cd ${env.PMEM_PATH} && \
_work/clear/ssh.0 kubectl get pods --all-namespaces -o wide && \
for pod in etcd kube-apiserver kube-controller-manager kube-scheduler; do \
for pod in ${env.LOGGING_PODS}; do \
_work/clear/ssh.0 kubectl logs -f -n kube-system \$pod-pmem-csi-clear-master | sed -e \"s/^/\$pod: /\" & \
done && \
_work/clear/ssh.0 tar -C / -cf - usr/bin/kubectl | tar -C /usr/local/bin --strip-components=2 -xf - && \
for ssh in \$(ls _work/clear/ssh.[0-9]); do \
hostname=\$(\$ssh hostname) && \
( set +x; \
while true; do \$ssh journalctl -f; done & \
if ${env.LOGGING_JOURNALCTL}; then while true; do \$ssh journalctl -f; done; fi & \
while true; do \
sleep 30; \
sleep ${env.LOGGING_SAMPLING_DELAY}; \
\$ssh top -b -n 1 -w 120 2>&1 | head -n 20; \
done ) | sed -e \"s/^/\$hostname: /\" & \
done && \
Expand Down

0 comments on commit 272df2d

Please sign in to comment.