Skip to content

Commit

Permalink
[warm/fast-reboot] Backup logs from tmpfs to disk during fast/warm sh…
Browse files Browse the repository at this point in the history
…utdown (#2714)

Goal: Preserve logs during TOR upgrades and shutdown

Need:

Below PRs moved logs from disk to tmpfs for specific hwskus.
Due to these changes, shutdown path logs are now lost.
The logs in shutdown path are crucial for debug purposes.

sonic-net/sonic-buildimage#13805
sonic-net/sonic-buildimage#13587
sonic-net/sonic-buildimage#13587

How I did it
Check if logs are on tmpfs. If yes, backup logs from /var/log

How to verify it
Verified on a physical device - logs on tmfs are backed up for past 30 minutes.
  • Loading branch information
vaibhavhd authored and StormLiangMS committed Mar 7, 2023
1 parent f2a31b3 commit 8c7ddf5
Showing 1 changed file with 11 additions and 0 deletions.
11 changes: 11 additions & 0 deletions scripts/fast-reboot
Original file line number Diff line number Diff line change
Expand Up @@ -788,6 +788,17 @@ fi
# Reboot: explicitly call Linux native reboot under sbin
debug "Rebooting with ${REBOOT_METHOD} to ${NEXT_SONIC_IMAGE} ..."
LOGS_ON_TMPFS=0
df --output=fstype /var/log* | grep -c 'tmpfs' || LOGS_ON_TMPFS=$?
if [[ LOGS_ON_TMPFS -eq 0 ]]; then
debug "Backup shutdown logs to /host/logs_before_reboot"
mkdir -p /host/logs_before_reboot || /bin/true
# maxdepth 2: find files within 2 nested directories (eg. /var/log/ and /var/log/swss/)
# mmin 30: find files written in past 30 minutes
find /var/log -maxdepth 2 -mmin -30 -type f | xargs -I {} cp {} /host/logs_before_reboot/ || /bin/true
fi
exec ${REBOOT_METHOD}
# Should never reach here
Expand Down

0 comments on commit 8c7ddf5

Please sign in to comment.