[config] Enable/disable container monitoring when starting/stopping the services. #1471
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
stopping services and monitor it again after restarting services when ran the commands `sudo config reload`, `sudo config load` and `sudo config load_minigraph`. Signed-off-by: Yong Zhao <[email protected]>
Signed-off-by: Yong Zhao <[email protected]>
Signed-off-by: Yong Zhao <[email protected]>
jleveque
suggested changes
Mar 2, 2021
jleveque
suggested changes
Mar 2, 2021
Signed-off-by: Yong Zhao <[email protected]>
yozhao101
changed the title
Great suggestion, Updated! |
jleveque
suggested changes
Mar 2, 2021
Signed-off-by: Yong Zhao <[email protected]>
Signed-off-by: Yong Zhao <[email protected]>
Signed-off-by: Yong Zhao <[email protected]>
jleveque
previously approved these changes
Mar 2, 2021
Signed-off-by: Yong Zhao <[email protected]>
jleveque
approved these changes
Mar 3, 2021
|
@jleveque I will create a separate PR to update the submodule. |
|
@yozhao101, @jleveque this change cannot be cherry-picked to 202012 cleanly, Is there a dependency needs to be cherry-picked? I think this PR is depended on this one: #1199. |
|
This commit has been reverted from the master branch, so I am removing the "Request for 202012 branch" label. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Yong Zhao [email protected]
What I did
When we ran the command
sudo config load,sudo config reloadorsudo config load_minigraph, the containersswss, snmp, lldp, teamd, syncd, snmp, bgp, radv, pmon, dhcp_relay, telemetryandrestapiwould be stopped and then restarted. The scriptcontainer_checkerran by Monit will generate false alerting messages into syslog to indicate some containers were not running during such stopping and restarting process. So this PR aims to prevent Monit from generating false alarm messages.How I did it
Before stopping services, we disable Monit to monitor the running status of containers. After restarting services, we enable Monit to monitor the running status of containers again.
How to verify it
I deliberately reduce the monitoring interval of Monit from 60 seconds to 10 seconds to ensure the alerting messages from the script
container_checkerwas generated duringsudo config reload, sudo config load and sudo config load_minigraph. After this change was added into_stop_services(...)and_restart_services(...), I checked that the alerting messages fromcontainer_checkerdid not appear in the syslog.I verified this change on the device
str-a7050-acs-3.Previous command output (if the output of a command-line utility has changed)
New command output (if the output of a command-line utility has changed)