Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AssertionError: Assertion failed for start.sh: RUNNING not in STARTING #2136

Closed
lguohan opened this issue Oct 11, 2018 · 3 comments
Closed

Comments

@lguohan
Copy link
Collaborator

lguohan commented Oct 11, 2018

Description

supervisord crashed due to clock moved backward.

Sep 27 09:44:37.256048 hostname INFO supervisord: start.sh teamd: started
Sep 27 09:44:37.272313 hostname INFO teamd.sh[4245]: Traceback (most recent call last):
Sep 27 09:44:37.272802 hostname INFO teamd.sh[4245]: File "/usr/bin/supervisord", line 9, in <module>
Sep 27 09:44:37.273493 hostname INFO teamd.sh[4245]: load_entry_point('supervisor==3.3.3', 'console_scripts', 'supervisord')()
Sep 27 09:44:37.273906 hostname INFO teamd.sh[4245]: File "/usr/lib/python2.7/dist-packages/supervisor/supervisord.py", line 363, in main
Sep 27 09:44:37.274271 hostname INFO teamd.sh[4245]: go(options)
Sep 27 09:44:37.274637 hostname INFO teamd.sh[4245]: File "/usr/lib/python2.7/dist-packages/supervisor/supervisord.py", line 373, in go
Sep 27 09:44:37.275001 hostname INFO teamd.sh[4245]: d.main()
Sep 27 09:44:37.275388 hostname INFO teamd.sh[4245]: File "/usr/lib/python2.7/dist-packages/supervisor/supervisord.py", line 84, in main
Sep 27 09:44:37.275763 hostname INFO teamd.sh[4245]: self.run()
Sep 27 09:44:37.276154 hostname INFO teamd.sh[4245]: File "/usr/lib/python2.7/dist-packages/supervisor/supervisord.py", line 101, in run
Sep 27 09:44:37.276525 hostname INFO teamd.sh[4245]: self.runforever()
Sep 27 09:44:37.276892 hostname INFO teamd.sh[4245]: File "/usr/lib/python2.7/dist-packages/supervisor/supervisord.py", line 253, in runforever
Sep 27 09:44:37.277268 hostname INFO teamd.sh[4245]: self.reap()
Sep 27 09:44:37.277636 hostname INFO teamd.sh[4245]: File "/usr/lib/python2.7/dist-packages/supervisor/supervisord.py", line 289, in reap
Sep 27 09:44:37.278005 hostname INFO teamd.sh[4245]: process.finish(pid, sts)
Sep 27 09:44:37.278379 hostname INFO teamd.sh[4245]: File "/usr/lib/python2.7/dist-packages/supervisor/process.py", line 527, in finish
Sep 27 09:44:37.278757 hostname INFO teamd.sh[4245]: self._assertInState(ProcessStates.STARTING)
Sep 27 09:44:37.279137 hostname INFO teamd.sh[4245]: File "/usr/lib/python2.7/dist-packages/supervisor/process.py", line 179, in _assertInState
Sep 27 09:44:37.279508 hostname INFO teamd.sh[4245]: self.config.name, current_state, allowable_states))
Sep 27 09:44:37.279885 hostname INFO teamd.sh[4245]: AssertionError: Assertion failed for start.sh: RUNNING not in STARTING

time shift backward after teamd docker starts

Sep 27 09:44:40.973916 hostname INFO snmp.sh[4299]: 2018-09-27 09:44:40,973 INFO spawned: 'snmpd' with pid 33
Sep 27 09:44:27.315683 hostname INFO systemd[1]: Time has been changed
Sep 27 09:44:27.464486 hostname INFO snmpd-config-updater: Platform is not an Arista platform. Exiting...

supervisord version 3.3.3

Steps to reproduce the issue:
1.
2.
3.

Describe the results you received:

Describe the results you expected:

Additional information you deem important (e.g. issue happens only occasionally):

**Output of `show version`:**

```
(paste your output here)
```

**Attach debug file `sudo generate_dump`:**

```
(paste your output here)
```
@lguohan
Copy link
Collaborator Author

lguohan commented Oct 11, 2018

#762

@stcheng
Copy link
Contributor

stcheng commented Nov 20, 2018

This bug is still open.

@jleveque
Copy link
Contributor

jleveque commented Jul 3, 2019

Fixed in #2624

@jleveque jleveque closed this as completed Jul 3, 2019
stepanblyschak added a commit to stepanblyschak/sonic-buildimage that referenced this issue Apr 22, 2022
```
22a388b [show] fix get routing stack routine (sonic-net#2137)
cb3a047 Support option --ports of config qos reload for reloading ports' QoS and buffer configuration to default (sonic-net#2125)
154a801 Enhance "config interface type/advertised-type" to be blocked on RJ45 ports  (sonic-net#2112)
3732ac5 Add CLI for route flow counter feature (sonic-net#2031)
29771e7 [techsupport] improve robustness (sonic-net#2117)
f9dc681 [intfutil] Display RJ45 port and portchannel speed in 'M' instead of 'G' when it's <= 1000M (sonic-net#2110)
781ae9f [config] Do not enable pfcwd for BmcMgmtToRRouter (sonic-net#2136)
23e9398 [scripts/fast-reboot] Shutdown remaining containers through systemd (sonic-net#2133)
576c9ef [scripts/fast-reboot] stop timers in advance (sonic-net#2131)
4dad79c bugfix: incorrect command for portchannel creation (sonic-net#2134)
c17b1f4 [show][muxcable] Decrease the timeout for show mux status/hwmode (sonic-net#2130)
49d61f8 [scripts/fast-reboot] cleanup (sonic-net#2132)
52ca324 [config/config_mgmt.py]: Fix dpb issue with upper case mac in (sonic-net#2066)
9e2fbf4 Update db_migrator to support `pfcwd_sw_enable` (sonic-net#2087)
4010bd0 FGNHG CLI changes (sonic-net#1588)
6bd54d0 Fix 'show mac' output when FDB entry for default vlan is None instead of 1 (sonic-net#2126)
```

Signed-off-by: Stepan Blyschak <[email protected]>
liat-grozovik pushed a commit that referenced this issue May 12, 2022
288c2d8 Revert "[scripts/fast-reboot] Shutdown remaining containers through systemd (#2133)" (#2161)
bce4694 [autoneg] add support for remote speed advertisement (#2124)
a73f156 [show][vrf]Fixing show vrf to include vlan subinterface (#2158)
7a06457 [auto_ts] Enable register/de-register auto_ts config for APP Extension (#2139)
083ebcc Add transceiver-info items advertised for cmis-supported moddules (#2135)
0811214 Validate destination port is not LAG (#2053)
6ab1c51 [minigraph]  Consume golden_config_db.json while loading minigraph (#2140)
c37a957 [Kdump] Remove the duplicate logic if Kdump was disabled (#2128)
1143869 Ordering fix for sfpshow eeprom (#2113)
fdb79b8 Allow fw update for other boot type against on the previous "none" boot fw update (#2040)
a54a091 [GCU] Supressing YANG errors from libyang while sorting (#1991)
fbfa8bc [GCU] Enabling AddRack and adding RemoveRack tests (#2143)
d012be9 [Command-Reference] Add CLI docs for route flow counter (#2069)
8c07d59 [Mellanox] [reboot] [asan] stop asan-enabled containers on reboot (#2107)
697aae3 Fix speed parsing when speed is NOT fetched from APPL_DB (#2138)
22a388b [show] fix get routing stack routine (#2137)
cb3a047 Support option --ports of config qos reload for reloading ports' QoS and buffer configuration to default (#2125)
154a801 Enhance "config interface type/advertised-type" to be blocked on RJ45 ports  (#2112)
3732ac5 Add CLI for route flow counter feature (#2031)
29771e7 [techsupport] improve robustness (#2117)
f9dc681 [intfutil] Display RJ45 port and portchannel speed in 'M' instead of 'G' when it's <= 1000M (#2110)
781ae9f [config] Do not enable pfcwd for BmcMgmtToRRouter (#2136)
23e9398 [scripts/fast-reboot] Shutdown remaining containers through systemd (#2133)
576c9ef [scripts/fast-reboot] stop timers in advance (#2131)
4dad79c bugfix: incorrect command for portchannel creation (#2134)
c17b1f4 [show][muxcable] Decrease the timeout for show mux status/hwmode (#2130)
49d61f8 [scripts/fast-reboot] cleanup (#2132)
52ca324 [config/config_mgmt.py]: Fix dpb issue with upper case mac in (#2066)
9e2fbf4 Update db_migrator to support `pfcwd_sw_enable` (#2087)
4010bd0 FGNHG CLI changes (#1588)
6bd54d0 Fix 'show mac' output when FDB entry for default vlan is None instead of 1 (#2126)
liushilongbuaa pushed a commit to liushilongbuaa/sonic-buildimage that referenced this issue Jun 20, 2022
…anch

Related work items: #52, #71, #73, #75, #77, sonic-net#1306, sonic-net#1588, sonic-net#1991, sonic-net#2031, sonic-net#2040, sonic-net#2053, sonic-net#2066, sonic-net#2069, sonic-net#2087, sonic-net#2107, sonic-net#2110, sonic-net#2112, sonic-net#2113, sonic-net#2117, sonic-net#2124, sonic-net#2125, sonic-net#2126, sonic-net#2128, sonic-net#2130, sonic-net#2131, sonic-net#2132, sonic-net#2133, sonic-net#2134, sonic-net#2135, sonic-net#2136, sonic-net#2137, sonic-net#2138, sonic-net#2139, sonic-net#2140, sonic-net#2143, sonic-net#2158, sonic-net#2161, sonic-net#2233, sonic-net#2243, sonic-net#2250, sonic-net#2254, sonic-net#2260, sonic-net#2261, sonic-net#2267, sonic-net#2278, sonic-net#2282, sonic-net#2285, sonic-net#2288, sonic-net#2289, sonic-net#2292, sonic-net#2294, sonic-net#8887, sonic-net#9279, sonic-net#9390, sonic-net#9511, sonic-net#9700, sonic-net#10025, sonic-net#10322, sonic-net#10479, sonic-net#10484, sonic-net#10493, sonic-net#10500, sonic-net#10580, sonic-net#10595, sonic-net#10628, sonic-net#10634, sonic-net#10635, sonic-net#10644, sonic-net#10670, sonic-net#10691, sonic-net#10716, sonic-net#10731, sonic-net#10750, sonic-net#10751, sonic-net#10752, sonic-net#10761, sonic-net#10769, sonic-net#10775, sonic-net#10776, sonic-net#10779, sonic-net#10786, sonic-net#10792, sonic-net#10793, sonic-net#10800, sonic-net#10806, sonic-net#10826, sonic-net#10839, sonic-net#10840, sonic-net#10842, sonic-net#10844, sonic-net#10847, sonic-net#10849, sonic-net#10852, sonic-net#10865, sonic-net#10872, sonic-net#10877, sonic-net#10886, sonic-net#10889, sonic-net#10903, sonic-net#10904, sonic-net#10905, sonic-net#10913, sonic-net#10914, sonic-net#10916, sonic-net#10919, sonic-net#10925, sonic-net#10926, sonic-net#10929, sonic-net#10933, sonic-net#10934, sonic-net#10937, sonic-net#10941, sonic-net#10947, sonic-net#10952, sonic-net#10953, sonic-net#10957, sonic-net#10959, sonic-net#10971, sonic-net#10972, sonic-net#10980
tshalvi pushed a commit to tshalvi/sonic-buildimage that referenced this issue Dec 20, 2022
* [fdborch] Fix FDB flush issues

Signed-off-by: Oleksandr Kolomeiets <[email protected]>

* Fix coverage

* Fix test_FdbAddedAfterMemberCreated.

* Fix flush_syncd_notif_ut.cpp.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants