Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

run sonic but no docker images load #1654

Closed
lockless opened this issue Apr 27, 2018 · 4 comments
Closed

run sonic but no docker images load #1654

lockless opened this issue Apr 27, 2018 · 4 comments

Comments

@lockless
Copy link

lockless commented Apr 27, 2018

Description

I install sonic-os on my switch, but none of the docker images (such as quagga/lldp/snmp ) is loaded

Steps to reproduce the issue:

  1. build the sonic os - 201803 branch
  2. install the os by onie
  3. power on, login

Describe the results you received:
this is the output

root@sonic:/home/admin# 
root@sonic:/home/admin# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
root@sonic:/home/admin# 
root@sonic:/home/admin# 
root@sonic:/home/admin# 
root@sonic:/home/admin# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
root@sonic:/home/admin# 
root@sonic:/home/admin# 
root@sonic:/home/admin# 

as you see, no docker images loaded

Describe the results you expected:
some docker images should be load..

Additional information you deem important (e.g. issue happens only occasionally):

Output of show version:
i forgot the get the "show version" information, but i get this

*SONiC-OS-201803.0-dirty-20180427.135350  

Attach debug file sudo generate_dump:

some reboot log, like this

Loading SONiC-OS OS kernel ...
Loading SONiC-OS OS initial ramdisk ...
ÿ[    1.777268] i8042: No controller found
Loading, please wait...
[    2.253502] irq 4: nobody cared (try booting with the "irqpoll" option)
[    2.332735] handlers:
[    2.359894] [<ffffffff81395610>] serial8250_interrupt
[    2.420446] Disabling IRQ #4
fsck from util-linux 2.25.2
SONiC-OS: clean, 54257/2097152 files, 379862/8388608 blocks
[  OK  ] Started LSB: Raise network interfaces..
         Starting ifup for eth0...
[  OK  ] Started ifup for eth0.
[  OK  ] Reached target Network.
[  OK  ] Reached target Network is Online.
[  OK  ] Reached target System Initialization.
[  OK  ] Listening on D-Bus System Message Bus Socket.
[  OK  ] Reached target Timers.
[  OK  ] Listening on Docker Socket for the API.
[  OK  ] Reached target Sockets.
[  OK  ] Reached target Basic System.
         Starting Regular background program processing daemon...
[  OK  ] Started Regular background program processing daemon.
         Starting OpenBSD Secure Shell server...
         Starting /etc/rc.local Compatibility...
         Starting Login Service...
         Starting LSB: Set sysfs variables from /etc/sysfs.conf...
         Starting LSB: Start NTP daemon...
         Starting LSB: service and resource monitoring daemon...
         Starting LSB: Execute the kexec -e command to reboot system...
         Starting D-Bus System Message Bus...
[  OK  ] Started D-Bus System Message Bus.
         Starting System Logging Service...
         Starting Permit User Sessions...
         Starting Docker Application Container Engine...
[  OK  ] Started System Logging Service.
[  OK  ] Started OpenBSD Secure Shell server.
[  OK  ] Started /etc/rc.local Compatibility.
[  OK  ] Started LSB: Set sysfs variables from /etc/sysfs.conf.
[  OK  ] Started LSB: Start NTP daemon.
[  OK  ] Started LSB: service and resource monitoring daemon.
[  OK  ] Started LSB: Execute the kexec -e command to reboot system.
[  OK  ] Started Permit User Sessions.
         Starting LSB: Load kernel image with kexec...
         Starting Getty on tty1...
[  OK  ] Started Getty on tty1.
         Starting Serial Getty on ttyS0...
[  OK  ] Started Serial Getty on ttyS0.
[  OK  ] Reached target Login Prompts.
[  OK  ] Started LSB: Load kernel image with kexec.
[  OK  ] Started Login Service.
[  OK  ] Started Docker Application Container Engine.
         Starting Database container...
[   74.861148] irq 4: nobody cared (try booting with the "irqpoll" option)
[   74.940384] handlers:
[   74.967539] [<ffffffff81395610>] serial8250_interrupt
[   75.028093] Disabling IRQ #4

Debian GNU/Linux 8 sonic ttyS0

sonic login: admin
Password: 
Linux sonic 3.16.0-5-amd64 #1 SMP Debian 3.16.51-3+deb8u1 (2018-01-08) x86_64
You are on
  ____   ___  _   _ _  ____
 / ___| / _ \| \ | (_)/ ___|
 \___ \| | | |  \| | | |
  ___) | |_| | |\  | | |___
 |____/ \___/|_| \_|_|\____|

-- Software for Open Networking in the Cloud --

Unauthorized access and/or use are prohibited.
All access and/or use are subject to monitoring.

Help:    http://azure.github.io/SONiC/

admin@sonic:~$ 
admin@sonic:~$ 

Thanks for some advise.
thanks.

@lockless lockless changed the title run sonic run sonic but no docker images load Apr 27, 2018
@stcheng
Copy link
Contributor

stcheng commented May 1, 2018

could you share the syslog?

@lockless
Copy link
Author

lockless commented May 3, 2018

@stcheng

Thanks for attention.
I have solved this problem, it cause by my build machine where the docker storage is overlay, and my switch device do not support it.

So, i change the docker storage mode, and rebuild it, it works.

I think this issue is worth noting, as most of the docker configure it's storage to overlay mode on an Ubuntu 16.04 OS default, and it would cause this problem.

@lockless lockless closed this as completed May 3, 2018
@stcheng
Copy link
Contributor

stcheng commented May 3, 2018

thanks for sharing the experience

jleveque added a commit that referenced this issue Mar 26, 2021
Update sonic-sairedis submodule and also update sonic-swss submodule as there are interdependent changes.

* src/sonic-sairedis 13474d1...bc58b0f (12):
  > Add gbsyncdmgrd; deprecate gbsyncd_startup.py (#809)
  > Remove gbsyncd_start.sh (#808)
  > [gbsyncd] Fix shebang in gbsyncd_startup.py; Make script executable (#807)
  > [saiasiccmp] Add saiasiccmp tool to compare 2 asic views (#791)
  > [configure] Add -Wno-psabi to remove "passing argument changed in GCC 7.1" (#799)
  > Update FlexCounter.cpp, use m_pollInterval in MUTEX lock (#797)
  > [vs] Add special warm boot logic to populate default attributes (#796)
  > [ci]: add vstest (#795)
  > [tests] Add macsec unittest (#782)
  > [debian/control] libsairedis-dev depends on libzmq5-dev (#794)
  > [ci]: use build template (#793)
  > Rename duplicate file name (#773)

* src/sonic-swss 0b0d24c...5adb73e (47):
  > Initialize system port type variable (#1681)
  > [Dynamic Buffer Calc] Enhance the field checking in table handling (#1680)
  > Handle the clear request for 'Q_SHARED_ALL' (#1653)
  > [MuxOrch] FDB ageout safety check (#1674)
  > Deactivate mirror session only when session status is true in updateLagMember (#1666)
  > Revert "[buffermgr] Support maximum port headroom checking (#1607)" (#1675)
  > reduce severity of log to info in case of flush on non-existing member (#1669)
  > Revert "[Dynamic buffer calc] Bug fix: Remove PGs from an administratively down port. (#1652)" (#1676)
  > [Dynamic buffer calc] Bug fix: Remove PGs from an administratively down port. (#1652)
  > [acl] Move ACL table constants to acltable.h (#1671)
  > [nbrmgrd] added function to parse IP address from APP_DB (#1672)
  > [MUX/PFCWD] Use in_ports for acls instead of seperate ACL table (#1670)
  > [vog/systemlag] Voq lagid allocator (#1603)
  > Add table descriptions for dynamic buffer calculation to the documents (#1664)
  > [vstest/subintf] Add vs test case to validate processing sequence of APPL DB keys (#1663)
  > Remove vxlanmgrd dependency on orchagent (#1647)
  > Keep attribute order in bulk mode (#1659)
  > [mux] VS test for neigh, route and fdb (#1656)
  > [linksync] Netdev oper status determination using IFF_RUNNING (#1568)
  > [portorch] parse on/off value from autoneg (#1658)
  > [intfsorch] Create subport with the entry contains necessary attributes (#1650)
  > [ci]: Purge swss before install (#1654)
  > Update StateDB with error if state change failed, Update APP_DB in all state chg req (#1662)
  > Added changes to handle dependency check in FdbSyncd and FpmSyncd for warm-boot (#1556)
  > [synchronous mode] Add failure notification for SAI failures in synchronous mode (#1596)
  > [acl] Enable VLAN ID qualifier for ACL rules (#1648)
  > Updated PFCWD to use single ACL table for PFCWD and MUX (#1620)
  > [orchagent] Increase SAI REDIS response timeout to support FW upgrade during init (Mellanox only). (#1637)
  > [vstest/nhg]: use dvs_route fixture to make test_nhg more robust
  > [vstest]: add dvs_route fixture
  > [vstest/subintf] Update vs tests to validate physical port host interface vlan tag attribute (#1634)
  > Remove useless header  in macsecorch (#1628)
  > Add SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS counter, create new FlexCounter group (#1600)
  > fixed unsupported resource issue (#1641)
  > [test_virtual_chassis]: use wait_for to make test more robust (#1640)
  > spell check fixes (#1630)
  > [bufferorch] Handle NOT IMPLEMENTED status returned during set attr operation (#1639)
  > [ci]: run vstest
  > [test_virtual_chassis]: use wait_for function to improve test robustness
  > [Mux] Neighbor handling based on FDB entry (#1631)
  > [ci]: use build template (#1633)
  > Log level change from ERR to INFO for fetch systemports issue (#1632)
  > Migrate serdes programming to port serdes object (#1611)
  > [tests] Remove legacy saiattributelist.h dependency (#1608)
  > [buffermgr] Support maximum port headroom checking (#1607)
  > Support shared headroom pool on top of dynamic buffer calculation (#1581)
  > Fix the compiling errors in gcc9 (#1621)
@bidhovbizar
Copy link

@stcheng

Thanks for attention.
I have solved this problem, it cause by my build machine where the docker storage is overlay, and my switch device do not support it.

So, i change the docker storage mode, and rebuild it, it works.

I think this issue is worth noting, as most of the docker configure it's storage to overlay mode on an Ubuntu 16.04 OS default, and it would cause this problem.

I too have the same problem.

How did you change the docker storage mode? Please advice.

raphaelt-nvidia pushed a commit to raphaelt-nvidia/sonic-buildimage that referenced this issue May 23, 2021
Update sonic-sairedis submodule and also update sonic-swss submodule as there are interdependent changes.

* src/sonic-sairedis 13474d1...bc58b0f (12):
  > Add gbsyncdmgrd; deprecate gbsyncd_startup.py (sonic-net#809)
  > Remove gbsyncd_start.sh (sonic-net#808)
  > [gbsyncd] Fix shebang in gbsyncd_startup.py; Make script executable (sonic-net#807)
  > [saiasiccmp] Add saiasiccmp tool to compare 2 asic views (sonic-net#791)
  > [configure] Add -Wno-psabi to remove "passing argument changed in GCC 7.1" (sonic-net#799)
  > Update FlexCounter.cpp, use m_pollInterval in MUTEX lock (sonic-net#797)
  > [vs] Add special warm boot logic to populate default attributes (sonic-net#796)
  > [ci]: add vstest (sonic-net#795)
  > [tests] Add macsec unittest (sonic-net#782)
  > [debian/control] libsairedis-dev depends on libzmq5-dev (sonic-net#794)
  > [ci]: use build template (sonic-net#793)
  > Rename duplicate file name (sonic-net#773)

* src/sonic-swss 0b0d24c...5adb73e (47):
  > Initialize system port type variable (sonic-net#1681)
  > [Dynamic Buffer Calc] Enhance the field checking in table handling (sonic-net#1680)
  > Handle the clear request for 'Q_SHARED_ALL' (sonic-net#1653)
  > [MuxOrch] FDB ageout safety check (sonic-net#1674)
  > Deactivate mirror session only when session status is true in updateLagMember (sonic-net#1666)
  > Revert "[buffermgr] Support maximum port headroom checking (sonic-net#1607)" (sonic-net#1675)
  > reduce severity of log to info in case of flush on non-existing member (sonic-net#1669)
  > Revert "[Dynamic buffer calc] Bug fix: Remove PGs from an administratively down port. (sonic-net#1652)" (sonic-net#1676)
  > [Dynamic buffer calc] Bug fix: Remove PGs from an administratively down port. (sonic-net#1652)
  > [acl] Move ACL table constants to acltable.h (sonic-net#1671)
  > [nbrmgrd] added function to parse IP address from APP_DB (sonic-net#1672)
  > [MUX/PFCWD] Use in_ports for acls instead of seperate ACL table (sonic-net#1670)
  > [vog/systemlag] Voq lagid allocator (sonic-net#1603)
  > Add table descriptions for dynamic buffer calculation to the documents (sonic-net#1664)
  > [vstest/subintf] Add vs test case to validate processing sequence of APPL DB keys (sonic-net#1663)
  > Remove vxlanmgrd dependency on orchagent (sonic-net#1647)
  > Keep attribute order in bulk mode (sonic-net#1659)
  > [mux] VS test for neigh, route and fdb (sonic-net#1656)
  > [linksync] Netdev oper status determination using IFF_RUNNING (sonic-net#1568)
  > [portorch] parse on/off value from autoneg (sonic-net#1658)
  > [intfsorch] Create subport with the entry contains necessary attributes (sonic-net#1650)
  > [ci]: Purge swss before install (sonic-net#1654)
  > Update StateDB with error if state change failed, Update APP_DB in all state chg req (sonic-net#1662)
  > Added changes to handle dependency check in FdbSyncd and FpmSyncd for warm-boot (sonic-net#1556)
  > [synchronous mode] Add failure notification for SAI failures in synchronous mode (sonic-net#1596)
  > [acl] Enable VLAN ID qualifier for ACL rules (sonic-net#1648)
  > Updated PFCWD to use single ACL table for PFCWD and MUX (sonic-net#1620)
  > [orchagent] Increase SAI REDIS response timeout to support FW upgrade during init (Mellanox only). (sonic-net#1637)
  > [vstest/nhg]: use dvs_route fixture to make test_nhg more robust
  > [vstest]: add dvs_route fixture
  > [vstest/subintf] Update vs tests to validate physical port host interface vlan tag attribute (sonic-net#1634)
  > Remove useless header  in macsecorch (sonic-net#1628)
  > Add SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS counter, create new FlexCounter group (sonic-net#1600)
  > fixed unsupported resource issue (sonic-net#1641)
  > [test_virtual_chassis]: use wait_for to make test more robust (sonic-net#1640)
  > spell check fixes (sonic-net#1630)
  > [bufferorch] Handle NOT IMPLEMENTED status returned during set attr operation (sonic-net#1639)
  > [ci]: run vstest
  > [test_virtual_chassis]: use wait_for function to improve test robustness
  > [Mux] Neighbor handling based on FDB entry (sonic-net#1631)
  > [ci]: use build template (sonic-net#1633)
  > Log level change from ERR to INFO for fetch systemports issue (sonic-net#1632)
  > Migrate serdes programming to port serdes object (sonic-net#1611)
  > [tests] Remove legacy saiattributelist.h dependency (sonic-net#1608)
  > [buffermgr] Support maximum port headroom checking (sonic-net#1607)
  > Support shared headroom pool on top of dynamic buffer calculation (sonic-net#1581)
  > Fix the compiling errors in gcc9 (sonic-net#1621)
lguohan pushed a commit that referenced this issue Jun 21, 2021
Update sonic-utilities submodule. This include the following commits:

285960d [config]: Update environment file during config reload (#1673)
3f0ecd5 [config] Remove "reset failed" print lines from config reload (#1654) 
a1c8751 Make the soft-reboot available in the SONiC image on master (#1681)
45e7b71 [Mellanox] Add all results from saisdkdump to the techsupport on Mellanox switches (#1660)
carl-nokia pushed a commit to carl-nokia/sonic-buildimage that referenced this issue Aug 7, 2021
Update sonic-sairedis submodule and also update sonic-swss submodule as there are interdependent changes.

* src/sonic-sairedis 13474d1...bc58b0f (12):
  > Add gbsyncdmgrd; deprecate gbsyncd_startup.py (sonic-net#809)
  > Remove gbsyncd_start.sh (sonic-net#808)
  > [gbsyncd] Fix shebang in gbsyncd_startup.py; Make script executable (sonic-net#807)
  > [saiasiccmp] Add saiasiccmp tool to compare 2 asic views (sonic-net#791)
  > [configure] Add -Wno-psabi to remove "passing argument changed in GCC 7.1" (sonic-net#799)
  > Update FlexCounter.cpp, use m_pollInterval in MUTEX lock (sonic-net#797)
  > [vs] Add special warm boot logic to populate default attributes (sonic-net#796)
  > [ci]: add vstest (sonic-net#795)
  > [tests] Add macsec unittest (sonic-net#782)
  > [debian/control] libsairedis-dev depends on libzmq5-dev (sonic-net#794)
  > [ci]: use build template (sonic-net#793)
  > Rename duplicate file name (sonic-net#773)

* src/sonic-swss 0b0d24c...5adb73e (47):
  > Initialize system port type variable (sonic-net#1681)
  > [Dynamic Buffer Calc] Enhance the field checking in table handling (sonic-net#1680)
  > Handle the clear request for 'Q_SHARED_ALL' (sonic-net#1653)
  > [MuxOrch] FDB ageout safety check (sonic-net#1674)
  > Deactivate mirror session only when session status is true in updateLagMember (sonic-net#1666)
  > Revert "[buffermgr] Support maximum port headroom checking (sonic-net#1607)" (sonic-net#1675)
  > reduce severity of log to info in case of flush on non-existing member (sonic-net#1669)
  > Revert "[Dynamic buffer calc] Bug fix: Remove PGs from an administratively down port. (sonic-net#1652)" (sonic-net#1676)
  > [Dynamic buffer calc] Bug fix: Remove PGs from an administratively down port. (sonic-net#1652)
  > [acl] Move ACL table constants to acltable.h (sonic-net#1671)
  > [nbrmgrd] added function to parse IP address from APP_DB (sonic-net#1672)
  > [MUX/PFCWD] Use in_ports for acls instead of seperate ACL table (sonic-net#1670)
  > [vog/systemlag] Voq lagid allocator (sonic-net#1603)
  > Add table descriptions for dynamic buffer calculation to the documents (sonic-net#1664)
  > [vstest/subintf] Add vs test case to validate processing sequence of APPL DB keys (sonic-net#1663)
  > Remove vxlanmgrd dependency on orchagent (sonic-net#1647)
  > Keep attribute order in bulk mode (sonic-net#1659)
  > [mux] VS test for neigh, route and fdb (sonic-net#1656)
  > [linksync] Netdev oper status determination using IFF_RUNNING (sonic-net#1568)
  > [portorch] parse on/off value from autoneg (sonic-net#1658)
  > [intfsorch] Create subport with the entry contains necessary attributes (sonic-net#1650)
  > [ci]: Purge swss before install (sonic-net#1654)
  > Update StateDB with error if state change failed, Update APP_DB in all state chg req (sonic-net#1662)
  > Added changes to handle dependency check in FdbSyncd and FpmSyncd for warm-boot (sonic-net#1556)
  > [synchronous mode] Add failure notification for SAI failures in synchronous mode (sonic-net#1596)
  > [acl] Enable VLAN ID qualifier for ACL rules (sonic-net#1648)
  > Updated PFCWD to use single ACL table for PFCWD and MUX (sonic-net#1620)
  > [orchagent] Increase SAI REDIS response timeout to support FW upgrade during init (Mellanox only). (sonic-net#1637)
  > [vstest/nhg]: use dvs_route fixture to make test_nhg more robust
  > [vstest]: add dvs_route fixture
  > [vstest/subintf] Update vs tests to validate physical port host interface vlan tag attribute (sonic-net#1634)
  > Remove useless header  in macsecorch (sonic-net#1628)
  > Add SAI_INGRESS_PRIORITY_GROUP_STAT_DROPPED_PACKETS counter, create new FlexCounter group (sonic-net#1600)
  > fixed unsupported resource issue (sonic-net#1641)
  > [test_virtual_chassis]: use wait_for to make test more robust (sonic-net#1640)
  > spell check fixes (sonic-net#1630)
  > [bufferorch] Handle NOT IMPLEMENTED status returned during set attr operation (sonic-net#1639)
  > [ci]: run vstest
  > [test_virtual_chassis]: use wait_for function to improve test robustness
  > [Mux] Neighbor handling based on FDB entry (sonic-net#1631)
  > [ci]: use build template (sonic-net#1633)
  > Log level change from ERR to INFO for fetch systemports issue (sonic-net#1632)
  > Migrate serdes programming to port serdes object (sonic-net#1611)
  > [tests] Remove legacy saiattributelist.h dependency (sonic-net#1608)
  > [buffermgr] Support maximum port headroom checking (sonic-net#1607)
  > Support shared headroom pool on top of dynamic buffer calculation (sonic-net#1581)
  > Fix the compiling errors in gcc9 (sonic-net#1621)
carl-nokia pushed a commit to carl-nokia/sonic-buildimage that referenced this issue Aug 7, 2021
Update sonic-utilities submodule. This include the following commits:

285960d [config]: Update environment file during config reload (sonic-net#1673)
3f0ecd5 [config] Remove "reset failed" print lines from config reload (sonic-net#1654) 
a1c8751 Make the soft-reboot available in the SONiC image on master (sonic-net#1681)
45e7b71 [Mellanox] Add all results from saisdkdump to the techsupport on Mellanox switches (sonic-net#1660)
theasianpianist pushed a commit to theasianpianist/sonic-buildimage that referenced this issue Feb 5, 2022
Observe persistent failure of newly added vs test on testing code change in intfmgrd after migrating to azure pipeline. sonic-net#1521 passed in legacy Jenkins pipeline.

What we found is that the code change in the swss compiled with PR change is not properly installed in vs docker for a direct installation using dpkg -i. This is confirmed by verifying pipeline artifacts that md5sum value of /user/bin/intfmgrd changes if we install swss deb aritfact into the docker vs artifact, while that of /usr/bin/orchangent stays unchanged.

md5sum before swss install in vs docker

# md5sum /usr/bin/orchagent
28307a7805ea6f3bc5057c0257bf46e6  /usr/bin/orchagent
 
# md5sum /usr/bin/intfmgrd  
fa2b06e20be683286adb47c55635a86d  /usr/bin/intfmgrd
md5sum after swss install

# dpkg -i swss_1.0.0_amd64.deb 
(Reading database ... 19180 files and directories currently installed.)
Preparing to unpack swss_1.0.0_amd64.deb ...
Unpacking swss (1.0.0) over (1.0.0) ...
Setting up swss (1.0.0) ...
# md5sum /usr/bin/orchagent
28307a7805ea6f3bc5057c0257bf46e6  /usr/bin/orchagent

# md5sum /usr/bin/intfmgrd 
e959340709e7aedd7489e69dfd19768f  /usr/bin/intfmgrd

Signed-off-by: Wenda Ni <[email protected]>
taras-keryk pushed a commit to taras-keryk/sonic-buildimage that referenced this issue Apr 28, 2022
…net#1654)

#### What I did
I removed the following line from the `config reload` flow....

```
click.echo("Resetting failed status on {}".format(service))
```

This is called in the `_reset_failed_services()` function which is used to reset the systemd restart timers on services so that it doesn't prevent them from starting from the config reload. 

The above line results in the following lines being printed to the terminal when `config reload` is run

```
Resetting failed status on <service>.service
```

This gives the impression that the listed services failed at some point in their lifetime which is untrue and misleading to print to the terminal (in addition many rich-text terminals mark keywords such as "failed" red which further increases the implied severity of these log lines). I do not believe this operation needs any verbosity as it is a very simple operation that makes no user-visible changes thus the user should not need to be notified of it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants