Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-ASIC implementation #3888

Merged
merged 62 commits into from
Mar 31, 2020

Conversation

SuvarnaMeenakshi
Copy link
Contributor

@SuvarnaMeenakshi SuvarnaMeenakshi commented Dec 12, 2019

- What I did
To support SONiC on multi-asic platforms, the approach is to create SONiC docker stack for each ASIC. Each SONiC docker stack is separated by network namespaces for each ASIC.
Each network namespace will comprise of key dockers :
database
swss
syncd
bgp
teamd
interfaces-config
lldp

- How I did it
Main changes done to support SONiC on multi-asic platform are:

  1. systemd service template for multi-instance service is added to start N number of systemd services where N is the number of ASICs in the platform.
  2. Changes in FRR config to support BGP between the ASIC namespaces. To support this, a new type is added in DEVICE_METADATA called "InternalFrontend" and "InternalBackend" to differentiate between a front-end ASIC and back-end ASIC.
  3. Change in docker_img_ctl.j2 to support starting dockers with a specific ASIC instance number which will be passed to the <docker_name>.sh script by the systemd service.
  4. Apart from multiple instances of database docker, a global database docker will exist in the host namespace.

- How to verify it
Build a multi-asic VS image.
Follow instruction (in README.vsd) to bring up multi-asic VS image by adding the number of ASICs in device/platform/asic.conf and also adding a specific topology.sh in HWSKU directory to create a virtual ASIC connectivity in the VS.
Once the VS image comes up, ensure the dockers come up and verify BGP sessions between ASICs.

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

Lawrence Lee and others added 14 commits December 12, 2019 10:48
* create systemd target to start/stop all containers in a specific namespace at once

Signed-off-by: Lawrence Lee <[email protected]>
* add multi-instance option to redis-cli
* create database template service
* change docker_image_ctl to use new redis sockets
* increase database service timeout duration to 120s

Signed-off-by: Lawrence Lee <[email protected]>
* add instance number option and unix socket option to docker_image_ctl for swss
* change swss service to service template
* add instance number option and unix socket option to swss.sh
* force all redis-cli calls through redis-cli wrapper

Signed-off-by: Lawrence Lee <[email protected]>
* add syncd service template
* add instance number argument for syncd script
* add rsyslog service template compatible with network namespaces
* change rsyslog_config to use database instance unix socket

Signed-off-by: Lawrence Lee <[email protected]>
(cherry picked from commit e871f11)
* add wrapper for 'sudo ip netns exec' command

Signed-off-by: Lawrence Lee <[email protected]>
(cherry picked from commit 85745b7)
* increase from 2GB to 8GB to prevent OOM crashes
* necessary because multi-ASIC devices have increased memory requirements

Signed-off-by: Lawrence Lee <[email protected]>
(cherry picked from commit 845778e)
* add bgp service template
* create /etc/sonic/frr* for multiple instances of bgp docker container

[frr docker]: add multi-ASIC support to bgpd.conf
* add `redistribute connected` option to advertise routes to local interfaces
* add `all` option to next-hop-self
* add route-map to hide internal routes
* add route-map to allow duplicate frontend router id

Signed-off-by: Lawrence Lee <[email protected]>
(cherry picked from commit 4ea01b7)
* change teamd service file to template
* force teamd to restart when swss does

Signed-off-by: Lawrence Lee <[email protected]>
(cherry picked from commit 0297279)
* set submodule to point at newest commit

Signed-off-by: Lawrence Lee <[email protected]>
(cherry picked from commit ebab66d)
* create service template
* enable template instances in sonic_debian_extension
* use iproute2 in interfaces-config script if working with network namespaces
* make swss multi-instance dependent on multi-instance interfaces config

Signed-off-by: Lawrence Lee <[email protected]>
(cherry picked from commit 08af0db)
* use 'asic' as namespace prefix instead of 'namespace' in accordance
with PR sonic-net#3269

Signed-off-by: Lawrence Lee <[email protected]>
(cherry picked from commit c1444e3)
@lguohan
Copy link
Collaborator

lguohan commented Jan 29, 2020

@pavel-shirshov , can you check the bgp template?

@@ -0,0 +1,13 @@
[Unit]
Description=Update rsyslog configuration
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i do not understand why when need multi-instance for rsyslog.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed multiple instances for rsyslog service. Changes required for rsyslog service support for multi-asic platform will be added in a different PR.

@@ -83,7 +79,11 @@ start_peer_and_dependent_services() {
check_warm_boot

if [[ x"$WARM_BOOT" != x"true" ]]; then
/bin/systemctl start ${PEER}
if [[ ! -z $DEV ]]; then
/bin/systemctl start ${PEER}@$DEV
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need consistent spacing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

@@ -0,0 +1,16 @@
[Unit]
Description=BGP container
#Requires=updategraph.service
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SuvarnaMeenakshi @lguohan : we don't need this dependency for Multi Instance ? Single Instance still have this dependency.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

support for updategraph and config-setup service for multi-asic platform will be added in a separate PR. As updategraph service is not updated to support multiple config_db, this dependency cannot be added here.

@qiluo-msft qiluo-msft self-assigned this Jan 31, 2020
@qiluo-msft
Copy link
Collaborator

qiluo-msft commented Jan 31, 2020

Please solve the conflicts #Closed

@SuvarnaMeenakshi
Copy link
Contributor Author

retest vsimage please

Copy link

@paulnice paulnice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good for me. Please wait for others.

@lguohan
Copy link
Collaborator

lguohan commented Mar 20, 2020

retest vsimage please

@SuvarnaMeenakshi
Copy link
Contributor Author

retest vsimage please

1 similar comment
@SuvarnaMeenakshi
Copy link
Contributor Author

retest vsimage please

@lguohan
Copy link
Collaborator

lguohan commented Mar 24, 2020

pending on vsimage test passing.

@SuvarnaMeenakshi
Copy link
Contributor Author

retest vsimage please

1 similar comment
@SuvarnaMeenakshi
Copy link
Contributor Author

retest vsimage please

@SuvarnaMeenakshi SuvarnaMeenakshi merged commit 4b8067e into sonic-net:master Mar 31, 2020
abdosi pushed a commit that referenced this pull request Apr 2, 2020
Changes made to support multi-asic platform. Added multi-instance support for swss, syncd, database, bgp, teamd and lldp.
abdosi added a commit that referenced this pull request Apr 3, 2020
abdosi pushed a commit that referenced this pull request Apr 15, 2020
Changes made to support multi-asic platform. Added multi-instance support for swss, syncd, database, bgp, teamd and lldp.
tiantianlv pushed a commit to SONIC-DEV/sonic-buildimage that referenced this pull request Apr 24, 2020
Changes made to support multi-asic platform. Added multi-instance support for swss, syncd, database, bgp, teamd and lldp.
tiantianlv pushed a commit to SONIC-DEV/sonic-buildimage that referenced this pull request Apr 24, 2020
tiantianlv pushed a commit to SONIC-DEV/sonic-buildimage that referenced this pull request Apr 24, 2020
Changes made to support multi-asic platform. Added multi-instance support for swss, syncd, database, bgp, teamd and lldp.
@wendani
Copy link
Contributor

wendani commented May 9, 2020

Multi-ASIC is not part of the 201911 release https://github.com/Azure/SONiC/wiki/Release-Progress-Tracking-201911

Why do we include it in 201911 @abdosi ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.