[config-engine] minigraph.py refactoring #448

taoyl-ms · 2017-03-28T01:47:51Z

There is inconsistency among the interface of different field in current minigraph.py, and that is causing confusion and leads to misconfiguration in some of our j2 templates. This PR makes the following modification to the minigraph.py output:

Consolidate the port/interface output. Now we have minigraph_portchannels and minigraph_vlans which are dictionaries with port channel name / vlan name as key, and portchannel members / vlan members and vlan ids as value. The configuration that needs to be generated by iterating though all portchannels / vlans should use these fields.
On the other hand, we have minigraph_portchannel_interfaces and minigraph_vlan_interfaces for the ip address configured on portchannels and vlans. Those fields are formatted in lists and have the similar format with minigraph_interfaces. Those fields should be used when needs to iterate though all ip addresses / networks.
This PR also fixes several templates where they iterate through the wrong field and have potential issue when there are multiple addresses configured on a single port.
To keep it consistent, front panel ports now also have a minigraph_ports dictionary, together with the existing minigraph_interfaces list. The current values of this dictionary are port names and alias, and we can consider extending it with front panel indices.
In the previous version, name and alias in minigraph_interfaces are opposite from which in port_config.ini. This is fixed in the new minigraph_ports. (name - sonic name; alias - vendor-specific name)
minigraph_interfaces will no longer have name and alias, but only a attachto where port sonic name will be put into.
As alias information is included in minigraph_ports, alias_map field is no longer needed.

taoyl-ms · 2017-03-28T01:53:01Z

Sample outputs after this PR:

minigraph_ports:
{'Ethernet0': {'alias': 'fortyGigE0/0', 'name': 'Ethernet0'}, 'Ethernet4': {'alias': 'fortyGigE0/4', 'name': 'Ethernet4'}}
minigraph_vlans:
{'Vlan1000': {'name': 'Vlan1000', 'members': ['Ethernet0', 'Ethernet4']}}
minigraph_portchannels:
{'PortChannel01': {'name': 'PortChannel13', 'members': ['Ethernet8']}}
minigraph_interfaces / minigraph_portchannel_interfaces / minigraph_vlan_interfaces (in the same format):
[{'subnet': IPv4Network('10.0.0.56/31'), 'peer_addr': IPv4Address('10.0.0.57'), 'addr': IPv4Address('10.0.0.56'), 'mask': IPv4Address('255.255.255.254'), 'attachto': 'PortChannel01', 'prefixlen': 31}, {'subnet': IPv6Network('fc00::70/126'), 'peer_addr': IPv6Address('fc00::72'), 'addr': IPv6Address('fc00::71'), 'mask': '126', 'attachto': 'PortChannel01', 'prefixlen': 126}] #Closed

stcheng · 2017-03-28T02:12:13Z

Based on the sample output, when will the SONiC alias used? when will the 'fortyGigE0/4' used? Will there be any replacement in the output? #Resolved

stcheng · 2017-03-28T02:13:14Z

Could you also add some basic tests to ensure some output? and attach a generated t0 /etc/network/interfaces file here in the comment #Resolved

stcheng · 2017-03-28T02:18:21Z

files/image_config/interfaces/interfaces.j2

 {% endfor %}
-{% for vlan_interface in minigraph_vlan_interfaces|unique_name %}


could you also remove the unique_name filter in sonic-cfggen? #WontFix

I feel it's OK to keep it?

In reply to: 108321887 [](ancestors = 108321887)

stcheng · 2017-03-28T02:21:40Z

files/image_config/interfaces/interfaces.j2

+{% for vlan in minigraph_vlans.keys()|sort %}
+auto {{ vlan }}
+{% endfor %}
+


i don't know if it will work or not to list all the vlan interfaces aside and the ip addresses aside. if doing in this way, move the 'bridge_ports none' to here so that even if we have multiple IPs, we don't have multiple 'bridge_ports none' lines. #Resolved

it turns out to be that bridge_ports should follow iface line. thanks @taoyl-ms . let's stick auto stanza with iface stanza together for better readability. #Resolved

stcheng · 2017-03-28T02:29:56Z

files/image_config/interfaces/interfaces.j2

+{% for pc in minigraph_portchannels.keys()|sort %}
+auto {{ pc }}
+allow-hotplug {{ pc }}
+{% endfor %}


i don't know if it is good to separate the iface stanza with the auto stanzas. #Resolved

stcheng

comments left.

lguohan · 2017-03-29T08:22:46Z

does this pass the sonic-cfggen unit test? Is it running? #Resolved

taoyl-ms · 2017-03-29T21:24:07Z

The only output where 'fortyGigE0/4' will present is in the "alias" field of minigraph_ports. All the other fields use SONiC name.

In reply to: 289641473 [](ancestors = 289641473)

taoyl-ms · 2017-03-29T21:24:53Z

It seems the test is not running automatically after we moved from deb to whl. Just edited slave.mk to reenable it.

In reply to: 290018772 [](ancestors = 290018772)

taoyl-ms · 2017-03-29T21:47:08Z

Upload a sample /etc/network/interfaces file for T0 after this refactor.
interfaces.txt
#Resolved

taoyl-ms · 2017-03-29T22:17:50Z

All CR comments resolved. Please help review. #Closed

taoyl-ms · 2017-03-30T00:25:20Z

This will resolve #409

stcheng · 2017-03-30T00:38:35Z

error: [Errno 2] No such file or directory: 'build/bdist.linux-x86_64/wheel/openconfig_acl.py'
do you know why this error appears?

stcheng · 2017-03-30T01:35:59Z

resolve the conflicts?

Which was introduced in #448.

* [snmp] Fix a bug in SNMP alias mapping Which was introduced in #448.

* Adjusting FRR's jinja template to meet latest sonic-cfgen requirements. Basically, i'm just extending sonic-net#448 changes into FRR. * Eliminate FRR's integrated-config file to prevent daemons from bypassing their own config files. FRR daemons now default to an integrated-config file for config-parsing purposes. But we are still interested in having each daemon looking in their specific config file (bgpd.conf, zebra.conf, etc). So here i'm just deleting this integrating-config file to prevent FRR from running from a bogus config-file. RB= G=lnos-reviewers R=ntrianta,rjonnadu,rmolina,sfardeen,zxu A=

#895) * Adjusting FRR's jinja template to meet latest sonic-cfgen requirements. Basically, i'm just extending #448 changes into FRR. * Eliminate FRR's integrated-config file to prevent daemons from bypassing their own config files. FRR daemons now default to an integrated-config file for config-parsing purposes. But we are still interested in having each daemon looking in their specific config file (bgpd.conf, zebra.conf, etc). So here i'm just deleting this integrating-config file to prevent FRR from running from a bogus config-file. RB= G=lnos-reviewers R=ntrianta,rjonnadu,rmolina,sfardeen,zxu A=

…er (#903) * Fixing a couple of issues to enable FRR to run with latest SONiC code. * Adjusting FRR's jinja template to meet latest sonic-cfgen requirements. Basically, i'm just extending #448 changes into FRR. * Eliminate FRR's integrated-config file to prevent daemons from bypassing their own config files. FRR daemons now default to an integrated-config file for config-parsing purposes. But we are still interested in having each daemon looking in their specific config file (bgpd.conf, zebra.conf, etc). So here i'm just deleting this integrating-config file to prevent FRR from running from a bogus config-file. * Allows interactive session with the slave docker-container after conclusion of the building process. Proposed changes provide a more direct access to the building pipeline, by allowing users to launch consecutive compilation tasks from the same slave container.

* 1b0d609 2019-04-24 [SAI] Advance submodule to v1.4.1 (sonic-net#450) [Marian Pritsak] * 3d87ad1 2019-04-23 Add metadata serialization support for buffer pool stats (sonic-net#448) [Wenda Ni] * a17e54e 2019-04-23 [SAI]: Move SAI pointer to v1.4 (sonic-net#447) [Shuotian Cheng] * f8950b7 2019-04-23 Add warm-boot feature processing for wedge100bf_32x/65x platforms (sonic-net#434) [Pavlo Yadvichuk] * e9e9dc3 2019-04-18 Add a comment to fix the docker issue (sonic-net#442) [pavel-shirshov] Signed-off-by: Volodymyr Samotiy <[email protected]>

* update to SAI 1.2 * set LAG PORT_VLAN_ID instead of member port VLAN id * [test]: Add test for the CRM feature Signed-off-by: Volodymyr Samotiy <[email protected]> * [test]: Add test for the CRM feature * Fix review comments Signed-off-by: Volodymyr Samotiy <[email protected]> * update to SAI 1.2 * set LAG PORT_VLAN_ID instead of member port VLAN id * [test]: Add ACL test for the CRM feature Signed-off-by: Volodymyr Samotiy <[email protected]> * update to SAI 1.2 * set LAG PORT_VLAN_ID instead of member port VLAN id * [test]: Add test for the CRM feature * Use static neighbors instead of configuring server side * Change polling interval value to 1 second * Rename "nexthop_group_object" resource to "nexthop_group" Signed-off-by: Volodymyr Samotiy <[email protected]> * [crmorch]: Implement CRM feature (sonic-net#447) Signed-off-by: Volodymyr Samotiy <[email protected]> * [lua]: Only set the entry values when they are not nil (sonic-net#446) * [lua]: Only set the entries when they are not nil Signed-off-by: Sihui Han <[email protected]> * udpate * further fix the detection time nil error Signed-off-by: Sihui Han <[email protected]> * [test]: Add test for the CRM feature * Fix fails in some test cases Signed-off-by: Volodymyr Samotiy <[email protected]> * [test]: Add test for the CRM feature * Fix fails in IPv6 test cases Signed-off-by: Volodymyr Samotiy <[email protected]>

Signed-off-by: Wenda Ni <[email protected]>

Change in this update: b75aab7 [swss-common] Add LINKMGR CFG and MUX LINKMGR state table names (sonic-net#421) 4a77d1c [ci]: add vstest (sonic-net#459) 07258a6 [ci]: use build template (sonic-net#457) ddcae3e runRedisScript api to process integer returned by script run in the redis (sonic-net#447) 33d89c7 [systemlag] Schema defs for system lag (sonic-net#448) af01f37 spell check fixes (sonic-net#456) 7afd43d Update to make getNamespaces() API at par with the get_ns_list() swssdk-py API. (sonic-net#455) signed-off-by: Tamer Ahmed <[email protected]>

Change in this update: b75aab7 [swss-common] Add LINKMGR CFG and MUX LINKMGR state table names (#421) 4a77d1c [ci]: add vstest (#459) 07258a6 [ci]: use build template (#457) ddcae3e runRedisScript api to process integer returned by script run in the redis (#447) 33d89c7 [systemlag] Schema defs for system lag (#448) af01f37 spell check fixes (#456) 7afd43d Update to make getNamespaces() API at par with the get_ns_list() swssdk-py API. (#455) signed-off-by: Tamer Ahmed <[email protected]>

Change in this update: b75aab7 [swss-common] Add LINKMGR CFG and MUX LINKMGR state table names (sonic-net#421) 4a77d1c [ci]: add vstest (sonic-net#459) 07258a6 [ci]: use build template (sonic-net#457) ddcae3e runRedisScript api to process integer returned by script run in the redis (sonic-net#447) 33d89c7 [systemlag] Schema defs for system lag (sonic-net#448) af01f37 spell check fixes (sonic-net#456) 7afd43d Update to make getNamespaces() API at par with the get_ns_list() swssdk-py API. (sonic-net#455) signed-off-by: Tamer Ahmed <[email protected]>

taoyl-ms added 3 commits March 25, 2017 01:36

Refactor minigraph.py

efb7e95

Update j2 templates according to minigraph.py change

424d931

Remove deprecated options in sonic-cfggen

cbc5ec3

stcheng reviewed Mar 28, 2017

View reviewed changes

stcheng suggested changes Mar 28, 2017

View reviewed changes

stcheng assigned taoyl-ms Mar 28, 2017

stcheng added the Enhancement ➕ label Mar 28, 2017

Resolve CR comments

b5bd1b0

Update config engine tests

2f80407

stcheng approved these changes Mar 30, 2017

View reviewed changes

Split test and build wheel process

4a08e55

taoyl-ms added 2 commits March 29, 2017 20:42

Merge branch 'master' into taoyl/mini_refactor

160b535

Fix test break

06c3346

taoyl-ms merged commit fed908f into sonic-net:master Mar 30, 2017

taoyl-ms added a commit that referenced this pull request Apr 3, 2017

[snmp] Fix a bug in SNMP alias mapping

aaa5773

Which was introduced in #448.

taoyl-ms mentioned this pull request Apr 3, 2017

[snmp] Fix a bug in SNMP alias mapping #473

Merged

taoyl-ms added a commit that referenced this pull request Apr 4, 2017

[snmp] Fix a bug in SNMP alias mapping (#473)

8391d7f

* [snmp] Fix a bug in SNMP alias mapping Which was introduced in #448.

volodymyrsamotiy mentioned this pull request Apr 25, 2019

[sairedis]: Update sonic-sairedis pointer and Mellanox SAI/SDK/FW #2826

Merged

dmytroxshevchuk pushed a commit to dmytroxshevchuk/sonic-buildimage that referenced this pull request Aug 31, 2020

Add metadata serialization support for buffer pool stats (sonic-net#448)

3d87ad1

Signed-off-by: Wenda Ni <[email protected]>

tahmed-dev mentioned this pull request Feb 18, 2021

[sonic-swss-common] Submodule Update #6812

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[config-engine] minigraph.py refactoring #448

[config-engine] minigraph.py refactoring #448

taoyl-ms commented Mar 28, 2017

taoyl-ms commented Mar 28, 2017 •

edited

Loading

stcheng commented Mar 28, 2017 •

edited by taoyl-ms

Loading

stcheng commented Mar 28, 2017 •

edited by taoyl-ms

Loading

stcheng Mar 28, 2017 •

edited by taoyl-ms

Loading

taoyl-ms Mar 29, 2017

stcheng Mar 28, 2017 •

edited by taoyl-ms

Loading

stcheng Mar 29, 2017 •

edited by taoyl-ms

Loading

stcheng Mar 28, 2017 •

edited by taoyl-ms

Loading

stcheng left a comment

lguohan commented Mar 29, 2017 •

edited by taoyl-ms

Loading

taoyl-ms commented Mar 29, 2017

taoyl-ms commented Mar 29, 2017

taoyl-ms commented Mar 29, 2017 •

edited

Loading

taoyl-ms commented Mar 29, 2017 •

edited

Loading

taoyl-ms commented Mar 30, 2017 •

edited

Loading

stcheng commented Mar 30, 2017

stcheng commented Mar 30, 2017

		{% endfor %}
		{% for vlan_interface in minigraph_vlan_interfaces\|unique_name %}

[config-engine] minigraph.py refactoring #448

[config-engine] minigraph.py refactoring #448

Conversation

taoyl-ms commented Mar 28, 2017

taoyl-ms commented Mar 28, 2017 • edited Loading

stcheng commented Mar 28, 2017 • edited by taoyl-ms Loading

stcheng commented Mar 28, 2017 • edited by taoyl-ms Loading

stcheng Mar 28, 2017 • edited by taoyl-ms Loading

Choose a reason for hiding this comment

taoyl-ms Mar 29, 2017

Choose a reason for hiding this comment

stcheng Mar 28, 2017 • edited by taoyl-ms Loading

Choose a reason for hiding this comment

stcheng Mar 29, 2017 • edited by taoyl-ms Loading

Choose a reason for hiding this comment

stcheng Mar 28, 2017 • edited by taoyl-ms Loading

Choose a reason for hiding this comment

stcheng left a comment

Choose a reason for hiding this comment

lguohan commented Mar 29, 2017 • edited by taoyl-ms Loading

taoyl-ms commented Mar 29, 2017

taoyl-ms commented Mar 29, 2017

taoyl-ms commented Mar 29, 2017 • edited Loading

taoyl-ms commented Mar 29, 2017 • edited Loading

taoyl-ms commented Mar 30, 2017 • edited Loading

stcheng commented Mar 30, 2017

stcheng commented Mar 30, 2017

taoyl-ms commented Mar 28, 2017 •

edited

Loading

stcheng commented Mar 28, 2017 •

edited by taoyl-ms

Loading

stcheng commented Mar 28, 2017 •

edited by taoyl-ms

Loading

stcheng Mar 28, 2017 •

edited by taoyl-ms

Loading

stcheng Mar 28, 2017 •

edited by taoyl-ms

Loading

stcheng Mar 29, 2017 •

edited by taoyl-ms

Loading

stcheng Mar 28, 2017 •

edited by taoyl-ms

Loading

lguohan commented Mar 29, 2017 •

edited by taoyl-ms

Loading

taoyl-ms commented Mar 29, 2017 •

edited

Loading

taoyl-ms commented Mar 29, 2017 •

edited

Loading

taoyl-ms commented Mar 30, 2017 •

edited

Loading