Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[201911-Mellanox] SKU creator Tool (#1163) #1250

Merged

Conversation

madhanmellanox
Copy link
Contributor

- What I did
I added a new script file sonic_sku_create.py to generate a new HWSKU based on XML file or Minigraph. But, focus is on creating HWSKU based on XML file and not on the minigraph and l2 mode. I also added a unit test pyTest script sku_create_test.py in sonic-utilities-test folder to test the script in Unit Testing mode.

Motivation: To create SKU for Mellanox platforms based on XML file with Port related inputs and also through Minigraph file.
This tool also allows to split a port or unsplit ports based on configuration which modifies the port related information in port_config.ini and config_db.json

usage: sonic_sku_create.py [-h] [-v]
(-f FILE | -m [MINIGRAPH_FILE] | -j JSON_FILE | -pp PORT_SPLIT PORT_SPLIT)
[-b BASE] [-r] [-k HWSKU] [-p] [-vv]

Create a new SKU

optional arguments:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-f FILE, --file FILE SKU definition from xml file. -f OR -m or -j must be provided when creating a new SKU
-m [MINIGRAPH_FILE], --minigraph_file [MINIGRAPH_FILE]
SKU definition from minigraph file. -f OR -m or -j must be provided when creating a new SKU
-j JSON_FILE, --json_file JSON_FILE
SKU definition from config_db.json file. -f OR -m OR -j must be provided when creating a new SKU
-pp PORT_SPLIT PORT_SPLIT, --port_split PORT_SPLIT PORT_SPLIT
port name and split
-b BASE, --base BASE SKU base definition
-r, --remove Remove SKU folder
-k HWSKU, --hwsku HWSKU
SKU name to be used when creating a new SKU or for L2 configuration mode
-p, --print Print port_config.ini without creating a new SKU
-vv, --verbose Verbose output

- How I did it
I did it by adding these files which are required for new SKU generation in the forthcoming releases.

- How to verify it
I verified all the options of sonic_sku_create.py file and tested them thoroughly as well as created a test case in sku_create_test.py file to test the functionality of sonic_sku_create.py in Unit Test mode.

- Previous command output (if the output of a command-line utility has changed)
None

- New command output (if the output of a command-line utility has changed)

admin@r-qa-sw-eth-2134:/tmp$ sudo python sonic_sku_create.py -f Mellanox-SN2700-C28D10.xml
MSN2700 -  even front panel ports ( 32 ) are not allowed to split by 4
Error - Illegal split by 4
admin@r-qa-sw-eth-2134:/tmp$ sudo python sonic_sku_create.py -f Mellanox-SN2700-C28D11.xml
SKU directory: /usr/share/sonic/device/x86_64-mlnx_msn2700-r0/Mellanox-SN2700-C28D11/ already exists
 Please use -r flag to remove the SKU dir first
admin@r-qa-sw-eth-2134:/tmp$
admin@r-qa-sw-eth-2134:/tmp$ sudo python sonic_sku_create.py -r -f Mellanox-SN2700-C28D11.xml
You are about to permanently delete the SKU /usr/share/sonic/device/x86_64-mlnx_msn2700-r0/Mellanox-SN2700-C28D11/ !!
Do you want to continue (Yes/No)?Yes
SKU directory: /usr/share/sonic/device/x86_64-mlnx_msn2700-r0/Mellanox-SN2700-C28D11/ was removed
admin@r-qa-sw-eth-2134:/tmp$ sudo python sonic_sku_create.py  -f Mellanox-SN2700-C28D11.xml
Created a new sku (Location: /usr/share/sonic/device/x86_64-mlnx_msn2700-r0/Mellanox-SN2700-C28D11/)
admin@r-qa-sw-eth-2134:/tmp$
admin@r-qa-sw-eth-2134:/tmp$ sudo python sonic_sku_create.py -m ams20-0101-0623-08t0.xml
SKU directory: /usr/share/sonic/device/x86_64-mlnx_msn2700-r0/Mellanox-SN2700-D48C8/ already exists
 Please use -r flag to remove the SKU dir first
admin@r-qa-sw-eth-2134:/tmp$ sudo python sonic_sku_create.py -r -m ams20-0101-0623-08t0.xml
You are about to permanently delete the SKU /usr/share/sonic/device/x86_64-mlnx_msn2700-r0/Mellanox-SN2700-D48C8/ !!
Do you want to continue (Yes/No)?Yes
SKU directory: /usr/share/sonic/device/x86_64-mlnx_msn2700-r0/Mellanox-SN2700-D48C8/ was removed
admin@r-qa-sw-eth-2134:/tmp$ sudo python sonic_sku_create.py -m ams20-0101-0623-08t0.xml
Created a new sku (Location: /usr/share/sonic/device/x86_64-mlnx_msn2700-r0/Mellanox-SN2700-D48C8/)
admin@r-qa-sw-eth-2134:/tmp$
admin@r-qa-sw-eth-2134:/tmp$ sudo python sonic_sku_create.py -j config_db_fra.json
Created a new sku (Location: /usr/share/sonic/device/x86_64-mlnx_msn2700-r0/Mellanox-SN2700-test/)
admin@r-qa-sw-eth-2134:/tmp$ sudo python sonic_sku_create.py -pp Ethernet0 4x25
['name', 'lanes', 'alias', 'index']
port_config.ini file does not contain all fields, Exiting...
admin@r-qa-sw-eth-2134:/tmp$
admin@r-qa-sw-eth-2134:/tmp$ sudo python sonic_sku_create.py -pp Ethernet0 4x25 -vv
ARGS:  Namespace(base=None, file=None, hwsku=None, json_file=None, minigraph_file=None, port_split=['Ethernet0', '4x25'], print=False, remove=False, verbose=True)
['name', 'lanes', 'alias', 'index', 'speed']
msn2700_specific -> Removing  ['Ethernet5', [1, 2, 3, 4], 'etp2a', '5', '50000']
msn2700_specific -> Removing  ['Ethernet6', [1, 2, 3, 4], 'etp2b', '6', '50000']
MSN2700 - Front panel port  2  should be removed due to port  1 Split by 4
Error - Illegal split by 4
admin@r-qa-sw-eth-2134:/tmp$
madhan@arc-build-server:/builds2/madhan/utilitiescloneskucreate0714/sonic-utilities/sonic-utilities-tests$ sudo pytest -v sku_create_test.py -s
==================================================================== test session starts ====================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.0, py-1.5.2, pluggy-0.3.1 -- /usr/bin/python2
cachedir: .cache
rootdir: /builds2/madhan/utilitiescloneskucreate0714/sonic-utilities/sonic-utilities-tests, inifile: pytest.ini
collected 1 items

sku_create_test.py::TestSkuCreate::test_no_param Created a new sku (Location: /builds2/madhan/utilitiescloneskucreate0714/sonic-utilities/sonic-utilities-tests/sku_create_input/Mellanox-SN2700-D48C8_NEW/)

('Output file: ', '/builds2/madhan/utilitiescloneskucreate0714/sonic-utilities/sonic-utilities-tests/sku_create_input/Mellanox-SN2700-D48C8_NEW/port_config.ini', 'exists. SUCCESS!')
('Output file: ', '/builds2/madhan/utilitiescloneskucreate0714/sonic-utilities/sonic-utilities-tests/sku_create_input/Mellanox-SN2700-D48C8_NEW/port_config.ini', ' and model file: ', '/builds2/madhan/utilitiescloneskucreate0714/sonic-utilities/sonic-utilities-tests/sku_create_input/Mellanox-SN2700-D48C8/port_config.ini', 'contents are same. SUCCESS!')
PASSEDTEARDOWN


================================================================= 1 passed in 0.47 seconds ==================================================================
madhan@arc-build-server:/builds2/madhan/utilitiescloneskucreate0714/sonic-utilities/sonic-utilities-tests$

I added a new script file sonic_sku_create.py to generate a new HWSKU based on XML file or Minigraph. But, focus is on creating HWSKU based on XML file and not on the minigraph and l2 mode. I also added a unit test pyTest script sku_create_test.py in sonic-utilities-test folder to test the script in Unit Testing mode.
Motivation: To create SKU for Mellanox platforms based on XML file with Port related inputs and also through Minigraph file.
This tool also allows to split a port or unsplit ports based on configuration which modifies the port related information in port_config.ini and config_db.json

usage: sonic_sku_create.py [-h] [-v]
(-f FILE | -m [MINIGRAPH_FILE] | -j JSON_FILE | -pp PORT_SPLIT PORT_SPLIT)
[-b BASE] [-r] [-k HWSKU] [-p] [-vv]

Create a new SKU

optional arguments:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-f FILE, --file FILE SKU definition from xml file. -f OR -m or -j must be provided when creating a new SKU
-m [MINIGRAPH_FILE], --minigraph_file [MINIGRAPH_FILE]
SKU definition from minigraph file. -f OR -m or -j must be provided when creating a new SKU
-j JSON_FILE, --json_file JSON_FILE
SKU definition from config_db.json file. -f OR -m OR -j must be provided when creating a new SKU
-pp PORT_SPLIT PORT_SPLIT, --port_split PORT_SPLIT PORT_SPLIT
port name and split
-b BASE, --base BASE SKU base definition
-r, --remove Remove SKU folder
-k HWSKU, --hwsku HWSKU
SKU name to be used when creating a new SKU or for L2 configuration mode
-p, --print Print port_config.ini without creating a new SKU
-vv, --verbose Verbose output
@jleveque
Copy link
Contributor

@madhanmellanox: 201911 PR test is now fixed. The build failure appears to be real. Is this because you need to delete the directory first, like you did in master?

@madhanmellanox
Copy link
Contributor Author

@jleveque I have handled deletion of output directory before running the test script in this PR as like master. The build failure points out it is not able to find sonic_sku_create.py, I am not sure why?

18:52:13 _________________________ TestSkuCreate.test_no_param __________________________
18:52:13
18:52:13 self = <sku_create_test.TestSkuCreate object at 0x7f92ace8f510>
18:52:13
18:52:13 def test_no_param(self):
18:52:13 if (os.path.exists(output_dir_path)):
18:52:13 shutil.rmtree(output_dir_path)
18:52:13
18:52:13 my_command = sku_create_script + " -f " + sku_def_file + " -d " + input_path
18:52:13
18:52:13 #Test case execution without stdout
18:52:13 > result = subprocess.check_output(my_command,stderr=subprocess.STDOUT,shell=True)
18:52:13
18:52:13 sonic-utilities-tests/sku_create_test.py:61:
18:52:13 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
18:52:13
18:52:13 popenargs = ('/sonic/sonic-utilities/deb_dist/sonic-utilities-1.2/scripts/sonic_sku_create.py -f /sonic/sonic-utilities/deb_dist/s...lanox-SN2700-D48C8.xml -d /sonic/sonic-utilities/deb_dist/sonic-utilities-1.2/sonic-utilities-tests/sku_create_input',)
18:52:13 kwargs = {'shell': True, 'stderr': -2}
18:52:13 process = <subprocess.Popen object at 0x7f92ace8fd50>
18:52:13 output = '/bin/sh: 1: /sonic/sonic-utilities/deb_dist/sonic-utilities-1.2/scripts/sonic_sku_create.py: not found\n'
18:52:13 unused_err = None, retcode = 127
18:52:13 cmd = '/sonic/sonic-utilities/deb_dist/sonic-utilities-1.2/scripts/sonic_sku_create.py -f /sonic/sonic-utilities/deb_dist/so...ellanox-SN2700-D48C8.xml -d /sonic/sonic-utilities/deb_dist/sonic-utilities-1.2/sonic-utilities-tests/sku_create_input'

@jleveque
Copy link
Contributor

I believe you need to change the line

sku_create_script = os.path.join(scripts_path, "sonic_sku_create.py")

to

sku_create_script = "sonic_sku_create.py"

Because scripts_path is added to PATH in the setup_class method.

@madhanmellanox
Copy link
Contributor Author

@jleveque even after mentioning just the script name, I am still getting file not found error.

14:54:17 _________________________ TestSkuCreate.test_no_param __________________________
14:54:17
14:54:17 self = <sku_create_test.TestSkuCreate object at 0x7f7094b9d510>
14:54:17
14:54:17 def test_no_param(self):
14:54:17 if (os.path.exists(output_dir_path)):
14:54:17 shutil.rmtree(output_dir_path)
14:54:17
14:54:17 my_command = sku_create_script + " -f " + sku_def_file + " -d " + input_path
14:54:17
14:54:17 #Test case execution without stdout
14:54:17 > result = subprocess.check_output(my_command,stderr=subprocess.STDOUT,shell=True)
14:54:17
14:54:17 sonic-utilities-tests/sku_create_test.py:61:
14:54:17 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
14:54:17
14:54:17 popenargs = ('sonic_sku_create.py -f /sonic/sonic-utilities/deb_dist/sonic-utilities-1.2/sonic-utilities-tests/sku_create_input/Mellanox-SN2700-D48C8.xml -d /sonic/sonic-utilities/deb_dist/sonic-utilities-1.2/sonic-utilities-tests/sku_create_input',)
14:54:17 kwargs = {'shell': True, 'stderr': -2}
14:54:17 process = <subprocess.Popen object at 0x7f7094b9dd50>
14:54:17 output = '/bin/sh: 1: sonic_sku_create.py: not found\n', unused_err = None
14:54:17 retcode = 127
14:54:17 cmd = 'sonic_sku_create.py -f /sonic/sonic-utilities/deb_dist/sonic-utilities-1.2/sonic-utilities-tests/sku_create_input/Mellanox-SN2700-D48C8.xml -d /sonic/sonic-utilities/deb_dist/sonic-utilities-1.2/sonic-utilities-tests/sku_create_input'

@jleveque
Copy link
Contributor

@madhanmellanox: I have no other suggestions. You will need to reproduce locally.

@madhanmellanox
Copy link
Contributor Author

@jleveque if I run my test individually on my server, it runs fine. But, if I run pytest without parameters, ie, running all the test cases, other test cases like intfutil.py, portstat_test.py, sfp_test.py etc fails. Since my test case is passing in my build server when I try it manually, I am not sure why in the PR build environment it is failing with error: "sonic_sku_create.py - file not found".

@madhanmellanox
Copy link
Contributor Author

retest this please

@madhanmellanox
Copy link
Contributor Author

retest this please

2 similar comments
@madhanmellanox
Copy link
Contributor Author

retest this please

@madhanmellanox
Copy link
Contributor Author

retest this please

@jleveque
Copy link
Contributor

Retest this please

Copy link
Contributor

@jleveque jleveque left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abdosi, @rlhui: Please merge this PR when you next update the 201911 branch.

@qiluo-msft qiluo-msft merged commit 28399bf into sonic-net:201911 Dec 2, 2020
stepanblyschak pushed a commit to stepanblyschak/sonic-utilities that referenced this pull request Apr 28, 2022
ccb5245 (HEAD -> 201911, origin/201911) [fast-reboot] Fix fast-reboot when NDP entries are present (sonic-net#1295)
d09667b Multi-ASIC support for show ip(v6) route (201911 branch) (sonic-net#1283)
28399bf [201911-Mellanox] SKU creator Tool (sonic-net#1163) (sonic-net#1250)

Signed-off-by: Abhishek Dosi <[email protected]>
stepanblyschak pushed a commit to stepanblyschak/sonic-utilities that referenced this pull request Apr 28, 2022
ce8b1af9d2200c570c102a38e04cbd9659470585 (HEAD -> 201911, origin/201911) const initializer_list is not a constant expression (sonic-net#1250)
acdb03324fc268f5330d2632a02d0856f35fb880 [201911] Add lgtm.yml (sonic-net#1901)

Signed-off-by: Abhishek Dosi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants