Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[QoS] Update QoS pytests to use the multi-ASIC framework. #2993

Merged
merged 9 commits into from
Feb 24, 2021

Conversation

smaheshm
Copy link
Contributor

@smaheshm smaheshm commented Feb 17, 2021

Summary:
Update QoS pytests to use the multi ASIC framework.

Major changes:

  • Update QoS test script to use the multi ASIC framework
  • Update QoS tests so that the tests can be run independently on each ASIC. All tests may not fall into this category, and will be decided on a case by case basis.

NOTE: The changes here do not add new functionality or alter current logic. The changes are in preparation to run QoS tests on each ASIC independently on multi ASIC platforms.

Minor changes:

  • code refactoring to include "asic index" or "dut_asic" as a parameter.
  • API to get active IP interfaces on T1 topology.

SSH tunnel:

This is a new API required to establish RPC tunnel between host namespace and the ASIC namespace on multi-ASIC platforms. QoS SAI tests use syncd RPC binary and use thrift APIs to to set/get port parameters. Thrift server port is constant across all namespaces. SAI tests use management IP and thrift port to communicate with thrift server. On a multi ASIC platform thrift server ports are opened in their respective namespaces. To facilitate communication with pytests, ssh tunnel is created between host namespace and the asic namespace on which test will run.

Fixes # (issue)

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Approach

What is the motivation for this PR?

Support QoS/PFC tests on multi-ASIC platforms. Ability to run QoS tests on each ASIC independently.

How did you do it?

Use 'asic index' fixture to generate tests per ASIC.

How did you verify/test it?

  • Verified on T0 for regression.
samaddik@str-serv-acs-14:/var/sonic-mgmt/tests$ pytest qos/test_qos_sai.py --testbed=vms3-t0-s6100 --inventory=../ansible/str --testbed_file=../ansible/testbed.csv --host-pattern=str-s6100-acs-1 --module-path=../ansible/library --disable_loganalyzer --skip_sanity
/usr/local/lib/python2.7/dist-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
  from cryptography.exceptions import InvalidSignature                         
=========================================================================================================================================================================== test session starts ============================================================================================================================================================================
platform linux2 -- Python 2.7.17, pytest-4.6.5, py-1.10.0, pluggy-0.13.1      
ansible: 2.8.12                
rootdir: /var/sonic-mgmt/tests, inifile: pytest.ini
plugins: forked-1.3.0, metadata-1.11.0, xdist-1.28.0, html-1.22.1, repeat-0.9.1, ansible-2.2.2
collected 19 items                                                                                                                                                                                                                                                                                                                                                         
                                                                                                            
qos/test_qos_sai.py ..Loading callback plugin json of type stdout, v2.0 from /usr/local/lib/python2.7/dist-packages/ansible/plugins/callback/json.pyc         
Loading callback plugin json of type stdout, v2.0 from /usr/local/lib/python2.7/dist-packages/ansible/plugins/callback/json.pyc                                                             
F.ss......s..Loading callback plugin json of type stdout, v2.0 from /usr/local/lib/python2.7/dist-packages/ansible/plugins/callback/json.pyc                                                                                                                
Loading callback plugin json of type stdout, v2.0 from /usr/local/lib/python2.7/dist-packages/ansible/plugins/callback/json.pyc                                                                                                                                
Fs..                                                                                                                                                                                                                                  [100%]
============================================================================================================== 2 failed, 13 passed, 4 skipped in 1653.78 seconds ===============================================================================================================

NOTE: The failed tests are not due to this change. Verified the dutConfig and dutQosConfig have valid values. There is no change in test case logic.

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

@smaheshm smaheshm requested a review from a team as a code owner February 17, 2021 05:13
@lgtm-com
Copy link

lgtm-com bot commented Feb 17, 2021

This pull request introduces 3 alerts when merging 9da5f83 into 5ed5cab - view on LGTM.com

new alerts:

  • 2 for Unused local variable
  • 1 for Comparison using is when operands support `__eq__`

Copy link
Contributor

@neethajohn neethajohn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once you have the PR for the portchannel support in sai_qos, please update the description of this PR to include that info

tests/common/devices.py Outdated Show resolved Hide resolved
tests/common/devices.py Show resolved Hide resolved
tests/common/devices.py Show resolved Hide resolved
tests/qos/qos_sai_base.py Outdated Show resolved Hide resolved
tests/qos/qos_sai_base.py Outdated Show resolved Hide resolved
tests/qos/qos_sai_base.py Show resolved Hide resolved
tests/qos/qos_sai_base.py Show resolved Hide resolved
tests/qos/qos_sai_base.py Outdated Show resolved Hide resolved
@neethajohn neethajohn changed the title [QoS] Update QoS pytests to use the mulit-ASIC framework. [QoS] Update QoS pytests to use the multi-ASIC framework. Feb 20, 2021
@lgtm-com
Copy link

lgtm-com bot commented Feb 23, 2021

This pull request introduces 1 alert when merging 8df4655 into b0890db - view on LGTM.com

new alerts:

  • 1 for Unused local variable

@lgtm-com
Copy link

lgtm-com bot commented Feb 23, 2021

This pull request introduces 1 alert when merging 8f92224 into 63c3781 - view on LGTM.com

new alerts:

  • 1 for Unused local variable

@smaheshm smaheshm merged commit 0e97bba into sonic-net:master Feb 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants