[sonic-mgmt]: Upgrade scapy #8554

Pterosaur · 2021-08-23T05:29:46Z

Why I did it

The version of scapy in apt source is out-of-date to MACsec.

How I did it

Upgrade scapy by pip

How to verify it

Check the version of scapy that should be 2.4.5

root@5e327a6ebcb5:/# python
Python 2.7.17 (default, Feb 27 2021, 15:10:58)
[GCC 7.5.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import scapy
>>> scapy.__version__
'2.4.5'

Which release branch to backport (provide reason below if selected)

Description for the changelog

A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: Ze Gan <[email protected]>

lguohan · 2021-09-03T15:22:19Z

have you do some more ptf testing? will this cause regression for the nightly?

Pterosaur · 2021-11-11T04:18:51Z

/azp run

azure-pipelines · 2021-11-11T04:18:56Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

Pterosaur · 2021-11-11T05:24:46Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2021-11-11T05:24:57Z

Azure Pipelines successfully started running 1 pipeline(s).

wangxin · 2021-11-12T00:38:01Z

dockers/docker-sonic-mgmt/Dockerfile.j2

@@ -88,7 +88,8 @@ RUN pip install cffi==1.10.0 \
    && rm -fr nanomsg-1.0.0 \
    && rm -f 1.0.0.tar.gz  \
    && pip install nnpy    \
-    && pip install dpkt
+    && pip install dpkt    \
+    && pip install scapy --upgrade


What is the relationship between this pip installed scapy and the apt installed python-scapy above?

python-ptf will check whether python-scapy was installed, but the python-scapy is the previous version that cannot meet our requirement. So I install python-ptf firstly and then upgrade scapy by pip.

Pterosaur · 2021-11-17T14:29:01Z

/azp run

azure-pipelines · 2021-11-17T14:29:06Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

Pterosaur · 2021-11-19T04:40:11Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2021-11-19T04:40:23Z

Azure Pipelines successfully started running 1 pipeline(s).

Signed-off-by: Ze Gan <[email protected]>

…mt docker image The scapy package in sonic-mgmt docker image was upgraded in PR sonic-net/sonic-buildimage#8554. The new version of scapy uses a different way to represent and dissect BGP messages. This caused the failure of test_bgp_update_time.py. This PR is to address the compatibility issue of scapy. The test_bgp_update_timer.py script was updated to be be compatible with both old and new scapy versions. Signed-off-by: Xin Wang <[email protected]>

… docker image (#4774) What is the motivation for this PR? The scapy package in sonic-mgmt docker image was upgraded in PR sonic-net/sonic-buildimage#8554. The new version of scapy uses a different way to represent and dissect BGP messages. This caused the failure of test_bgp_update_time.py. How did you do it? This PR is to address the compatibility issue of scapy. The test_bgp_update_timer.py script was updated to be be compatible with both old and new scapy versions. How did you verify/test it? Run bgp/test_bgp_update_timer.py using both old and new sonic-mgmt docker image. Signed-off-by: Xin Wang <[email protected]>

… docker image (sonic-net#4774) What is the motivation for this PR? The scapy package in sonic-mgmt docker image was upgraded in PR sonic-net/sonic-buildimage#8554. The new version of scapy uses a different way to represent and dissect BGP messages. This caused the failure of test_bgp_update_time.py. How did you do it? This PR is to address the compatibility issue of scapy. The test_bgp_update_timer.py script was updated to be be compatible with both old and new scapy versions. How did you verify/test it? Run bgp/test_bgp_update_timer.py using both old and new sonic-mgmt docker image. Signed-off-by: Xin Wang <[email protected]>

Summary: Fixes # (issue) Need the PR: sonic-net/sonic-buildimage#8554  - [ ] Bug fix - [ ] Testbed and Framework(new/improvement) - [x] Test case(new/improvement) This PR includes the basic test, control plane and data plane, for MACsec. - Control plane 1. Check the control plane processes, wpa_supplicant, can running 2. Check the related entries in APP_DB 3. To virtual switch, Check the mka session by iproute2 - Data Plane 1. Check the traffic from down link to up link 2. Check the traffic from a neighbor device to others Build testbed of SONiC neighbor devices Please refer this document https://github.com/Azure/sonic-mgmt/blob/master/docs/testbed/README.testbed.VsSetup.md to 1setup your environment. the neighbor devices should be SONiC with the **latest image** and the vm_type should choose `vsonic` ``` ./testbed-cli.sh -m veos_vtb -n 4 -k vsonic start-vms server_1 password.txt ./testbed-cli.sh -t vtestbed.csv -m veos_vtb -k vsonic add-topo vms-kvm-t0 password.txt ./testbed-cli.sh -t vtestbed.csv -m veos_vtb deploy-mg vms-kvm-t0 veos_vtb password.txt ``` Verify health ``` ./run_tests.sh -u -n vms-kvm-t0 -d vlab-01 -c test_nbr_health.py -f vtestbed.csv -i veos_vtb -e "--neighbor_type=sonic --skip_sanity --disable_loganalyzer" ``` Run MACsec Test ``` ./run_tests.sh -u -n vms-kvm-t0 -d vlab-01 -c macsec/test_macsec.py -f vtestbed.csv -i veos_vtb -e "--neighbor_type=sonic --skip_sanity --disable_loganalyzer" ``` You should get ``` === Running tests in groups === /usr/local/lib/python2.7/dist-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. from cryptography.exceptions import InvalidSignature ============================================================================================= test session starts ============================================================================================== platform linux2 -- Python 2.7.17, pytest-4.6.5, py-1.11.0, pluggy-0.13.1 -- /usr/bin/python cachedir: .pytest_cache metadata: {'Python': '2.7.17', 'Platform': 'Linux-5.4.0-37-generic-x86_64-with-Ubuntu-18.04-bionic', 'Packages': {'py': '1.11.0', 'pytest': '4.6.5', 'pluggy': '0.13.1'}, 'Plugins': {u'repeat': u'0.9.1', u'ordering': u'0.6', u'ansible': u'2.2.2', u'xdist': u'1.28.0', u'al lure-pytest': u'2.8.22', u'html': u'1.22.1', u'forked': u'1.3.0', u'metadata': u'1.11.0'}} ansible: 2.8.12 rootdir: /data/sonic-mgmt_sonic_vm_topology/tests, inifile: pytest.ini plugins: forked-1.3.0, xdist-1.28.0, repeat-0.9.1, metadata-1.11.0, html-1.22.1, allure-pytest-2.8.22, ordering-0.6, ansible-2.2.2 collecting ... ['conf-name', 'group-name', 'topo', 'ptf_image_name', 'ptf', 'ptf_ip', 'ptf_ipv6', 'server', 'vm_base', 'dut', 'inv_name', 'auto_recover', 'comment'] Finished testbed info generating. /usr/local/lib/python2.7/dist-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. from cryptography.exceptions import InvalidSignature collecting 0 items ['conf-name', 'group-name', 'topo', 'ptf_image_name', 'ptf', 'pt f_ip', 'ptf_ipv6', 'server', 'vm_base', 'dut', 'inv_name', 'auto_recover', 'comment'] Finished testbed info generating. collected 40 items macsec/test_macsec.py::TestControlPlane::test_wpa_supplicant_processes[GCM-AES-128-security-true] PASSED [ 2%] macsec/test_macsec.py::TestControlPlane::test_appl_db[security-GCM-AES-128-true] PASSED [ 5%] macsec/test_macsec.py::TestControlPlane::test_mka_session[security-GCM-AES-128-true] PASSED [ 7%] macsec/test_macsec.py::TestDataPlane::test_server_to_neighbor[GCM-AES-128-security-true] PASSED [ 10%] macsec/test_macsec.py::TestDataPlane::test_neighbor_to_neighbor[GCM-AES-128-security-true] PASSED [ 12%] macsec/test_macsec.py::TestControlPlane::test_wpa_supplicant_processes[GCM-AES-128-security-false] PASSED [ 15%] macsec/test_macsec.py::TestControlPlane::test_appl_db[security-GCM-AES-128-false] PASSED [ 17%] macsec/test_macsec.py::TestControlPlane::test_mka_session[security-GCM-AES-128-false] PASSED [ 20%] macsec/test_macsec.py::TestDataPlane::test_server_to_neighbor[GCM-AES-128-security-false] PASSED [ 22%] macsec/test_macsec.py::TestDataPlane::test_neighbor_to_neighbor[GCM-AES-128-security-false] PASSED [ 25%] macsec/test_macsec.py::TestControlPlane::test_wpa_supplicant_processes[GCM-AES-256-security-false] PASSED [ 27%] macsec/test_macsec.py::TestControlPlane::test_appl_db[security-GCM-AES-256-false] PASSED [ 30%] macsec/test_macsec.py::TestControlPlane::test_mka_session[security-GCM-AES-256-false] PASSED [ 32%] macsec/test_macsec.py::TestDataPlane::test_server_to_neighbor[GCM-AES-256-security-false] PASSED [ 35%] macsec/test_macsec.py::TestDataPlane::test_neighbor_to_neighbor[GCM-AES-256-security-false] PASSED [ 37%] macsec/test_macsec.py::TestControlPlane::test_wpa_supplicant_processes[GCM-AES-256-security-true] PASSED [ 40%] macsec/test_macsec.py::TestControlPlane::test_appl_db[security-GCM-AES-256-true] PASSED [ 42%] macsec/test_macsec.py::TestControlPlane::test_mka_session[security-GCM-AES-256-true] PASSED [ 45%] macsec/test_macsec.py::TestDataPlane::test_server_to_neighbor[GCM-AES-256-security-true] PASSED [ 47%] macsec/test_macsec.py::TestDataPlane::test_neighbor_to_neighbor[GCM-AES-256-security-true] PASSED [ 50%] macsec/test_macsec.py::TestControlPlane::test_wpa_supplicant_processes[GCM-AES-XPN-128-security-false] PASSED [ 52%] macsec/test_macsec.py::TestControlPlane::test_appl_db[security-GCM-AES-XPN-128-false] PASSED [ 55%] macsec/test_macsec.py::TestControlPlane::test_mka_session[security-GCM-AES-XPN-128-false] PASSED [ 57%] macsec/test_macsec.py::TestDataPlane::test_server_to_neighbor[GCM-AES-XPN-128-security-false] PASSED [ 60%] macsec/test_macsec.py::TestDataPlane::test_neighbor_to_neighbor[GCM-AES-XPN-128-security-false] PASSED [ 62%] macsec/test_macsec.py::TestControlPlane::test_wpa_supplicant_processes[GCM-AES-XPN-128-security-true] PASSED [ 65%] macsec/test_macsec.py::TestControlPlane::test_appl_db[security-GCM-AES-XPN-128-true] PASSED [ 67%] macsec/test_macsec.py::TestControlPlane::test_mka_session[security-GCM-AES-XPN-128-true] PASSED [ 70%] macsec/test_macsec.py::TestDataPlane::test_server_to_neighbor[GCM-AES-XPN-128-security-true] PASSED [ 72%] macsec/test_macsec.py::TestDataPlane::test_neighbor_to_neighbor[GCM-AES-XPN-128-security-true] PASSED [ 75%] macsec/test_macsec.py::TestControlPlane::test_wpa_supplicant_processes[GCM-AES-XPN-256-security-false] PASSED [ 77%] macsec/test_macsec.py::TestControlPlane::test_appl_db[security-GCM-AES-XPN-256-false] PASSED [ 80%] macsec/test_macsec.py::TestControlPlane::test_mka_session[security-GCM-AES-XPN-256-false] PASSED [ 82%] macsec/test_macsec.py::TestDataPlane::test_server_to_neighbor[GCM-AES-XPN-256-security-false] PASSED [ 85%] macsec/test_macsec.py::TestDataPlane::test_neighbor_to_neighbor[GCM-AES-XPN-256-security-false] PASSED [ 87%] macsec/test_macsec.py::TestControlPlane::test_wpa_supplicant_processes[GCM-AES-XPN-256-security-true] PASSED [ 90%] macsec/test_macsec.py::TestControlPlane::test_appl_db[security-GCM-AES-XPN-256-true] PASSED [ 92%] macsec/test_macsec.py::TestControlPlane::test_mka_session[security-GCM-AES-XPN-256-true] PASSED [ 95%] macsec/test_macsec.py::TestDataPlane::test_server_to_neighbor[GCM-AES-XPN-256-security-true] PASSED [ 97%] macsec/test_macsec.py::TestDataPlane::test_neighbor_to_neighbor[GCM-AES-XPN-256-security-true] PASSED [100%] ``` Tested in Virtual Switch and Arista 7280 T0 #4885

Upgrade scapy

23914a2

Signed-off-by: Ze Gan <[email protected]>

Pterosaur marked this pull request as ready for review August 23, 2021 06:21

Pterosaur requested a review from lguohan as a code owner August 23, 2021 06:21

Pterosaur mentioned this pull request Aug 23, 2021

[testcase]MACsec basic test sonic-net/sonic-mgmt#3571

Merged

3 tasks

Merge branch 'master' into upgrade_scapy

3323227

Pterosaur requested a review from wangxin November 11, 2021 15:55

wangxin reviewed Nov 12, 2021

View reviewed changes

wangxin previously approved these changes Nov 14, 2021

View reviewed changes

Add scapy version

44b7eb4

Signed-off-by: Ze Gan <[email protected]>

Pterosaur dismissed wangxin’s stale review via 44b7eb4 November 23, 2021 04:36

Pterosaur force-pushed the upgrade_scapy branch from cd74dc4 to 44b7eb4 Compare November 23, 2021 04:36

Merge branch 'master' into upgrade_scapy

c37c0fc

wangxin approved these changes Nov 24, 2021

View reviewed changes

Pterosaur merged commit 79b8ff5 into sonic-net:master Nov 24, 2021

wangxin mentioned this pull request Nov 29, 2021

Fix test_bgp_update_timer failure caused by scapy upgrade in sonic-mg docker image sonic-net/sonic-mgmt#4774

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[sonic-mgmt]: Upgrade scapy #8554

[sonic-mgmt]: Upgrade scapy #8554

Pterosaur commented Aug 23, 2021 •

edited

Loading

lguohan commented Sep 3, 2021

Pterosaur commented Nov 11, 2021

azure-pipelines bot commented Nov 11, 2021

Pterosaur commented Nov 11, 2021

azure-pipelines bot commented Nov 11, 2021

wangxin Nov 12, 2021

Pterosaur Nov 14, 2021

Pterosaur commented Nov 17, 2021

azure-pipelines bot commented Nov 17, 2021

Pterosaur commented Nov 19, 2021

azure-pipelines bot commented Nov 19, 2021

[sonic-mgmt]: Upgrade scapy #8554

[sonic-mgmt]: Upgrade scapy #8554

Conversation

Pterosaur commented Aug 23, 2021 • edited Loading

Why I did it

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Description for the changelog

A picture of a cute animal (not mandatory but encouraged)

lguohan commented Sep 3, 2021

Pterosaur commented Nov 11, 2021

azure-pipelines bot commented Nov 11, 2021

Pterosaur commented Nov 11, 2021

azure-pipelines bot commented Nov 11, 2021

wangxin Nov 12, 2021

Choose a reason for hiding this comment

Pterosaur Nov 14, 2021

Choose a reason for hiding this comment

Pterosaur commented Nov 17, 2021

azure-pipelines bot commented Nov 17, 2021

Pterosaur commented Nov 19, 2021

azure-pipelines bot commented Nov 19, 2021

Pterosaur commented Aug 23, 2021 •

edited

Loading