Add support for fast and cold upgrade in test_upgrade_path

[vaibhavhd]

Description of PR

Summary: Support fast and cold upgrades in test_upgrade_path. The default upgrade type remains as warm.
Fixes # (issue)

Type of change

• Bug fix
• Testbed and Framework(new/improvement)
• Test case(new/improvement)

Approach

Allow upgrade_type (fast/warm/cold) to be passed as an argument to test_upgrade_path testcase.

What is the motivation for this PR?

How did you do it?

Upgrade using the upgrade_type argument. This will support fast and cold upgrades too.

How did you verify/test it?

To be tested in a physical testbed.
Tested fastboot in a physical testbed:

./run_tests.sh -n vms7-t0-s6100 -d str-s6100-acs-2 -f ../ansible/testbed.csv -i ../ansible/str -u -c upgrade_path/test_upgrade_path.py -e "--base_image_list=http://1.1.1.1/installer/sonic-broadcom.bin --target_image_list=http://1.1.1.1/installer/public/sonic-broadcom.bin --upgrade_type=fast" -l info    

upgrade_path/test_upgrade_path.py::test_upgrade_path
--------------------------------------------------------------------------------------- live log call ---------------------------------------------------------------------------------------
18:16:14 INFO test_upgrade_path.py:test_upgrade_path:226: Test upgrade path from http://100.127.20.23/installer/sonic/broadcom/internal-201811/sonic-broadcom.bin to http://100.127.20.23/installer/sonic/broadcom/public/sonic-broadcom.bin
18:16:14 INFO test_upgrade_path.py:test_upgrade_path:228: Installing http://100.127.20.23/installer/sonic/broadcom/internal-201811/sonic-broadcom.bin
18:16:14 INFO devices.py:get_ip_route_info:799: route raw info for 0.0.0.0/0: [u'default proto bgp src 10.1.0.32 metric 20 ', u'\tnexthop via 10.0.0.1 dev PortChannel0001 weight 1 ', u'\tnexthop via 10.0.0.5 dev PortChannel0002 weight 1 ', u'\tnexthop via 10.0.0.9 dev PortChannel0003 weight 1 ', u'\tnexthop via 10.0.0.13 dev PortChannel0004 weight 1 ']
18:16:14 INFO devices.py:get_ip_route_info:834: route parsed info for 0.0.0.0/0: {'set_src': IPv4Address(u'10.1.0.32'), 'nexthops': [(IPv4Address(u'10.0.0.1'), u'PortChannel0001'), (IPv4Address(u'10.0.0.5'), u'PortChannel0002'), (IPv4Address(u'10.0.0.9'), u'PortChannel0003'), (IPv4Address(u'10.0.0.13'), u'PortChannel0004')]}
18:16:14 INFO test_upgrade_path.py:install_sonic:171: Add default mgmt-gateway-route to the device via 10.64.246.1
18:17:05 INFO test_upgrade_path.py:install_sonic:195: Remove default mgmt-gateway-route earlier added
18:17:06 INFO test_upgrade_path.py:test_upgrade_path:231: Cold reboot the DUT to make the base image as current
18:17:10 INFO reboot.py:reboot:124: waiting for ssh to drop
18:17:10 INFO reboot.py:execute_reboot_command:108: rebooting with command "reboot"
18:17:34 INFO reboot.py:reboot:145: waiting for ssh to startup
18:19:35 INFO reboot.py:reboot:156: ssh has started up
18:19:35 INFO reboot.py:reboot:158: waiting for switch to initialize
18:21:35 INFO reboot.py:reboot:186: cold reboot finished
18:21:36 INFO reboot.py:reboot:189: DUT up since 2021-02-26 18:18:56
18:21:37 INFO test_upgrade_path.py:test_upgrade_path:236: Upgrading to http://100.127.20.23/installer/sonic/broadcom/public/sonic-broadcom.bin
18:21:38 INFO devices.py:get_ip_route_info:799: route raw info for 0.0.0.0/0: [u'default proto zebra src 10.1.0.32 ', u'\tnexthop via 10.0.0.5  dev PortChannel0002 weight 1', u'\tnexthop via 10.0.0.1  dev PortChannel0001 weight 1', u'\tnexthop via 10.0.0.9  dev PortChannel0003 weight 1', u'\tnexthop via 10.0.0.13  dev PortChannel0004 weight 1']
18:21:38 INFO devices.py:get_ip_route_info:834: route parsed info for 0.0.0.0/0: {'set_src': IPv4Address(u'10.1.0.32'), 'nexthops': [(IPv4Address(u'10.0.0.5'), u'PortChannel0002'), (IPv4Address(u'10.0.0.1'), u'PortChannel0001'), (IPv4Address(u'10.0.0.9'), u'PortChannel0003'), (IPv4Address(u'10.0.0.13'), u'PortChannel0004')]}
18:21:38 INFO test_upgrade_path.py:install_sonic:171: Add default mgmt-gateway-route to the device via 10.64.246.1
18:22:38 INFO test_upgrade_path.py:install_sonic:195: Remove default mgmt-gateway-route earlier added
18:22:39 INFO ssh_utils.py:prepare_testbed_ssh_keys:17: Remove old keys from ptfhost
18:22:42 INFO ssh_utils.py:prepare_testbed_ssh_keys:26: Generate public key for ptf host
18:28:25 INFO reboot.py:get_reboot_cause:198: Getting reboot cause from dut str-s6100-acs-2
18:28:31 INFO test_upgrade_path.py:test_upgrade_path:256: Check reboot cause. Expected cause fast
18:28:31 INFO test_upgrade_path.py:check_services:204: Wait until DUT uptime reaches 300s
18:31:43 INFO test_upgrade_path.py:check_services:207: Wait until all critical services are fully started
18:31:43 INFO test_upgrade_path.py:check_services:208: Check critical service status
PASSED 

Cold upgrade:

18:02:03 INFO test_upgrade_path.py:test_upgrade_path:226: Test upgrade path from http://100.127.20.23/installer/sonic/broadcom/internal-201811/sonic-broadcom.bin to http://100.127.20.23/installer/sonic/broadcom/public/sonic-broadcom.bin
18:02:03 INFO test_upgrade_path.py:test_upgrade_path:228: Installing http://1.1.1.1/internal-201811/sonic-broadcom.bin
18:02:03 INFO devices.py:get_ip_route_info:799: route raw info for 0.0.0.0/0: [u'default proto bgp src 10.1.0.32 metric 20 ', u'\tnexthop via 10.0.0.1 dev PortChannel0001 weight 1 ', u'\tnexthop via 10.0.0.5 dev PortChannel0002 weight 1 ', u'\tnexthop via 10.0.0.9 dev PortChannel0003 weight 1 ', u'\tnexthop via 10.0.0.13 dev PortChannel0004 weight 1 ']
18:02:03 INFO devices.py:get_ip_route_info:834: route parsed info for 0.0.0.0/0: {'set_src': IPv4Address(u'10.1.0.32'), 'nexthops': [(IPv4Address(u'10.0.0.1'), u'PortChannel0001'), (IPv4Address(u'10.0.0.5'), u'PortChannel0002'), (IPv4Address(u'10.0.0.9'), u'PortChannel0003'), (IPv4Address(u'10.0.0.13'), u'PortChannel0004')]}
18:02:03 INFO test_upgrade_path.py:install_sonic:171: Add default mgmt-gateway-route to the device via 10.64.246.1
18:02:52 INFO test_upgrade_path.py:install_sonic:195: Remove default mgmt-gateway-route earlier added
18:02:53 INFO test_upgrade_path.py:test_upgrade_path:231: Cold reboot the DUT to make the base image as current
18:02:57 INFO reboot.py:reboot:124: waiting for ssh to drop
18:02:57 INFO reboot.py:execute_reboot_command:108: rebooting with command "reboot"
18:03:20 INFO reboot.py:reboot:145: waiting for ssh to startup
18:05:19 INFO reboot.py:reboot:156: ssh has started up
18:05:19 INFO reboot.py:reboot:158: waiting for switch to initialize
18:07:20 INFO reboot.py:reboot:186: cold reboot finished
18:07:21 INFO reboot.py:reboot:189: DUT up since 2021-02-26 18:04:40
18:07:22 INFO test_upgrade_path.py:test_upgrade_path:236: Upgrading to http://1.1.1.1/public/sonic-broadcom.bin
18:07:22 INFO devices.py:get_ip_route_info:799: route raw info for 0.0.0.0/0: [u'default proto zebra src 10.1.0.32 ', u'\tnexthop via 10.0.0.13  dev PortChannel0004 weight 1', u'\tnexthop via 10.0.0.1  dev PortChannel0001 weight 1', u'\tnexthop via 10.0.0.5  dev PortChannel0002 weight 1', u'\tnexthop via 10.0.0.9  dev PortChannel0003 weight 1']
18:07:22 INFO devices.py:get_ip_route_info:834: route parsed info for 0.0.0.0/0: {'set_src': IPv4Address(u'10.1.0.32'), 'nexthops': [(IPv4Address(u'10.0.0.13'), u'PortChannel0004'), (IPv4Address(u'10.0.0.1'), u'PortChannel0001'), (IPv4Address(u'10.0.0.5'), u'PortChannel0002'), (IPv4Address(u'10.0.0.9'), u'PortChannel0003')]}
18:07:22 INFO test_upgrade_path.py:install_sonic:171: Add default mgmt-gateway-route to the device via 1.1.1.1
18:08:22 INFO test_upgrade_path.py:install_sonic:195: Remove default mgmt-gateway-route earlier added
18:08:22 INFO ssh_utils.py:prepare_testbed_ssh_keys:17: Remove old keys from ptfhost
18:08:25 INFO ssh_utils.py:prepare_testbed_ssh_keys:26: Generate public key for ptf host
18:08:32 INFO reboot.py:reboot:124: waiting for ssh to drop
18:08:32 INFO reboot.py:execute_reboot_command:108: rebooting with command "reboot"
18:08:47 INFO reboot.py:reboot:145: waiting for ssh to startup
18:10:18 INFO reboot.py:reboot:156: ssh has started up
18:10:18 INFO reboot.py:reboot:158: waiting for switch to initialize
18:12:19 INFO reboot.py:reboot:186: cold reboot finished
18:12:23 INFO reboot.py:reboot:189: DUT up since 2021-02-26 18:09:30
18:12:23 INFO reboot.py:get_reboot_cause:198: Getting reboot cause from dut str-s6100-acs-2
18:12:24 INFO test_upgrade_path.py:test_upgrade_path:256: Check reboot cause. Expected cause cold
18:12:24 INFO test_upgrade_path.py:check_services:204: Wait until DUT uptime reaches 300s
18:14:30 INFO test_upgrade_path.py:check_services:207: Wait until all critical services are fully started
18:14:30 INFO test_upgrade_path.py:check_services:208: Check critical service status
PASSED

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

[sumukhatv]

@prsunny for awareness