Merge branch 'hash_4_add_hash_keys' of https://github.com/William-zx/…

…sonic-mgmt into hash_4_add_hash_keys

ansible/config_sonic_basedon_testbed.yml

@@ -144,6 +144,49 @@
         become: true
         when: stat_result.stat.exists is defined and stat_result.stat.exists
 
+      - name: Init account key and proxy
+        set_fact:
+            core_key: ""
+            core_proxy: ""
+
+      - name: read account key
+        set_fact:
+            core_key: "{{ corefile_uploader['azure_sonic_core_storage']['account_key'] }}"
+        when: corefile_uploader['azure_sonic_core_storage']['account_key'] is defined
+
+      - name: read https proxy
+        set_fact:
+            core_proxy: "{{ corefile_uploader['env']['https_proxy'] }}"
+        when: corefile_uploader['env']['https_proxy'] is defined
+
+      - name: Put secret in core_analyzer.rc.json
+        lineinfile:
+            name: /etc/sonic/core_analyzer.rc.json
+            regexp: '(^.*)account_key'
+            line: '\1account_key": "{{ core_key }}",'
+            backrefs: yes
+        become: true
+        when: core_key != ""
+
+      - name: Put https-proxy in core_analyzer.rc.json
+        lineinfile:
+            name: /etc/sonic/core_analyzer.rc.json
+            regexp: '(^.*)https_proxy'
+            line: '\1https_proxy": "{{ core_proxy }}"'
+            backrefs: yes
+        become: true
+        when: core_proxy != ""
+
+      - name: enable core uploader service
+        become: true
+        command: systemctl enable core_uploader.service
+        when: core_key != ""
+
+      - name: start core uploader service
+        become: true
+        command: systemctl start core_uploader.service
+        when: core_key != ""
+
       - name: Replace snmp community string
         lineinfile:
             name: /etc/sonic/snmp.yml

ansible/doc/README.testbed.cEOS.md

@@ -0,0 +1,131 @@
+# cEOS
+
+This document discusses how to use cEOS as DUT neighbor device.
+
+cEOS is the container-based EOS. All the software running inside
+the container. Compared with vEOS, cEOS has much smaller memory
+footprint.
+
+Follow [instruction](README.testbed.VsSetup.md) to setup cEOS testbed.
+
+In below example, there are four cEOS containers.
+
+```
+lgh@jenkins-worker-15:~$ docker ps
+CONTAINER ID        IMAGE                                                        COMMAND                  CREATED             STATUS              PORTS               NAMES
+fe48c207a51c        ceosimage:4.23.2F-1                                          "/sbin/init systemd.…"   8 days ago          Up 8 days                               ceos_vms6-1_VM0103
+52297010e66a        ceosimage:4.23.2F-1                                          "/sbin/init systemd.…"   8 days ago          Up 8 days                               ceos_vms6-1_VM0102
+8dd95269b312        ceosimage:4.23.2F-1                                          "/sbin/init systemd.…"   8 days ago          Up 8 days                               ceos_vms6-1_VM0101
+3a50dd481bfb        ceosimage:4.23.2F-1                                          "/sbin/init systemd.…"   8 days ago          Up 8 days                               ceos_vms6-1_VM0100
+b91b48145def        debian:jessie                                                "bash"                   8 days ago          Up 8 days                               net_vms6-1_VM0103
+d1ff26d84249        debian:jessie                                                "bash"                   8 days ago          Up 8 days                               net_vms6-1_VM0102
+1489f52b9617        debian:jessie                                                "bash"                   8 days ago          Up 8 days                               net_vms6-1_VM0101
+ce1214a008ed        debian:jessie                                                "bash"                   8 days ago          Up 8 days                               net_vms6-1_VM0100
+```
+
+## Resource consumption
+
+A cEOS containers consumes around 1G memory.
+
+```
+lgh@jenkins-worker-15:~$ docker stats --no-stream
+CONTAINER ID        NAME                 CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS          6
+fe48c207a51c        ceos_vms6-1_VM0103   2.04%               970.9MiB / 125.9GiB   0.75%               0B / 0B             365MB / 55.8GB      138
+52297010e66a        ceos_vms6-1_VM0102   2.19%               965.4MiB / 125.9GiB   0.75%               0B / 0B             237MB / 55.6GB      139
+8dd95269b312        ceos_vms6-1_VM0101   1.93%               980.9MiB / 125.9GiB   0.76%               0B / 0B             300MB / 55.9GB      138
+3a50dd481bfb        ceos_vms6-1_VM0100   2.05%               970.2MiB / 125.9GiB   0.75%               0B / 0B             365MB / 56.1GB      138
+```
+
+## Network Setup
+
+We first create a base container `net_${testbed_name}_${vm_name}`, inject six ethernet ports into the base container, 
+and then start cEOS `ceos_${testbed_name}_${vm_name}` container on top of the base container. The six ethernet ports
+are used for
+- 1 management port
+- 4 front panel ports to DUT
+- 1 backplane port to PTF docker
+
+```
+         +------------+                      +----+
+         |  cEOS  Ma0 +--------- VM0100-m ---+ br |
+         |            |                      +----+
+         |            |
+         |            |                      +--------------+
+         |        Et1 +----------VM0100-t0---+  br-VM0100-0 |
+         |            |                      +--------------+
+         |            |
+         |            |                      +--------------+
+         |        Et2 +----------VM0100-t1---+  br-VM0100-1 |
+         |            |                      +--------------+
+         |            |
+         |            |                      +--------------+
+         |        Et3 +----------VM0100-t2---+  br-VM0100-2 |
+         |            |                      +--------------+
+         |            |
+         |            |                      +--------------+
+         |        Et4 +----------VM0100-t3---+  br-VM0100-3 |
+         |            |                      +--------------+
+         |            |
+         |            |                       +--------------+
+         |        Et5 +----------VM0100-back--+  br-b-vms6-1 |
+         |            |                       +--------------+
+         +------------+
+```
+
+## Configuration
+
+The `/mnt/flash` in cEOS container is mount to `/data/ceos/ceos_${testbed_name}_${vm_name}` on the host. The `/mnt/flash`
+contiains the configuration file and logs.
+
+```
+lgh@jenkins-worker-15:~$ ls -l /data/ceos/ceos_vms6-1_VM0100/
+total 40
+-rw-rw-r--+ 1 root root  924 Mar 31 07:35 AsuFastPktTransmit.log
+drwxrwxr-x+ 2 root root 4096 Mar 31 03:31 Fossil
+-rw-rw-r--+ 1 root root  568 Mar 31 07:35 SsuRestore.log
+-rw-rw-r--+ 1 root root  568 Mar 31 07:35 SsuRestoreLegacy.log
+drwxr-xr-x+ 4  897   88 4096 Mar 31 07:35 archive
+drwxrwx---+ 3 root root 4096 Mar 18 06:12 debug
+drwxrwxr-x+ 2 root root 4096 Mar 18 06:12 fastpkttx.backup
+-rw-rw-r--+ 1 root root  180 Mar 31 07:35 kickstart-config
+drwxrwxr-x+ 3 root root 4096 Apr  8 09:11 persist
+-rw-rwxr--+ 1 root root 1915 Mar 18 06:12 startup-config
+```
+
+## Login
+
+There are two ways to get into cEOS container
+
+1. docker exec
+```
+lgh@jenkins-worker-15:~$ docker exec -it ceos_vms6-1_VM0100 Cli
+ARISTA01T1>show int status
+Port       Name      Status       Vlan     Duplex Speed  Type            Flags Encapsulation
+Et1                  connected    in Po1   full   unconf EbraTestPhyPort                    
+Et2                  connected    1        full   unconf EbraTestPhyPort                    
+Et3                  connected    1        full   unconf EbraTestPhyPort                    
+Et4                  connected    1        full   unconf EbraTestPhyPort                    
+Et5        backplane connected    routed   full   unconf EbraTestPhyPort                    
+Ma0                  connected    routed   full   10G    10/100/1000                        
+Po1                  connected    routed   full   unconf N/A    
+
+ARISTA01T1>
+```
+
+2. ssh
+```
+lgh@jenkins-worker-15:~$ ssh [email protected]
+Password: 
+ARISTA01T1>show int status
+Port       Name      Status       Vlan     Duplex Speed  Type            Flags Encapsulation
+Et1                  connected    in Po1   full   unconf EbraTestPhyPort                    
+Et2                  connected    1        full   unconf EbraTestPhyPort                    
+Et3                  connected    1        full   unconf EbraTestPhyPort                    
+Et4                  connected    1        full   unconf EbraTestPhyPort                    
+Et5        backplane connected    routed   full   unconf EbraTestPhyPort                    
+Ma0                  connected    routed   full   10G    10/100/1000                        
+Po1                  connected    routed   full   unconf N/A                                
+
+ARISTA01T1>
+```
+

ansible/group_vars/all/corefile_uploader.yml

@@ -0,0 +1,7 @@
+# Configure core file storage secret key and https-proxy as required
+#
+#corefile_uploader:
+#        azure_sonic_core_storage:
+#                account_key: "Your Secret"
+#        env:
+#                https_proxy: "http://10.10.10.10:8000"

ansible/roles/test/files/ptftests/advanced-reboot.py

@@ -57,12 +57,12 @@
 import re
 from collections import defaultdict
 import json
-import paramiko
 import Queue
 import pickle
 from operator import itemgetter
 import scapy.all as scapyall
 import itertools
+from device_connection import DeviceConnection
 
 from arista import Arista
 import sad_path as sp
@@ -125,6 +125,7 @@ def __init__(self):
         self.test_params = testutils.test_params_get()
         self.check_param('verbose', False, required=False)
         self.check_param('dut_username', '', required=True)
+        self.check_param('dut_password', '', required=True)
         self.check_param('dut_hostname', '', required=True)
         self.check_param('reboot_limit_in_seconds', 30, required=False)
         self.check_param('reboot_type', 'fast-reboot', required=False)
@@ -217,6 +218,12 @@ def __init__(self):
 
         self.allow_vlan_flooding = bool(self.test_params['allow_vlan_flooding'])
 
+        self.dut_connection = DeviceConnection(
+            self.test_params['dut_hostname'],
+            self.test_params['dut_username'],
+            password=self.test_params['dut_password']
+        )
+
         return
 
     def read_json(self, name):
@@ -411,7 +418,7 @@ def get_sad_info(self):
     def init_sad_oper(self):
         if self.sad_oper:
             self.log("Preboot/Inboot Operations:")
-            self.sad_handle = sp.SadTest(self.sad_oper, self.ssh_targets, self.portchannel_ports, self.vm_dut_map, self.test_params, self.dut_ssh, self.vlan_ports)
+            self.sad_handle = sp.SadTest(self.sad_oper, self.ssh_targets, self.portchannel_ports, self.vm_dut_map, self.test_params, self.vlan_ports)
             (self.ssh_targets, self.portchannel_ports, self.neigh_vm, self.vlan_ports), (log_info, fails) = self.sad_handle.setup()
             self.populate_fail_info(fails)
             for log in log_info:
@@ -480,7 +487,6 @@ def setUp(self):
         self.reboot_type = self.test_params['reboot_type']
         if self.reboot_type not in ['fast-reboot', 'warm-reboot']:
             raise ValueError('Not supported reboot_type %s' % self.reboot_type)
-        self.dut_ssh = self.test_params['dut_username'] + '@' + self.test_params['dut_hostname']
         self.dut_mac = self.test_params['dut_mac']
 
         # get VM info
@@ -509,7 +515,7 @@ def setUp(self):
         self.from_server_dst_ports = self.portchannel_ports
 
         self.log("Test params:")
-        self.log("DUT ssh: %s" % self.dut_ssh)
+        self.log("DUT ssh: %s@%s" % (self.test_params['dut_username'], self.test_params['dut_hostname']))
         self.log("DUT reboot limit in seconds: %s" % self.limit)
         self.log("DUT mac address: %s" % self.dut_mac)
 
@@ -1004,7 +1010,7 @@ def reboot_dut(self):
         time.sleep(self.reboot_delay)
 
         self.log("Rebooting remote side")
-        stdout, stderr, return_code = self.cmd(["ssh", "-oStrictHostKeyChecking=no", self.dut_ssh, "sudo " + self.reboot_type])
+        stdout, stderr, return_code = self.dut_connection.execCommand("sudo " + self.reboot_type)
         if stdout != []:
             self.log("stdout from %s: %s" % (self.reboot_type, str(stdout)))
         if stderr != []:

ansible/roles/test/files/ptftests/device_connection.py

@@ -0,0 +1,63 @@
+import paramiko
+import logging
+from paramiko.ssh_exception import BadHostKeyException, AuthenticationException, SSHException
+
+logger = logging.getLogger(__name__)
+
+DEFAULT_CMD_EXECUTION_TIMEOUT_SEC = 10
+
+class DeviceConnection:
+    '''
+    DeviceConnection uses Paramiko module to connect to devices
+
+    Paramiko module uses fallback mechanism where it would first try to use
+    ssh key and that fails, it will attempt username/password combination
+    '''
+    def __init__(self, hostname, username, password=None):
+        '''
+        Class constructor
+
+        @param hostname: hostname of device to connect to
+        @param username: username for device connection
+        @param password: password for device connection
+        '''
+        self.hostname = hostname
+        self.username = username
+        self.password = password
+
+    def execCommand(self, cmd, timeout=DEFAULT_CMD_EXECUTION_TIMEOUT_SEC):
+        '''
+        Executes command on remote device
+
+        @param cmd: command to be run on remote device
+        @param timeout: timeout for command run session
+        @return: stdout, stderr, value
+            stdout is a list of lines of the remote stdout gathered during command execution
+            stderr is a list of lines of the remote stderr gathered during command execution
+            value: 0 if command execution raised no exception
+                   nonzero if exception is raised
+        '''
+        client = paramiko.SSHClient()
+        client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+
+        if isinstance(cmd, list):
+            cmd = ' '.join(cmd)
+
+        stdOut = stdErr = []
+        retValue = 1
+        try:
+            client.connect(self.hostname, username=self.username, password=self.password, allow_agent=False)
+            si, so, se = client.exec_command(cmd, timeout=timeout)
+            stdOut = so.readlines()
+            stdErr = se.readlines()
+            retValue = 0
+        except SSHException as sshException:
+            logger.error('SSH Command failed with message: %s' % sshException)
+        except AuthenticationException as authenticationException:
+            logger.error('SSH Authentiaction failure with message: %s' % authenticationException)
+        except BadHostKeyException as badHostKeyException:
+            logger.error('SSH Authentiaction failure with message: %s' % badHostKeyException)
+        finally:
+            client.close()
+
+        return stdOut, stdErr, retValue