From 818dcbf0895d2511c1d7c055716218c35e0a548e Mon Sep 17 00:00:00 2001 From: Mohamed Ghoneim Date: Fri, 17 Dec 2021 13:08:39 -0800 Subject: [PATCH] [GCU] Implementing DryRun by printing patch-sorter steps/imitating config_db (#1973) #### What I did Implementing `dry-run` option flag. - Supports only printing the steps generated from patch-sorting - TODO in a future PR: Print the `SET` commands sent to `config_db` - TODO in a future PR: Print the service validation commands #### How I did it By implementing the DryRunConfigWrapper. - Whenever a dry-run is issued, the CLI output will start with `** DRY RUN EXECUTION **` - At each step we simulate `config_db`, we print a log msg starting with `** DryRun: Would ` #### How to verify it #### Previous command output (if the output of a command-line utility has changed) #### New command output (if the output of a command-line utility has changed) **Apply patch** ``` admin@vlab-01:~$ sudo config apply-patch remove-acl-table.json-patch -d -i /BGP_NEIGHBOR -i /FEATURE -i /QUEUE -i /VLAN/Vlan1000/members -i /DEVICE_METADATA -i /FLEX_COUNTER_TABLE -i /SCHEDULER ** DRY RUN EXECUTION ** Patch Applier: Patch application starting. Patch Applier: Patch: [{"op": "remove", "path": "/ACL_TABLE/DATAACL"}] Patch Applier: Getting current config db. Patch Applier: Simulating the target full config after applying the patch. Patch Applier: Validating target config does not have empty tables, since they do not show up in ConfigDb. Patch Applier: Sorting patch updates. Note: Below table(s) have no YANG models: BGP_PEER_RANGE, CABLE_LENGTH, CONSOLE_SWITCH, DEVICE_NEIGHBOR_METADATA, DHCP_SERVER, KDUMP, RESTAPI, SNMP, SNMP_COMMUNITY, SYSLOG_SERVER, TELEMETRY, Note: Below table(s) have no YANG models: BGP_PEER_RANGE, CABLE_LENGTH, CONSOLE_SWITCH, DEVICE_NEIGHBOR_METADATA, DHCP_SERVER, KDUMP, RESTAPI, SNMP, SNMP_COMMUNITY, SYSLOG_SERVER, TELEMETRY, Note: Below table(s) have no YANG models: BGP_PEER_RANGE, CABLE_LENGTH, CONSOLE_SWITCH, DEVICE_NEIGHBOR_METADATA, DHCP_SERVER, KDUMP, RESTAPI, SNMP, SNMP_COMMUNITY, SYSLOG_SERVER, TELEMETRY, libyang[0]: Invalid JSON data (unexpected value). (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='DATAACL']/ports) sonic_yang(3):Data Loading Failed:Invalid JSON data (unexpected value). libyang[0]: Invalid JSON data (unexpected value). (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='DATAACL']/ports) sonic_yang(3):Data Loading Failed:Invalid JSON data (unexpected value). libyang[0]: Missing required element "type" in "ACL_TABLE_LIST". (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='DATAACL']) sonic_yang(3):Data Loading Failed:Missing required element "type" in "ACL_TABLE_LIST". libyang[0]: Missing required element "type" in "ACL_TABLE_LIST". (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='DATAACL']) sonic_yang(3):Data Loading Failed:Missing required element "type" in "ACL_TABLE_LIST". Patch Applier: The patch was sorted into 7 changes: Patch Applier: * [{"op": "remove", "path": "/ACL_TABLE/DATAACL/policy_desc"}] Patch Applier: * [{"op": "remove", "path": "/ACL_TABLE/DATAACL/ports/0"}] Patch Applier: * [{"op": "remove", "path": "/ACL_TABLE/DATAACL/ports/0"}] Patch Applier: * [{"op": "remove", "path": "/ACL_TABLE/DATAACL/ports/0"}] Patch Applier: * [{"op": "remove", "path": "/ACL_TABLE/DATAACL/stage"}] Patch Applier: * [{"op": "remove", "path": "/ACL_TABLE/DATAACL/ports"}] Patch Applier: * [{"op": "remove", "path": "/ACL_TABLE/DATAACL"}] Patch Applier: Applying 7 changes in order: Patch Applier: * [{"op": "remove", "path": "/ACL_TABLE/DATAACL/policy_desc"}] ** DryRun: Would apply [{"op": "remove", "path": "/ACL_TABLE/DATAACL/policy_desc"}] Patch Applier: * [{"op": "remove", "path": "/ACL_TABLE/DATAACL/ports/0"}] ** DryRun: Would apply [{"op": "remove", "path": "/ACL_TABLE/DATAACL/ports/0"}] Patch Applier: * [{"op": "remove", "path": "/ACL_TABLE/DATAACL/ports/0"}] ** DryRun: Would apply [{"op": "remove", "path": "/ACL_TABLE/DATAACL/ports/0"}] Patch Applier: * [{"op": "remove", "path": "/ACL_TABLE/DATAACL/ports/0"}] ** DryRun: Would apply [{"op": "remove", "path": "/ACL_TABLE/DATAACL/ports/0"}] Patch Applier: * [{"op": "remove", "path": "/ACL_TABLE/DATAACL/stage"}] ** DryRun: Would apply [{"op": "remove", "path": "/ACL_TABLE/DATAACL/stage"}] Patch Applier: * [{"op": "remove", "path": "/ACL_TABLE/DATAACL/ports"}] ** DryRun: Would apply [{"op": "remove", "path": "/ACL_TABLE/DATAACL/ports"}] Patch Applier: * [{"op": "remove", "path": "/ACL_TABLE/DATAACL"}] ** DryRun: Would apply [{"op": "remove", "path": "/ACL_TABLE/DATAACL"}] Patch Applier: Verifying patch updates are reflected on ConfigDB. Patch Applier: Patch application completed. Patch applied successfully. admin@vlab-01:~$ ``` **Config rollback** ``` admin@vlab-01:~$ sudo config rollback cp1 -d -i /BGP_NEIGHBOR -i /FEATURE -i /QUEUE -i /VLAN/Vlan1000/members -i /DEVICE_METADATA -i /FLEX_COUNTER_TABLE -i /SCHEDULER ** DRY RUN EXECUTION ** Config Rollbacker: Config rollbacking starting. Config Rollbacker: Checkpoint name: cp1. Config Rollbacker: Verifying 'cp1' exists. Config Rollbacker: Loading checkpoint into memory. Config Rollbacker: Replacing config using 'Config Replacer'. Config Replacer: Config replacement starting. Config Replacer: Target config length: 49881. Config Replacer: Getting current config db. Config Replacer: Generating patch between target config and current config db. Config Replacer: Applying patch using 'Patch Applier'. Patch Applier: Patch application starting. Patch Applier: Patch: [{"op": "add", "path": "/ACL_TABLE/DATAACL", "value": {"policy_desc": "DATAACL", "ports": ["PortChannel0001", "PortChannel0002", "PortChannel0003", "PortChannel0004"], "stage": "ingress", "type": "L3"}}] Patch Applier: Getting current config db. Patch Applier: Simulating the target full config after applying the patch. Patch Applier: Validating target config does not have empty tables, since they do not show up in ConfigDb. Patch Applier: Sorting patch updates. Note: Below table(s) have no YANG models: BGP_PEER_RANGE, CABLE_LENGTH, CONSOLE_SWITCH, DEVICE_NEIGHBOR_METADATA, DHCP_SERVER, KDUMP, RESTAPI, SNMP, SNMP_COMMUNITY, SYSLOG_SERVER, TELEMETRY, Note: Below table(s) have no YANG models: BGP_PEER_RANGE, CABLE_LENGTH, CONSOLE_SWITCH, DEVICE_NEIGHBOR_METADATA, DHCP_SERVER, KDUMP, RESTAPI, SNMP, SNMP_COMMUNITY, SYSLOG_SERVER, TELEMETRY, Note: Below table(s) have no YANG models: BGP_PEER_RANGE, CABLE_LENGTH, CONSOLE_SWITCH, DEVICE_NEIGHBOR_METADATA, DHCP_SERVER, KDUMP, RESTAPI, SNMP, SNMP_COMMUNITY, SYSLOG_SERVER, TELEMETRY, libyang[0]: Missing required element "type" in "ACL_TABLE_LIST". (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='DATAACL']) sonic_yang(3):Data Loading Failed:Missing required element "type" in "ACL_TABLE_LIST". libyang[0]: Missing required element "type" in "ACL_TABLE_LIST". (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='DATAACL']) sonic_yang(3):Data Loading Failed:Missing required element "type" in "ACL_TABLE_LIST". libyang[0]: Missing required element "type" in "ACL_TABLE_LIST". (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='DATAACL']) sonic_yang(3):Data Loading Failed:Missing required element "type" in "ACL_TABLE_LIST". libyang[0]: Missing required element "type" in "ACL_TABLE_LIST". (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='DATAACL']) sonic_yang(3):Data Loading Failed:Missing required element "type" in "ACL_TABLE_LIST". libyang[0]: Missing required element "type" in "ACL_TABLE_LIST". (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='DATAACL']) sonic_yang(3):Data Loading Failed:Missing required element "type" in "ACL_TABLE_LIST". libyang[0]: Missing required element "type" in "ACL_TABLE_LIST". (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='DATAACL']) sonic_yang(3):Data Loading Failed:Missing required element "type" in "ACL_TABLE_LIST". Patch Applier: The patch was sorted into 7 changes: Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL", "value": {"type": "L3"}}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/policy_desc", "value": "DATAACL"}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports", "value": ["PortChannel0001"]}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/1", "value": "PortChannel0002"}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/2", "value": "PortChannel0003"}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/3", "value": "PortChannel0004"}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/stage", "value": "ingress"}] Patch Applier: Applying 7 changes in order: Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL", "value": {"type": "L3"}}] ** DryRun: Would apply [{"op": "add", "path": "/ACL_TABLE/DATAACL", "value": {"type": "L3"}}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/policy_desc", "value": "DATAACL"}] ** DryRun: Would apply [{"op": "add", "path": "/ACL_TABLE/DATAACL/policy_desc", "value": "DATAACL"}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports", "value": ["PortChannel0001"]}] ** DryRun: Would apply [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports", "value": ["PortChannel0001"]}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/1", "value": "PortChannel0002"}] ** DryRun: Would apply [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/1", "value": "PortChannel0002"}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/2", "value": "PortChannel0003"}] ** DryRun: Would apply [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/2", "value": "PortChannel0003"}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/3", "value": "PortChannel0004"}] ** DryRun: Would apply [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/3", "value": "PortChannel0004"}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/stage", "value": "ingress"}] ** DryRun: Would apply [{"op": "add", "path": "/ACL_TABLE/DATAACL/stage", "value": "ingress"}] Patch Applier: Verifying patch updates are reflected on ConfigDB. Patch Applier: Patch application completed. Config Replacer: Verifying config replacement is reflected on ConfigDB. Config Replacer: Config replacement completed. Config Rollbacker: Config rollbacking completed. Config rolled back successfully. admin@vlab-01:~$ ``` **Config replace** ``` admin@vlab-01:~$ sudo config replace ~/cur.json -d -i /BGP_NEIGHBOR -i /FEATURE -i /QUEUE -i /VLAN/Vlan1000/members -i /DEVICE_METADATA -i /FLEX_COUNTER_TABLE -i /SCHEDULER -d ** DRY RUN EXECUTION ** Config Replacer: Config replacement starting. Config Replacer: Target config length: 49881. Config Replacer: Getting current config db. Config Replacer: Generating patch between target config and current config db. Config Replacer: Applying patch using 'Patch Applier'. Patch Applier: Patch application starting. Patch Applier: Patch: [{"op": "add", "path": "/ACL_TABLE/DATAACL", "value": {"policy_desc": "DATAACL", "ports": ["PortChannel0001", "PortChannel0002", "PortChannel0003", "PortChannel0004"], "stage": "ingress", "type": "L3"}}] Patch Applier: Getting current config db. Patch Applier: Simulating the target full config after applying the patch. Patch Applier: Validating target config does not have empty tables, since they do not show up in ConfigDb. Patch Applier: Sorting patch updates. Note: Below table(s) have no YANG models: BGP_PEER_RANGE, CABLE_LENGTH, CONSOLE_SWITCH, DEVICE_NEIGHBOR_METADATA, DHCP_SERVER, KDUMP, RESTAPI, SNMP, SNMP_COMMUNITY, SYSLOG_SERVER, TELEMETRY, Note: Below table(s) have no YANG models: BGP_PEER_RANGE, CABLE_LENGTH, CONSOLE_SWITCH, DEVICE_NEIGHBOR_METADATA, DHCP_SERVER, KDUMP, RESTAPI, SNMP, SNMP_COMMUNITY, SYSLOG_SERVER, TELEMETRY, Note: Below table(s) have no YANG models: BGP_PEER_RANGE, CABLE_LENGTH, CONSOLE_SWITCH, DEVICE_NEIGHBOR_METADATA, DHCP_SERVER, KDUMP, RESTAPI, SNMP, SNMP_COMMUNITY, SYSLOG_SERVER, TELEMETRY, libyang[0]: Missing required element "type" in "ACL_TABLE_LIST". (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='DATAACL']) sonic_yang(3):Data Loading Failed:Missing required element "type" in "ACL_TABLE_LIST". libyang[0]: Missing required element "type" in "ACL_TABLE_LIST". (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='DATAACL']) sonic_yang(3):Data Loading Failed:Missing required element "type" in "ACL_TABLE_LIST". libyang[0]: Missing required element "type" in "ACL_TABLE_LIST". (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='DATAACL']) sonic_yang(3):Data Loading Failed:Missing required element "type" in "ACL_TABLE_LIST". libyang[0]: Missing required element "type" in "ACL_TABLE_LIST". (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='DATAACL']) sonic_yang(3):Data Loading Failed:Missing required element "type" in "ACL_TABLE_LIST". libyang[0]: Missing required element "type" in "ACL_TABLE_LIST". (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='DATAACL']) sonic_yang(3):Data Loading Failed:Missing required element "type" in "ACL_TABLE_LIST". libyang[0]: Missing required element "type" in "ACL_TABLE_LIST". (path: /sonic-acl:sonic-acl/ACL_TABLE/ACL_TABLE_LIST[ACL_TABLE_NAME='DATAACL']) sonic_yang(3):Data Loading Failed:Missing required element "type" in "ACL_TABLE_LIST". Patch Applier: The patch was sorted into 7 changes: Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL", "value": {"type": "L3"}}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/policy_desc", "value": "DATAACL"}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports", "value": ["PortChannel0001"]}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/1", "value": "PortChannel0002"}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/2", "value": "PortChannel0003"}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/3", "value": "PortChannel0004"}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/stage", "value": "ingress"}] Patch Applier: Applying 7 changes in order: Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL", "value": {"type": "L3"}}] ** DryRun: Would apply [{"op": "add", "path": "/ACL_TABLE/DATAACL", "value": {"type": "L3"}}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/policy_desc", "value": "DATAACL"}] ** DryRun: Would apply [{"op": "add", "path": "/ACL_TABLE/DATAACL/policy_desc", "value": "DATAACL"}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports", "value": ["PortChannel0001"]}] ** DryRun: Would apply [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports", "value": ["PortChannel0001"]}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/1", "value": "PortChannel0002"}] ** DryRun: Would apply [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/1", "value": "PortChannel0002"}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/2", "value": "PortChannel0003"}] ** DryRun: Would apply [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/2", "value": "PortChannel0003"}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/3", "value": "PortChannel0004"}] ** DryRun: Would apply [{"op": "add", "path": "/ACL_TABLE/DATAACL/ports/3", "value": "PortChannel0004"}] Patch Applier: * [{"op": "add", "path": "/ACL_TABLE/DATAACL/stage", "value": "ingress"}] ** DryRun: Would apply [{"op": "add", "path": "/ACL_TABLE/DATAACL/stage", "value": "ingress"}] Patch Applier: Verifying patch updates are reflected on ConfigDB. Patch Applier: Patch application completed. Config Replacer: Verifying config replacement is reflected on ConfigDB. Config Replacer: Config replacement completed. Config replaced successfully. admin@vlab-01:~$ ``` --- config/main.py | 10 +++++ generic_config_updater/change_applier.py | 10 +++++ generic_config_updater/generic_updater.py | 26 +++++++++-- generic_config_updater/gu_common.py | 21 ++++++++- .../change_applier_test.py | 16 ++++++- .../generic_updater_test.py | 11 +++++ .../generic_config_updater/gu_common_test.py | 44 +++++++++++++++++++ 7 files changed, 130 insertions(+), 8 deletions(-) diff --git a/config/main.py b/config/main.py index f914e6465380..370958856b30 100644 --- a/config/main.py +++ b/config/main.py @@ -1164,6 +1164,10 @@ def load(filename, yes): log.log_info("'load' executing...") clicommon.run_command(command, display_cmd=True) +def print_dry_run_message(dry_run): + if dry_run: + click.secho("** DRY RUN EXECUTION **", fg="yellow", underline=True) + @config.command('apply-patch') @click.argument('patch-file-path', type=str, required=True) @click.option('-f', '--format', type=click.Choice([e.name for e in ConfigFormat]), @@ -1182,6 +1186,8 @@ def apply_patch(ctx, patch_file_path, format, dry_run, ignore_non_yang_tables, i : Path to the patch file on the file-system.""" try: + print_dry_run_message(dry_run) + with open(patch_file_path, 'r') as fh: text = fh.read() patch_as_json = json.loads(text) @@ -1214,6 +1220,8 @@ def replace(ctx, target_file_path, format, dry_run, ignore_non_yang_tables, igno : Path to the target file on the file-system.""" try: + print_dry_run_message(dry_run) + with open(target_file_path, 'r') as fh: target_config_as_text = fh.read() target_config = json.loads(target_config_as_text) @@ -1241,6 +1249,8 @@ def rollback(ctx, checkpoint_name, dry_run, ignore_non_yang_tables, ignore_path, : The checkpoint name, use `config list-checkpoints` command to see available checkpoints.""" try: + print_dry_run_message(dry_run) + GenericUpdater().rollback(checkpoint_name, verbose, dry_run, ignore_non_yang_tables, ignore_path) click.secho("Config rolled back successfully.", fg="cyan", underline=True) diff --git a/generic_config_updater/change_applier.py b/generic_config_updater/change_applier.py index 23b938381425..3786e1a3722d 100644 --- a/generic_config_updater/change_applier.py +++ b/generic_config_updater/change_applier.py @@ -55,6 +55,16 @@ def prune_empty_table(data): return data +class DryRunChangeApplier: + + def __init__(self, config_wrapper): + self.config_wrapper = config_wrapper + + + def apply(self, change): + self.config_wrapper.apply_change_to_config_db(change) + + class ChangeApplier: updater_conf = None diff --git a/generic_config_updater/generic_updater.py b/generic_config_updater/generic_updater.py index a4ea6f5ee6cf..ee7af656203d 100644 --- a/generic_config_updater/generic_updater.py +++ b/generic_config_updater/generic_updater.py @@ -5,7 +5,7 @@ DryRunConfigWrapper, PatchWrapper, genericUpdaterLogging from .patch_sorter import StrictPatchSorter, NonStrictPatchSorter, ConfigSplitter, \ TablesWithoutYangConfigSplitter, IgnorePathsFromYangConfigSplitter -from .change_applier import ChangeApplier +from .change_applier import ChangeApplier, DryRunChangeApplier CHECKPOINTS_DIR = "/etc/sonic/checkpoints" CHECKPOINT_EXT = ".cp.json" @@ -299,9 +299,13 @@ class GenericUpdateFactory: def create_patch_applier(self, config_format, verbose, dry_run, ignore_non_yang_tables, ignore_paths): self.init_verbose_logging(verbose) config_wrapper = self.get_config_wrapper(dry_run) + change_applier = self.get_change_applier(dry_run, config_wrapper) patch_wrapper = PatchWrapper(config_wrapper) patch_sorter = self.get_patch_sorter(ignore_non_yang_tables, ignore_paths, config_wrapper, patch_wrapper) - patch_applier = PatchApplier(config_wrapper=config_wrapper, patchsorter=patch_sorter, patch_wrapper=patch_wrapper) + patch_applier = PatchApplier(config_wrapper=config_wrapper, + patchsorter=patch_sorter, + patch_wrapper=patch_wrapper, + changeapplier=change_applier) if config_format == ConfigFormat.CONFIGDB: pass @@ -320,9 +324,13 @@ def create_config_replacer(self, config_format, verbose, dry_run, ignore_non_yan self.init_verbose_logging(verbose) config_wrapper = self.get_config_wrapper(dry_run) + change_applier = self.get_change_applier(dry_run, config_wrapper) patch_wrapper = PatchWrapper(config_wrapper) patch_sorter = self.get_patch_sorter(ignore_non_yang_tables, ignore_paths, config_wrapper, patch_wrapper) - patch_applier = PatchApplier(config_wrapper=config_wrapper, patchsorter=patch_sorter, patch_wrapper=patch_wrapper) + patch_applier = PatchApplier(config_wrapper=config_wrapper, + patchsorter=patch_sorter, + patch_wrapper=patch_wrapper, + changeapplier=change_applier) config_replacer = ConfigReplacer(patch_applier=patch_applier, config_wrapper=config_wrapper) if config_format == ConfigFormat.CONFIGDB: @@ -342,9 +350,13 @@ def create_config_rollbacker(self, verbose, dry_run=False, ignore_non_yang_table self.init_verbose_logging(verbose) config_wrapper = self.get_config_wrapper(dry_run) + change_applier = self.get_change_applier(dry_run, config_wrapper) patch_wrapper = PatchWrapper(config_wrapper) patch_sorter = self.get_patch_sorter(ignore_non_yang_tables, ignore_paths, config_wrapper, patch_wrapper) - patch_applier = PatchApplier(config_wrapper=config_wrapper, patchsorter=patch_sorter, patch_wrapper=patch_wrapper) + patch_applier = PatchApplier(config_wrapper=config_wrapper, + patchsorter=patch_sorter, + patch_wrapper=patch_wrapper, + changeapplier=change_applier) config_replacer = ConfigReplacer(config_wrapper=config_wrapper, patch_applier=patch_applier) config_rollbacker = FileSystemConfigRollbacker(config_wrapper = config_wrapper, config_replacer = config_replacer) @@ -363,6 +375,12 @@ def get_config_wrapper(self, dry_run): else: return ConfigWrapper() + def get_change_applier(self, dry_run, config_wrapper): + if dry_run: + return DryRunChangeApplier(config_wrapper) + else: + return ChangeApplier() + def get_patch_sorter(self, ignore_non_yang_tables, ignore_paths, config_wrapper, patch_wrapper): if not ignore_non_yang_tables and not ignore_paths: return StrictPatchSorter(config_wrapper, patch_wrapper) diff --git a/generic_config_updater/gu_common.py b/generic_config_updater/gu_common.py index ea4954ba248c..8f8f509e177f 100644 --- a/generic_config_updater/gu_common.py +++ b/generic_config_updater/gu_common.py @@ -152,9 +152,26 @@ def remove_empty_tables(self, config): return config_with_non_empty_tables class DryRunConfigWrapper(ConfigWrapper): - # TODO: implement DryRunConfigWrapper # This class will simulate all read/write operations to ConfigDB on a virtual storage unit. - pass + def __init__(self, initial_imitated_config_db = None): + super().__init__() + self.logger = genericUpdaterLogging.get_logger(title="** DryRun", print_all_to_console=True) + self.imitated_config_db = copy.deepcopy(initial_imitated_config_db) + + def apply_change_to_config_db(self, change): + self._init_imitated_config_db_if_none() + self.logger.log_notice(f"Would apply {change}") + self.imitated_config_db = change.apply(self.imitated_config_db) + + def get_config_db_as_json(self): + self._init_imitated_config_db_if_none() + return self.imitated_config_db + + def _init_imitated_config_db_if_none(self): + # if there is no initial imitated config_db and it is the first time calling this method + if self.imitated_config_db is None: + self.imitated_config_db = super().get_config_db_as_json() + class PatchWrapper: def __init__(self, config_wrapper=None): diff --git a/tests/generic_config_updater/change_applier_test.py b/tests/generic_config_updater/change_applier_test.py index b734485ffd32..55df84c855e1 100644 --- a/tests/generic_config_updater/change_applier_test.py +++ b/tests/generic_config_updater/change_applier_test.py @@ -4,7 +4,7 @@ import os import unittest from collections import defaultdict -from unittest.mock import patch +from unittest.mock import patch, Mock, call import generic_config_updater.change_applier import generic_config_updater.services_validator @@ -269,4 +269,16 @@ def test_change_apply(self, mock_set, mock_db, mock_os_sys): debug_print("all good for applier") - +class TestDryRunChangeApplier(unittest.TestCase): + def test_apply__calls_apply_change_to_config_db(self): + # Arrange + change = Mock() + config_wrapper = Mock() + applier = generic_config_updater.change_applier.DryRunChangeApplier(config_wrapper) + + # Act + applier.apply(change) + + # Assert + applier.config_wrapper.apply_change_to_config_db.assert_has_calls([call(change)]) + diff --git a/tests/generic_config_updater/generic_updater_test.py b/tests/generic_config_updater/generic_updater_test.py index 1a8151f398d3..aab2eae275d5 100644 --- a/tests/generic_config_updater/generic_updater_test.py +++ b/tests/generic_config_updater/generic_updater_test.py @@ -7,6 +7,7 @@ import generic_config_updater.generic_updater as gu import generic_config_updater.patch_sorter as ps +import generic_config_updater.change_applier as ca # import sys # sys.path.insert(0,'../../generic_config_updater') @@ -420,8 +421,11 @@ def validate_create_patch_applier(self, params, expected_decorators): self.assertIsInstance(patch_applier, gu.PatchApplier) if params["dry_run"]: self.assertIsInstance(patch_applier.config_wrapper, gu.DryRunConfigWrapper) + self.assertIsInstance(patch_applier.changeapplier, ca.DryRunChangeApplier) + self.assertIsInstance(patch_applier.changeapplier.config_wrapper, gu.DryRunConfigWrapper) else: self.assertIsInstance(patch_applier.config_wrapper, gu.ConfigWrapper) + self.assertIsInstance(patch_applier.changeapplier, ca.ChangeApplier) if params["ignore_non_yang_tables"] or params["ignore_paths"]: self.assertIsInstance(patch_applier.patchsorter, ps.NonStrictPatchSorter) @@ -451,9 +455,12 @@ def validate_create_config_replacer(self, params, expected_decorators): if params["dry_run"]: self.assertIsInstance(config_replacer.config_wrapper, gu.DryRunConfigWrapper) self.assertIsInstance(config_replacer.patch_applier.config_wrapper, gu.DryRunConfigWrapper) + self.assertIsInstance(config_replacer.patch_applier.changeapplier, ca.DryRunChangeApplier) + self.assertIsInstance(config_replacer.patch_applier.changeapplier.config_wrapper, gu.DryRunConfigWrapper) else: self.assertIsInstance(config_replacer.config_wrapper, gu.ConfigWrapper) self.assertIsInstance(config_replacer.patch_applier.config_wrapper, gu.ConfigWrapper) + self.assertIsInstance(config_replacer.patch_applier.changeapplier, ca.ChangeApplier) if params["ignore_non_yang_tables"] or params["ignore_paths"]: self.assertIsInstance(config_replacer.patch_applier.patchsorter, ps.NonStrictPatchSorter) @@ -482,11 +489,15 @@ def validate_create_config_rollbacker(self, params, expected_decorators): self.assertIsInstance(config_rollbacker.config_replacer.config_wrapper, gu.DryRunConfigWrapper) self.assertIsInstance( config_rollbacker.config_replacer.patch_applier.config_wrapper, gu.DryRunConfigWrapper) + self.assertIsInstance(config_rollbacker.config_replacer.patch_applier.changeapplier, ca.DryRunChangeApplier) + self.assertIsInstance( + config_rollbacker.config_replacer.patch_applier.changeapplier.config_wrapper, gu.DryRunConfigWrapper) else: self.assertIsInstance(config_rollbacker.config_wrapper, gu.ConfigWrapper) self.assertIsInstance(config_rollbacker.config_replacer.config_wrapper, gu.ConfigWrapper) self.assertIsInstance( config_rollbacker.config_replacer.patch_applier.config_wrapper, gu.ConfigWrapper) + self.assertIsInstance(config_rollbacker.config_replacer.patch_applier.changeapplier, ca.ChangeApplier) if params["ignore_non_yang_tables"] or params["ignore_paths"]: self.assertIsInstance(config_rollbacker.config_replacer.patch_applier.patchsorter, ps.NonStrictPatchSorter) diff --git a/tests/generic_config_updater/gu_common_test.py b/tests/generic_config_updater/gu_common_test.py index 56cebe786b94..7d0c9124bce9 100644 --- a/tests/generic_config_updater/gu_common_test.py +++ b/tests/generic_config_updater/gu_common_test.py @@ -7,6 +7,50 @@ from .gutest_helpers import create_side_effect_dict, Files import generic_config_updater.gu_common as gu_common +class TestDryRunConfigWrapper(unittest.TestCase): + def test_get_config_db_as_json__returns_imitated_config_db(self): + # Arrange + config_wrapper = gu_common.DryRunConfigWrapper(Files.CONFIG_DB_AS_JSON) + expected = Files.CONFIG_DB_AS_JSON + + # Act + actual = config_wrapper.get_config_db_as_json() + + # Assert + self.assertDictEqual(expected, actual) + + def test_get_sonic_yang_as_json__returns_imitated_config_db_as_yang(self): + # Arrange + config_wrapper = gu_common.DryRunConfigWrapper(Files.CONFIG_DB_AS_JSON) + expected = Files.SONIC_YANG_AS_JSON + + # Act + actual = config_wrapper.get_sonic_yang_as_json() + + # Assert + self.assertDictEqual(expected, actual) + + def test_apply_change_to_config_db__multiple_calls__changes_imitated_config_db(self): + # Arrange + imitated_config_db = Files.CONFIG_DB_AS_JSON + config_wrapper = gu_common.DryRunConfigWrapper(imitated_config_db) + + changes = [gu_common.JsonChange(jsonpatch.JsonPatch([{'op':'remove', 'path':'/VLAN'}])), + gu_common.JsonChange(jsonpatch.JsonPatch([{'op':'remove', 'path':'/ACL_TABLE'}])), + gu_common.JsonChange(jsonpatch.JsonPatch([{'op':'remove', 'path':'/PORT'}])) + ] + + expected = imitated_config_db + for change in changes: + # Act + config_wrapper.apply_change_to_config_db(change) + + actual = config_wrapper.get_config_db_as_json() + expected = change.apply(expected) + + # Assert + self.assertDictEqual(expected, actual) + class TestConfigWrapper(unittest.TestCase): def setUp(self): self.config_wrapper_mock = gu_common.ConfigWrapper()