[k8s-configuration] Update sourceControlConfiguration SDK Models to T…

…rack2 (Azure#3676) * Create pull.yml * Update pull.yml * Update azure-pipelines.yml * Initial commit of k8s-extension * Update CODEOWNERS * Update azure-pipelines.yml * Create pull.yml * Update pull.yml * Update pull.yml * Update pipelines file * Update k8s-configuration name * Update test script params * Update pipeline file * Remove codeowners * Update pipelines file * Update CODEOWNERS * Update private preview pipelines * Remove open service mesh from public release * Update pipeline files * Update custom pipelines files * Add publish step to k8s-configuration * Update pipeline to publish extension * Update public extension pipeline * Change condition variable * Update pipeline naming * Add version to public preview/private preview * Update pipelines * Add different testing based on private branch * Add annotations to extension model * Update k8s-custom-pipelines.yml * Update SDKs with Updated Swagger Spec for 2020-07-01-preview (#13) * Update sdks with updated swagger spec * Update version and history rst * Reorder release history timeline * Fix ExtensionInstanceForCreate for import * remove py2 bdist support * Add custom table formatting * Remove unnecessary files * Fix style issues * Fix branch based on comments * Update identity piece manually * Don't handle defaults at the CLI level * Remove defaults from CLI client * Check null target namespace with namespace scope * Update style * Add cassandra operator and location to model * Stage Public Version of k8s-extension 0.2.0 for official release (#15) * Create pull.yml * Update pull.yml * Update azure-pipelines.yml * Initial commit of k8s-extension * Update pipelines file * Update CODEOWNERS * Update private preview pipelines * Remove open service mesh from public release * Update pipeline files * Update public extension pipeline * Change condition variable * Add version to public preview/private preview * Update pipelines * Add different testing based on private branch * Add annotations to extension model * Update k8s-custom-pipelines.yml * Update SDKs with Updated Swagger Spec for 2020-07-01-preview (#13) * Update sdks with updated swagger spec * Update version and history rst * Reorder release history timeline * Fix ExtensionInstanceForCreate for import * remove py2 bdist support * Add custom table formatting * Remove unnecessary files * Fix style issues * Fix branch based on comments * Update identity piece manually * Don't handle defaults at the CLI level * Remove defaults from CLI client * Check null target namespace with namespace scope * Update style * Add cassandra operator and location to model Co-authored-by: [email protected] <Action - Fork Sync> * Remove custom pipelines file * Update extension description, remove private const * Update pipeline file * Disable check ref docs * Disable refs docs * Update to include better create warning logs and remove update context (#20) * Update to include better create warning logs and remove update context * Remove help text for update * Fix spelling error * Update message * Fix k8s-extension conflict with private version * Fix style errors * Fix filename * add customization for microsoft.azureml.kubernetes (#23) * add customization for microsoft.azureml.kubernetes * Update release history Co-authored-by: Yue Yu <[email protected]> Co-authored-by: jonathan-innis <[email protected]> * Add E2E Testing from Separate branch into internal code (#26) * Add internal e2e testing * Change to testing folder * Inference CLI validation for Scoring FE (#24) * cli validation starter * added the call to the fe validation function * nodeport validation not required * test fix Co-authored-by: Jonathan Innis <[email protected]> * legal warning added (#27) * Remove deprecated method logger.warn * Update k8s-custom-pipelines.yml for Azure Pipelines * Update k8s-custom-pipelines.yml for Azure Pipelines * Add Azure Defender to E2E testing (#28) * Add azure defender testing to e2e * Remove the debug flag * Add configuration testing * Fix pipeline failures * Make test script more intuitive * Remove parameter from testing * Add some debug * Fix wrong location for k8s config whl * Fix pip install upgrade issue * Fix pip install upgrade issue * Add Check for Provider Registration and Refactor (#19) * Add check for provider registration and refactor * Fix bug in checking registration * Add license header to utils * Update private key check and error messaging * Update based on refactoring * Fix failing tests * Add provider registration check * Create a test for uppercase url, address comments * Add blank line to fix style check * Testing increase to ubuntu-latest * Update k8s-configuration Models to Track2 (#63) * Update models to track2 * Increase k8s-configuration version number * Update kind version * Change error to warning because of DSA failure * Remove unneeded files Co-authored-by: [email protected] <Action - Fork Sync> Co-authored-by: yuyue9284 <[email protected]> Co-authored-by: Yue Yu <[email protected]> Co-authored-by: Lia Kazakova <[email protected]>
AzureArcForKubernetes · Jul 23, 2021 · e298f57 · e298f57
1 parent 6d11a74
commit e298f57
Show file tree

Hide file tree

Showing 27 changed files with 2,033 additions and 1,201 deletions.
diff --git a/src/k8s-configuration/HISTORY.rst b/src/k8s-configuration/HISTORY.rst
@@ -3,6 +3,14 @@
 Release History
 ===============
 
+1.1.0
+++++++++++++++++++
+* Update sourceControlConfiguration resource models to Track2
+
+1.0.1
+++++++++++++++++++
+* Add provider registration check
+
 1.0.0
 ++++++++++++++++++
 * Support api-version 2021-03-01

diff --git a/src/k8s-configuration/azext_k8s_configuration/_client_factory.py b/src/k8s-configuration/azext_k8s_configuration/_client_factory.py
@@ -3,13 +3,18 @@
 # Licensed under the MIT License. See License.txt in the project root for license information.
 # --------------------------------------------------------------------------------------------
 
+from azure.cli.core.commands.client_factory import get_mgmt_service_client
 
-def cf_k8s_configuration(cli_ctx, *_):
 
-    from azure.cli.core.commands.client_factory import get_mgmt_service_client
+def cf_k8s_configuration(cli_ctx, *_):
     from azext_k8s_configuration.vendored_sdks import SourceControlConfigurationClient
     return get_mgmt_service_client(cli_ctx, SourceControlConfigurationClient)
 
 
 def cf_k8s_configuration_operation(cli_ctx, _):
     return cf_k8s_configuration(cli_ctx).source_control_configurations
+
+
+def _resource_providers_client(cli_ctx):
+    from azure.mgmt.resource import ResourceManagementClient
+    return get_mgmt_service_client(cli_ctx, ResourceManagementClient).providers
diff --git a/src/k8s-configuration/azext_k8s_configuration/_consts.py b/src/k8s-configuration/azext_k8s_configuration/_consts.py
@@ -0,0 +1,7 @@
+# --------------------------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for license information.
+# --------------------------------------------------------------------------------------------
+
+PROVIDER_NAMESPACE = 'Microsoft.KubernetesConfiguration'
+REGISTERED = "Registered"
diff --git a/src/k8s-configuration/azext_k8s_configuration/_params.py b/src/k8s-configuration/azext_k8s_configuration/_params.py
@@ -13,7 +13,7 @@
 )
 
 from azure.cli.core.commands.validators import get_default_location_from_resource_group
-from ._validators import validate_configuration_type, validate_operator_namespace, validate_operator_instance_name
+from ._validators import _validate_configuration_type, _validate_operator_namespace, _validate_operator_instance_name
 
 
 def load_arguments(self, _):
@@ -38,7 +38,7 @@ def load_arguments(self, _):
                    arg_type=get_enum_type(['namespace', 'cluster']),
                    help='''Specify scope of the operator to be 'namespace' or 'cluster' ''')
         c.argument('configuration_type',
-                   validator=validate_configuration_type,
+                   validator=_validate_configuration_type,
                    arg_type=get_enum_type(['sourceControlConfiguration']),
                    help='Type of the configuration')
         c.argument('enable_helm_operator',
@@ -60,11 +60,11 @@ def load_arguments(self, _):
         c.argument('operator_instance_name',
                    arg_group="Operator",
                    help='Instance name of the Operator',
-                   validator=validate_operator_instance_name)
+                   validator=_validate_operator_instance_name)
         c.argument('operator_namespace',
                    arg_group="Operator",
                    help='Namespace in which to install the Operator',
-                   validator=validate_operator_namespace)
+                   validator=_validate_operator_namespace)
         c.argument('operator_type',
                    arg_group="Operator",
                    help='''Type of the operator. Valid value is 'flux' ''')

diff --git a/src/k8s-configuration/azext_k8s_configuration/_utils.py b/src/k8s-configuration/azext_k8s_configuration/_utils.py
@@ -0,0 +1,61 @@
+# --------------------------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for license information.
+# --------------------------------------------------------------------------------------------
+
+import base64
+from azure.cli.core.azclierror import MutuallyExclusiveArgumentError, InvalidArgumentValueError
+
+
+def _get_cluster_type(cluster_type):
+    if cluster_type.lower() == 'connectedclusters':
+        return 'Microsoft.Kubernetes'
+    # Since cluster_type is an enum of only two values, if not connectedClusters, it will be managedClusters.
+    return 'Microsoft.ContainerService'
+
+
+def _fix_compliance_state(config):
+    # If we get Compliant/NonCompliant as compliance_sate, change them before returning
+    if config.compliance_status.compliance_state.lower() == 'noncompliant':
+        config.compliance_status.compliance_state = 'Failed'
+    elif config.compliance_status.compliance_state.lower() == 'compliant':
+        config.compliance_status.compliance_state = 'Installed'
+
+    return config
+
+
+def _get_data_from_key_or_file(key, filepath):
+    if key != '' and filepath != '':
+        raise MutuallyExclusiveArgumentError(
+            'Error! Both textual key and key filepath cannot be provided',
+            'Try providing the file parameter without providing the plaintext parameter')
+    data = ''
+    if filepath != '':
+        data = _read_key_file(filepath)
+    elif key != '':
+        data = key
+    return data
+
+
+def _read_key_file(path):
+    try:
+        with open(path, "r") as myfile:  # user passed in filename
+            data_list = myfile.readlines()  # keeps newline characters intact
+            data_list_len = len(data_list)
+            if (data_list_len) <= 0:
+                raise Exception("File provided does not contain any data")
+            raw_data = ''.join(data_list)
+        return _to_base64(raw_data)
+    except Exception as ex:
+        raise InvalidArgumentValueError(
+            'Error! Unable to read key file specified with: {0}'.format(ex),
+            'Verify that the filepath specified exists and contains valid utf-8 data') from ex
+
+
+def _from_base64(base64_str):
+    return base64.b64decode(base64_str)
+
+
+def _to_base64(raw_data):
+    bytes_data = raw_data.encode('utf-8')
+    return base64.b64encode(bytes_data).decode('utf-8')
diff --git a/src/k8s-configuration/azext_k8s_configuration/_validators.py b/src/k8s-configuration/azext_k8s_configuration/_validators.py
@@ -4,34 +4,48 @@
 # --------------------------------------------------------------------------------------------
 
 import re
-from azure.cli.core.azclierror import InvalidArgumentValueError
+import io
+from azure.cli.core.azclierror import InvalidArgumentValueError, MutuallyExclusiveArgumentError
+
+from knack.log import get_logger
+from azext_k8s_configuration._client_factory import _resource_providers_client
+from azext_k8s_configuration._utils import _from_base64
+import azext_k8s_configuration._consts as consts
+from urllib.parse import urlparse
+from paramiko.hostkeys import HostKeyEntry
+from paramiko.ed25519key import Ed25519Key
+from paramiko.ssh_exception import SSHException
+from Crypto.PublicKey import RSA, ECC, DSA
+
+
+logger = get_logger(__name__)
 
 
 # Parameter-Level Validation
-def validate_configuration_type(configuration_type):
+def _validate_configuration_type(configuration_type):
     if configuration_type.lower() != 'sourcecontrolconfiguration':
         raise InvalidArgumentValueError(
             'Invalid configuration-type',
             'Try specifying the valid value "sourceControlConfiguration"')
 
 
-def validate_operator_namespace(namespace):
+def _validate_operator_namespace(namespace):
     if namespace.operator_namespace:
-        __validate_k8s_name(namespace.operator_namespace, "--operator-namespace", 23)
+        _validate_k8s_name(namespace.operator_namespace, "--operator-namespace", 23)
 
 
-def validate_operator_instance_name(namespace):
+def _validate_operator_instance_name(namespace):
     if namespace.operator_instance_name:
-        __validate_k8s_name(namespace.operator_instance_name, "--operator-instance-name", 23)
+        _validate_k8s_name(namespace.operator_instance_name, "--operator-instance-name", 23)
 
 
 # Create Parameter Validation
-def validate_configuration_name(configuration_name):
-    __validate_k8s_name(configuration_name, "--name", 63)
+def _validate_configuration_name(configuration_name):
+    _validate_k8s_name(configuration_name, "--name", 63)
 
 
 # Helper
-def __validate_k8s_name(param_value, param_name, max_len):
+def _validate_k8s_name(param_value, param_name, max_len):
     if len(param_value) > max_len:
         raise InvalidArgumentValueError(
             'Error! Invalid {0}'.format(param_name),
@@ -44,3 +58,86 @@ def __validate_k8s_name(param_value, param_name, max_len):
         raise InvalidArgumentValueError(
             'Error! Invalid {0}'.format(param_name),
             'Parameter {0} can only contain lowercase alphanumeric characters and hyphens'.format(param_name))
+
+
+def _validate_url_with_params(repository_url, ssh_private_key_set, known_hosts_contents_set, https_auth_set):
+    scheme = urlparse(repository_url).scheme
+
+    if scheme.lower() in ('http', 'https'):
+        if ssh_private_key_set:
+            raise MutuallyExclusiveArgumentError(
+                'Error! An --ssh-private-key cannot be used with an http(s) url',
+                'Verify the url provided is a valid ssh url and not an http(s) url')
+        if known_hosts_contents_set:
+            raise MutuallyExclusiveArgumentError(
+                'Error! --ssh-known-hosts cannot be used with an http(s) url',
+                'Verify the url provided is a valid ssh url and not an http(s) url')
+        if not https_auth_set and scheme == 'https':
+            logger.warning('Warning! https url is being used without https auth params, ensure the repository '
+                           'url provided is not a private repo')
+    else:
+        if https_auth_set:
+            raise MutuallyExclusiveArgumentError(
+                'Error! https auth (--https-user and --https-key) cannot be used with a non-http(s) url',
+                'Verify the url provided is a valid http(s) url and not an ssh url')
+
+
+def _validate_known_hosts(knownhost_data):
+    try:
+        knownhost_str = _from_base64(knownhost_data).decode('utf-8')
+    except Exception as ex:
+        raise InvalidArgumentValueError(
+            'Error! ssh known_hosts is not a valid utf-8 base64 encoded string',
+            'Verify that the string provided safely decodes into a valid utf-8 format') from ex
+    lines = knownhost_str.split('\n')
+    for line in lines:
+        line = line.strip(' ')
+        line_len = len(line)
+        if (line_len == 0) or (line[0] == "#"):
+            continue
+        try:
+            host_key = HostKeyEntry.from_line(line)
+            if not host_key:
+                raise Exception('not enough fields found in known_hosts line')
+        except Exception as ex:
+            raise InvalidArgumentValueError(
+                'Error! ssh known_hosts provided in wrong format',
+                'Verify that all lines in the known_hosts contents are provided in a valid sshd(8) format') from ex
+
+
+def _validate_private_key(ssh_private_key_data):
+    try:
+        RSA.import_key(_from_base64(ssh_private_key_data))
+        return
+    except ValueError:
+        try:
+            ECC.import_key(_from_base64(ssh_private_key_data))
+            return
+        except ValueError:
+            try:
+                DSA.import_key(_from_base64(ssh_private_key_data))
+                return
+            except ValueError:
+                try:
+                    key_obj = io.StringIO(_from_base64(ssh_private_key_data).decode('utf-8'))
+                    Ed25519Key(file_obj=key_obj)
+                    return
+                except SSHException:
+                    raise InvalidArgumentValueError(
+                        'Error! --ssh-private-key provided in invalid format',
+                        'Verify the key provided is a valid PEM-formatted key of type RSA, ECC, DSA, or Ed25519')
+
+
+# pylint: disable=broad-except
+def _validate_cc_registration(cmd):
+    try:
+        rp_client = _resource_providers_client(cmd.cli_ctx)
+        registration_state = rp_client.get(consts.PROVIDER_NAMESPACE).registration_state
+
+        if registration_state.lower() != consts.REGISTERED.lower():
+            logger.warning("'Source Control Configuration' cannot be used because '%s' provider has not been "
+                           "registered. More details for registering this provider can be found here - "
+                           "https://aka.ms/RegisterKubernetesConfigurationProvider", consts.PROVIDER_NAMESPACE)
+    except Exception:
+        logger.warning("Unable to fetch registration state of '%s' provider. "
+                       "Failed to enable 'source control configuration' feature...", consts.PROVIDER_NAMESPACE)