[k8s-extension] Release v0.6.1 with Identity Fix (#3703)

* Create pull.yml * Update pull.yml * Update azure-pipelines.yml * Initial commit of k8s-extension * Update pipelines file * Update CODEOWNERS * Update private preview pipelines * Remove open service mesh from public release * Update pipeline files * Update public extension pipeline * Change condition variable * Add version to public preview/private preview * Update pipelines * Add different testing based on private branch * Add annotations to extension model * Update k8s-custom-pipelines.yml * Update SDKs with Updated Swagger Spec for 2020-07-01-preview (#13) * Update sdks with updated swagger spec * Update version and history rst * Reorder release history timeline * Fix ExtensionInstanceForCreate for import * remove py2 bdist support * Add custom table formatting * Remove unnecessary files * Fix style issues * Fix branch based on comments * Update identity piece manually * Don't handle defaults at the CLI level * Remove defaults from CLI client * Check null target namespace with namespace scope * Update style * Add cassandra operator and location to model * Stage Public Version of k8s-extension 0.2.0 for official release (#15) * Create pull.yml * Update pull.yml * Update azure-pipelines.yml * Initial commit of k8s-extension * Update pipelines file * Update CODEOWNERS * Update private preview pipelines * Remove open service mesh from public release * Update pipeline files * Update public extension pipeline * Change condition variable * Add version to public preview/private preview * Update pipelines * Add different testing based on private branch * Add annotations to extension model * Update k8s-custom-pipelines.yml * Update SDKs with Updated Swagger Spec for 2020-07-01-preview (#13) * Update sdks with updated swagger spec * Update version and history rst * Reorder release history timeline * Fix ExtensionInstanceForCreate for import * remove py2 bdist support * Add custom table formatting * Remove unnecessary files * Fix style issues * Fix branch based on comments * Update identity piece manually * Don't handle defaults at the CLI level * Remove defaults from CLI client * Check null target namespace with namespace scope * Update style * Add cassandra operator and location to model Co-authored-by: [email protected] <Action - Fork Sync> * Remove custom pipelines file * Update extension description, remove private const * Update pipeline file * Disable refs docs * Update to include better create warning logs and remove update context (#20) * Update to include better create warning logs and remove update context * Remove help text for update * Fix spelling error * Update message * Fix k8s-extension conflict with private version * Fix style errors * Fix filename * add customization for microsoft.azureml.kubernetes (#23) * add customization for microsoft.azureml.kubernetes * Update release history Co-authored-by: Yue Yu <[email protected]> Co-authored-by: jonathan-innis <[email protected]> * Add E2E Testing from Separate branch into internal code (#26) * Add internal e2e testing * Change to testing folder * Inference CLI validation for Scoring FE (#24) * cli validation starter * added the call to the fe validation function * nodeport validation not required * test fix Co-authored-by: Jonathan Innis <[email protected]> * legal warning added (#27) * Remove deprecated method logger.warn * Update k8s-custom-pipelines.yml for Azure Pipelines * Update k8s-custom-pipelines.yml for Azure Pipelines * Add Azure Defender to E2E testing (#28) * Add azure defender testing to e2e * Remove the debug flag * Add configuration testing * Fix pipeline failures * Make test script more intuitive * Remove parameter from testing * Fix wrong location for k8s config whl * Fix pip upgrade issue * Fix pip install upgrade issue * Fix pip install issue * delete resurce in testcase (#29) Co-authored-by: Yue Yu <[email protected]> Co-authored-by: Jonathan Innis <[email protected]> * Check Provider is Registered with Subscription Before Making Requests (#18) * Add check for KubernetesConfiguration * Disable pylint and rename * Update provider registration link * Update version * Remove extra blank line * Fix bug in import * only validate scoring fe when inference is enabled (#31) * only validate scoring fe when inference is enabled * Fix versioning Co-authored-by: Yue Yu <[email protected]> Co-authored-by: jonathan-innis <[email protected]> * Provider registration case insensitive * do not validate against scoring fe if inference is not enabled. (#33) * do not validate against scoring fe if inference is not enabled. * add inference enabled scenario * refine * increase sleeping time * fix Co-authored-by: Yue Yu <[email protected]> Co-authored-by: Jonathan Innis <[email protected]> * Add OSM as Public Preview Extension (#34) * Add OSM as public preview extension * Add osm testing * Add release train to tests * Fix failing osm test * Upgrade pip in integration testing * Remove ununsed import * Fix release train check in update * Parallelize E2E Testing (#36) * Add OSM as public preview extension * Add osm testing * Update test logic to parallelize * Fix test success checking * Parallelize extension testing * Better error checking logic * Fix azureml deletion * Fix private build (#40) * change amlk8s to amlarc (#42) Co-authored-by: Yue Yu <[email protected]> * Servicebus client model changes (#44) * Servicebus client model changes * Fix testing script * Update history file and pipeline * Update min cli core version for track 2 updates * Read SSL cert and key from files (#38) * first sketch of the change fixes removed extra blank lines changes regarding param renaming added ssl tests added more detail to the unit test additional import moved pem files out of public folder fixed import chenged import changed import unit tests fix unit test fix fixed unit tests fixed unit test unit test fix changes int test cert and key * test protected config * fix test typo * temporary changes reverted * fixing tests * fixed file paths * removed accidentally added file * changes according to review comments * more changes according to review comments * changes according to review comments Co-authored-by: Jonathan Innis <[email protected]> * Upgrade release version * Liakaz/inference read ssl from file (#47) * first sketch of the change fixes removed extra blank lines changes regarding param renaming added ssl tests added more detail to the unit test additional import moved pem files out of public folder fixed import chenged import changed import unit tests fix unit test fix fixed unit tests fixed unit test unit test fix changes int test cert and key * test protected config * fix test typo * temporary changes reverted * fixing tests * fixed file paths * removed accidentally added file * changes according to review comments * more changes according to review comments * changes according to review comments * fixed decode error * renamed the experimental param Co-authored-by: Jonathan Innis <[email protected]> * Fix style issues (#51) * Fixed scoring fe related extension param names (#49) * fixed scoring fe related extension params * bug fix and style fixes * variable rename * fixed the error type * set cluster to prod by default * Add distro validation for osm-arc (#50) * Add distro validation for osm-arc * fixed indentation * Fix linting * Resolve comments * Add unit test * fix lint Co-authored-by: Jonathan Innis <[email protected]> * Add distro validation for osm-arc (#50) * Add distro validation for osm-arc * fixed indentation * Fix linting * Resolve comments * Add unit test * fix lint Co-authored-by: Jonathan Innis <[email protected]> * Add distro validation for osm-arc (#53) removed release-train logic * Add Custom Delete Logic for Partners (#54) * Add custom delete logic * Fix failing unit tests * Add warning message when deleting amlarc extension (#55) * add warning message * fix indentation * Update release version * Remove Pyhelm from OSM customization (#58) * Fix OSM pyhelm bug * Debug bootstrap error * Update release message * Remove pyhelm dependency * Update tests to only check extensionconfig creation (#61) * Update tests to only check extensionconfig creation * Single set of CRUD for AzureML * Debug logs for connectedk8s * Increase open service mesh version number * Update k8s-extension Models to Track2 (#64) * Update k8s-extension models to track2 * Add debug for failed cleanup * Increase version number * Exit 0 on failed cleanup * Fix identity in wrong place in model (#66) * Readd osm-arc distro validation (#62) * Add distro validation for osm-arc removed release-train logic * Readd osm_arc distro validation * Fix style * Rm space * Edit test * Fixed tests and error logic * Remove dependency * Add delete method Co-authored-by: Jonathan Innis <[email protected]> * Don't Send Identity Headers If In DF (#67) * Don't send identity for clusters in dogfood * Add location to model for identity * Add identity validation to testing * Remove unneeded files Co-authored-by: [email protected] <Action - Fork Sync> Co-authored-by: yuyue9284 <[email protected]> Co-authored-by: Yue Yu <[email protected]> Co-authored-by: Lia Kazakova <[email protected]> Co-authored-by: Niranjan Shankar <[email protected]> Co-authored-by: jingyizhu99 <[email protected]>
Azure · Jul 27, 2021 · e7fa0b9 · e7fa0b9
1 parent fa20709
commit e7fa0b9
Show file tree

Hide file tree

Showing 8 changed files with 67 additions and 54 deletions.
diff --git a/src/k8s-extension/HISTORY.rst b/src/k8s-extension/HISTORY.rst
@@ -3,6 +3,12 @@
 Release History
 ===============
 
+0.6.1
+++++++++++++++++++
+* Remove sending identity for clusters in Dogfood
+* Provide fix for getting tested distros for microsoft.openservicemesh
+* Add location to model for identity
+
 0.6.0
 ++++++++++++++++++
 * Update extension resource models to Track2

diff --git a/src/k8s-extension/azext_k8s_extension/consts.py b/src/k8s-extension/azext_k8s_extension/consts.py
@@ -8,3 +8,4 @@
 EXTENSION_PACKAGE_NAME = "azext_k8s_extension"
 PROVIDER_NAMESPACE = 'Microsoft.KubernetesConfiguration'
 REGISTERED = "Registered"
+DF_RM_ENDPOINT = 'https://api-dogfood.resources.windows-int.net/'
diff --git a/src/k8s-extension/azext_k8s_extension/custom.py b/src/k8s-extension/azext_k8s_extension/custom.py
@@ -138,7 +138,8 @@ def create_k8s_extension(cmd, client, resource_group_name, cluster_name, name, c
     validate_cc_registration(cmd)
 
     # Create identity, if required
-    if create_identity:
+    # We don't create the identity if we are in DF
+    if create_identity and not __is_dogfood_cluster(cmd):
         extension_instance.identity, extension_instance.location = \
             __create_identity(cmd, resource_group_name, cluster_name, cluster_type, cluster_rp)
 
@@ -292,3 +293,7 @@ def __get_config_settings_from_file(file_path):
         raise Exception("File {} is empty".format(file_path))
 
     return settings
+
+
+def __is_dogfood_cluster(cmd):
+    return cmd.cli_ctx.cloud.endpoints.resource_manager == consts.DF_RM_ENDPOINT
diff --git a/src/k8s-extension/azext_k8s_extension/partner_extensions/OpenServiceMesh.py b/src/k8s-extension/azext_k8s_extension/partner_extensions/OpenServiceMesh.py
@@ -14,6 +14,7 @@
 
 from packaging import version
 import yaml
+import requests
 
 from ..partner_extensions import PartnerExtensionModel
 
@@ -32,8 +33,6 @@
 
 
 class OpenServiceMesh(PartnerExtensionModel):
-    CHART_NAME = "osm-arc"
-    CHART_LOCATION = "https://azure.github.io/osm-azure"
 
     def Create(self, cmd, client, resource_group_name, cluster_name, name, cluster_type, extension_type,
                scope, auto_upgrade_minor_version, release_train, version, target_namespace,
@@ -67,7 +66,7 @@ def Create(self, cmd, client, resource_group_name, cluster_name, name, cluster_t
         # NOTE-2: Return a valid ExtensionInstance object, Instance name and flag for Identity
         create_identity = False
 
-        # _validate_tested_distro(cmd, resource_group_name, cluster_name, version)
+        _validate_tested_distro(cmd, resource_group_name, cluster_name, version)
 
         extension_instance = ExtensionInstance(
             extension_type=extension_type,
@@ -105,58 +104,52 @@ def Delete(self, client, resource_group_name, cluster_name, name, cluster_type):
         pass
 
 
-# def _validate_tested_distro(cmd, cluster_resource_group_name, cluster_name, extension_version):
+def _validate_tested_distro(cmd, cluster_resource_group_name, cluster_name, extension_version):
 
-#     field_unavailable_error = '\"testedDistros\" field unavailable for version {0} of microsoft.openservicemesh, ' \
-#         'cannot determine if this Kubernetes distribution has been properly tested'.format(extension_version)
+    field_unavailable_error = '\"testedDistros\" field unavailable for version {0} of microsoft.openservicemesh, ' \
+        'cannot determine if this Kubernetes distribution has been properly tested'.format(extension_version)
 
-#     if version.parse(str(extension_version)) <= version.parse("0.8.3"):
-#         logger.warning(field_unavailable_error)
-#         return
+    if version.parse(str(extension_version)) <= version.parse("0.8.3"):
+        logger.warning(field_unavailable_error)
+        return
 
-#     subscription_id = get_subscription_id(cmd.cli_ctx)
-#     resources = cf_resources(cmd.cli_ctx, subscription_id)
+    subscription_id = get_subscription_id(cmd.cli_ctx)
+    resources = cf_resources(cmd.cli_ctx, subscription_id)
 
-#     cluster_resource_id = '/subscriptions/{0}/resourceGroups/{1}/providers/Microsoft.Kubernetes' \
-#         '/connectedClusters/{2}'.format(subscription_id, cluster_resource_group_name, cluster_name)
+    cluster_resource_id = '/subscriptions/{0}/resourceGroups/{1}/providers/Microsoft.Kubernetes' \
+        '/connectedClusters/{2}'.format(subscription_id, cluster_resource_group_name, cluster_name)
 
-#     resource = resources.get_by_id(cluster_resource_id, '2020-01-01-preview')
-#     cluster_distro = resource.properties['distribution'].lower()
+    resource = resources.get_by_id(cluster_resource_id, '2020-01-01-preview')
+    cluster_distro = resource.properties['distribution'].lower()
 
-#     if cluster_distro == "general":
-#         logger.warning('Unable to determine if distro has been tested for microsoft.openservicemesh, '
-#                        'kubernetes distro: \"general\"')
-#         return
+    if cluster_distro == "general":
+        logger.warning('Unable to determine if distro has been tested for microsoft.openservicemesh, '
+                       'kubernetes distro: \"general\"')
+        return
 
-#     tested_distros = _get_tested_distros(extension_version)
+    tested_distros = _get_tested_distros(extension_version)
 
-#     if tested_distros is None:
-#         logger.warning(field_unavailable_error)
-#     elif cluster_distro not in tested_distros.split():
-#         logger.warning('Untested kubernetes distro for microsoft.openservicemesh, Kubernetes distro is %s',
-#                        cluster_distro)
+    if tested_distros is None:
+        logger.warning(field_unavailable_error)
+    elif cluster_distro not in tested_distros.split():
+        logger.warning('Untested kubernetes distro for microsoft.openservicemesh, Kubernetes distro is %s',
+                       cluster_distro)
 
 
-# def _get_tested_distros(chart_version):
+def _get_tested_distros(chart_version):
 
-#     try:
-#         chart_arc = ChartBuilder({
-#             "name": OpenServiceMesh.CHART_NAME,
-#             "version": str(chart_version),
-#             "source": {
-#                 "type": "repo",
-#                 "location": OpenServiceMesh.CHART_LOCATION
-#             }
-#         })
-#     except VersionError:
-#         raise InvalidArgumentValueError(
-#             "Invalid version '{}' for microsoft.openservicemesh".format(chart_version)
-#         )
+    chart_url = 'https://raw.githubusercontent.com/Azure/osm-azure/' \
+        'v{0}/charts/osm-arc/values.yaml'.format(chart_version)
+    chart_request = requests.get(url=chart_url)
 
-#     values = chart_arc.get_values()
-#     values_yaml = yaml.load(values.raw, Loader=yaml.FullLoader)
+    if chart_request.status_code == 404:
+        raise InvalidArgumentValueError(
+            "Invalid version '{}' for microsoft.openservicemesh".format(chart_version)
+        )
+
+    values_yaml = yaml.load(chart_request.text, Loader=yaml.FullLoader)
 
-#     try:
-#         return values_yaml['OpenServiceMesh']['testedDistros']
-#     except KeyError:
-#         return None
+    try:
+        return values_yaml['OpenServiceMesh']['testedDistros']
+    except KeyError:
+        return None
diff --git a/src/k8s-extension/azext_k8s_extension/tests/latest/test_open_service_mesh.py b/src/k8s-extension/azext_k8s_extension/tests/latest/test_open_service_mesh.py
@@ -9,15 +9,14 @@
 import unittest
 
 from azure.cli.core.azclierror import InvalidArgumentValueError
-# from azext_k8s_extension.partner_extensions.OpenServiceMesh import _get_tested_distros
+from azext_k8s_extension.partner_extensions.OpenServiceMesh import _get_tested_distros
 
 TEST_DIR = os.path.abspath(os.path.join(os.path.abspath(__file__), '..'))
 
 class TestOpenServiceMesh(unittest.TestCase):
     def test_bad_osm_arc_version(self):
-        # version = "0.7.1"
-        # err = "Invalid version \'" + str(version) + "\' for microsoft.openservicemesh"
-        # with self.assertRaises(InvalidArgumentValueError) as argError:
-        #     _get_tested_distros(version)
-        # self.assertEqual(str(argError.exception), err)
-        pass
+        version = "0.7.1"
+        err = "Invalid version \'" + str(version) + "\' for microsoft.openservicemesh"
+        with self.assertRaises(InvalidArgumentValueError) as argError:
+            _get_tested_distros(version)
+        self.assertEqual(str(argError.exception), err)
diff --git a/src/k8s-extension/azext_k8s_extension/vendored_sdks/models/_models.py b/src/k8s-extension/azext_k8s_extension/vendored_sdks/models/_models.py
@@ -279,6 +279,8 @@ class ExtensionInstance(ProxyResource):
     :param identity: The identity of the configuration.
     :type identity:
      ~azure.mgmt.kubernetesconfiguration.v2020_07_01_preview.models.ConfigurationIdentity
+    :param location: Location of resource type
+    :type location: str
     """
 
     _validation = {
@@ -311,6 +313,7 @@ class ExtensionInstance(ProxyResource):
         'last_status_time': {'key': 'properties.lastStatusTime', 'type': 'str'},
         'error_info': {'key': 'properties.errorInfo', 'type': 'ErrorDefinition'},
         'identity': {'key': 'identity', 'type': 'ConfigurationIdentity'},
+        'location': {'key': 'location', 'type': 'str'},
     }
 
     def __init__(
@@ -332,6 +335,7 @@ def __init__(
         self.last_status_time = None
         self.error_info = None
         self.identity = kwargs.get('identity', None)
+        self.location = kwargs.get('location', None)
 
 
 class ExtensionInstancesList(msrest.serialization.Model):

diff --git a/src/k8s-extension/azext_k8s_extension/vendored_sdks/models/_models_py3.py b/src/k8s-extension/azext_k8s_extension/vendored_sdks/models/_models_py3.py
@@ -297,6 +297,8 @@ class ExtensionInstance(ProxyResource):
     :param identity: The identity of the configuration.
     :type identity:
      ~azure.mgmt.kubernetesconfiguration.v2020_07_01_preview.models.ConfigurationIdentity
+    :param location: Location of resource type
+    :type location: str
     """
 
     _validation = {
@@ -329,6 +331,7 @@ class ExtensionInstance(ProxyResource):
         'last_status_time': {'key': 'properties.lastStatusTime', 'type': 'str'},
         'error_info': {'key': 'properties.errorInfo', 'type': 'ErrorDefinition'},
         'identity': {'key': 'identity', 'type': 'ConfigurationIdentity'},
+        'location': {'key': 'location', 'type': 'str'},
     }
 
     def __init__(
@@ -344,6 +347,7 @@ def __init__(
         configuration_protected_settings: Optional[Dict[str, str]] = None,
         statuses: Optional[List["ExtensionStatus"]] = None,
         identity: Optional["ConfigurationIdentity"] = None,
+        location: Optional[str] = None,
         **kwargs
     ):
         super(ExtensionInstance, self).__init__(system_data=system_data, **kwargs)
@@ -361,6 +365,7 @@ def __init__(
         self.last_status_time = None
         self.error_info = None
         self.identity = identity
+        self.location = location
 
 
 class ExtensionInstancesList(msrest.serialization.Model):

diff --git a/src/k8s-extension/setup.py b/src/k8s-extension/setup.py
@@ -32,7 +32,7 @@
 # TODO: Add any additional SDK dependencies here
 DEPENDENCIES = []
 
-VERSION = "0.6.0"
+VERSION = "0.6.1"
 
 with open('README.rst', 'r', encoding='utf-8') as f:
     README = f.read()