[k8s-configuration] Release v1.4.0 with S3 Bucket as Source support (A…

…zure#4285)

* Create pull.yml

* Update pull.yml

* Update azure-pipelines.yml

* Initial commit of k8s-extension

* Update CODEOWNERS

* Update azure-pipelines.yml

* Create pull.yml

* Update pull.yml

* Update pull.yml

* Update pipelines file

* Update k8s-configuration name

* Update test script params

* Update pipeline file

* Remove codeowners

* Update pipelines file

* Update CODEOWNERS

* Update private preview pipelines

* Remove open service mesh from public release

* Update pipeline files

* Update custom pipelines files

* Add publish step to k8s-configuration

* Update pipeline to publish extension

* Update public extension pipeline

* Change condition variable

* Update pipeline naming

* Add version to public preview/private preview

* Update pipelines

* Add different testing based on private branch

* Add annotations to extension model

* Update k8s-custom-pipelines.yml

* Update SDKs with Updated Swagger Spec for 2020-07-01-preview (#13)

* Update sdks with updated swagger spec

* Update version and history rst

* Reorder release history timeline

* Fix ExtensionInstanceForCreate for import

* remove py2 bdist support

* Add custom table formatting

* Remove unnecessary files

* Fix style issues

* Fix branch based on comments

* Update identity piece manually

* Don't handle defaults at the CLI level

* Remove defaults from CLI client

* Check null target namespace with namespace scope

* Update style

* Add cassandra operator and location to model

* Stage Public Version of k8s-extension 0.2.0 for official release (#15)

* Create pull.yml

* Update pull.yml

* Update azure-pipelines.yml

* Initial commit of k8s-extension

* Update pipelines file

* Update CODEOWNERS

* Update private preview pipelines

* Remove open service mesh from public release

* Update pipeline files

* Update public extension pipeline

* Change condition variable

* Add version to public preview/private preview

* Update pipelines

* Add different testing based on private branch

* Add annotations to extension model

* Update k8s-custom-pipelines.yml

* Update SDKs with Updated Swagger Spec for 2020-07-01-preview (#13)

* Update sdks with updated swagger spec

* Update version and history rst

* Reorder release history timeline

* Fix ExtensionInstanceForCreate for import

* remove py2 bdist support

* Add custom table formatting

* Remove unnecessary files

* Fix style issues

* Fix branch based on comments

* Update identity piece manually

* Don't handle defaults at the CLI level

* Remove defaults from CLI client

* Check null target namespace with namespace scope

* Update style

* Add cassandra operator and location to model

Co-authored-by: [email protected] <Action - Fork Sync>

* Remove custom pipelines file

* Update extension description, remove private const

* Update pipeline file

* Disable check ref docs

* Disable refs docs

* Update to include better create warning logs and remove update context (#20)

* Update to include better create warning logs and remove update context

* Remove help text for update

* Fix spelling error

* Update message

* Fix k8s-extension conflict with private version

* Fix style errors

* Fix filename

* add customization for microsoft.azureml.kubernetes (#23)

* add customization for microsoft.azureml.kubernetes

* Update release history

Co-authored-by: Yue Yu <[email protected]>
Co-authored-by: jonathan-innis <[email protected]>

* Add E2E Testing from Separate branch into internal code (#26)

* Add internal e2e testing

* Change to testing folder

* Inference CLI validation for Scoring FE (#24)

* cli validation starter

* added the call to the fe validation function

* nodeport validation not required

* test fix

Co-authored-by: Jonathan Innis <[email protected]>

* legal warning added (#27)

* Remove deprecated method logger.warn

* Update k8s-custom-pipelines.yml for Azure Pipelines

* Update k8s-custom-pipelines.yml for Azure Pipelines

* Add Azure Defender to E2E testing (#28)

* Add azure defender testing to e2e

* Remove the debug flag

* Add configuration testing

* Fix pipeline failures

* Make test script more intuitive

* Remove parameter from testing

* Add some debug

* Fix wrong location for k8s config whl

* Fix pip install upgrade issue

* Fix pip install upgrade issue

* Add Check for Provider Registration and Refactor (#19)

* Add check for provider registration and refactor

* Fix bug in checking registration

* Add license header to utils

* Update private key check and error messaging

* Update based on refactoring

* Fix failing tests

* Add provider registration check

* Create a test for uppercase url, address comments

* Add blank line to fix style check

* Testing increase to ubuntu-latest

* Update k8s-configuration Models to Track2 (#63)

* Update models to track2

* Increase k8s-configuration version number

* Update kind version

* Change error to warning because of DSA failure

* Upgrade helm operator chart version (#75)

* Pin helm version

* Bump version

* Migrate pipeline (#90)

* Disable updates on configuration tests (#89)

* Release k8s-configuration v1.2.0 for Flux v2 Public Preview (#86)

* Scaffold out the k8s-config package

* Base implementation of CLI commands

* Add create scenario and cleanup in consts

* Add help text to commands, params

* Add other clients to client factory

* Automatically installing the flux extension

* Move flux and extension into modules

* Updated the versioned sdks

* Push working command for testing

* Update to multi api versioned sdk

* Support other extension methods

* Fix nullity check

* Add source control provider

* Add scc commands

* Add defer logic for create with cache

* Use default extension with identity

* Fix identity creation

* Add kustomization caching

* Add formatters

* Add scc provider

* Add help text for k8s-config fluxv1

* Add help text for extension

* Allow force delete of extension and fluxconfiguration

* Add location to the extension model

* Update with latest from k8s-extension

* Add k8s-config testing

* Add license header

* Fix all style issues

* Update codeowners file

* Validate data before checking cluster compliance

* No kustomizations warning

* Fix identity issue in 2020-07-01

* Fix k8s regex

* Fix configuration name regex validation

* Fix name length validation

* Adding some validation warnings

* Add protected settings to request

* Exclude private test path

* Add suspend functionality

* Add correct values to build Kustomization

* Add no_wait

* Fix style issues

* Use base64 encoded httpsUser

* Fix formatting error and base64 encoding error

* Fix style issues

* Fix force

* Updated help text

* Style fixes

* Increase namespace maximum len

* Add managed cluster support to k8s-config

* Custom confirmation when prune is enabled

* Add flux commands to existing k8s-configuration

* Remove extension provider from CLI

* Fix style issues

* Override extension variables

* Strip newlines from known_hosts file

* Update help text and validators

* Strip newlines from known hosts

* Add provisioning state check for flux extension

* Pin helm version

* Remove validation from create command

* Add patch support with new SDK

* Add implementation for CRUD of source and kustomization

* Fix errors on patch

* Fix some bugs in patching properties

* Add fixes for patch in k8s-configuraiton

* Change duration formatting in table output

* Add validation and conversion for durations

* Bump verison and fix typo

* Fix bug with dependencies

* Fix linter and style issues

* Fix delete prune check

* Add flux testing

* Create separate jobs for scenarios

* Update error text

* Fix filepath suggestion from CLI team

* Fix unneeded file edit

* Add a Deployed Object List to the Flux CLI (#91)

* Enable a deployed object list on the CLI

* Show detail when extension install fails

* Bump version

* Update deployed object format (#93)

* Fix help text for consistency

* Enable Bucket Support in the CLI (#92)

* Update vendored_sdks

* Enable source kind generation factories and use kwargs to pass to provider

* Add better validation logic to source generator

* Move away from classes in the provider directory

* Fix style issues using black auto-formatter

* Fix linter failures

* Update identity with api version and rp in same function

* Bucket Testing for E2E Testing (#96)

* Bucket testing

* Add switching kind test to update CLI

* Bump version

* Remove unneeded files

Co-authored-by: [email protected] <Action - Fork Sync>
Co-authored-by: yuyue9284 <[email protected]>
Co-authored-by: Yue Yu <[email protected]>
Co-authored-by: Lia Kazakova <[email protected]>

src/k8s-configuration/HISTORY.rst

@@ -3,6 +3,10 @@
 Release History
 ===============
 
+1.4.0
+++++++++++++++++++
+* Add `--kind bucket` for creation of S3 bucket as source for fluxConfigurations
+
 1.3.0
 ++++++++++++++++++
 * Add `deployed-object` command group for showing deployed Flux objects from configuration

src/k8s-configuration/azext_k8s_configuration/__init__.py

@@ -9,23 +9,25 @@
 
 
 class K8sConfigurationCommandsLoader(AzCommandsLoader):
-
     def __init__(self, cli_ctx=None):
         from azure.cli.core.commands import CliCommandType
         from azext_k8s_configuration._client_factory import k8s_configuration_client
+
         k8s_configuration_custom = CliCommandType(
-            operations_tmpl='azext_k8s_configuration.custom#{}',
-            client_factory=k8s_configuration_client)
-        super().__init__(cli_ctx=cli_ctx,
-                         custom_command_type=k8s_configuration_custom)
+            operations_tmpl="azext_k8s_configuration.custom#{}",
+            client_factory=k8s_configuration_client,
+        )
+        super().__init__(cli_ctx=cli_ctx, custom_command_type=k8s_configuration_custom)
 
     def load_command_table(self, args):
         from azext_k8s_configuration.commands import load_command_table
+
         load_command_table(self, args)
         return self.command_table
 
     def load_arguments(self, command):
         from azext_k8s_configuration._params import load_arguments
+
         load_arguments(self, command)
 
 

src/k8s-configuration/azext_k8s_configuration/_client_factory.py

@@ -10,39 +10,51 @@
 
 def k8s_configuration_client(cli_ctx, **kwargs):
     from azext_k8s_configuration.vendored_sdks import SourceControlConfigurationClient
+
     return get_mgmt_service_client(cli_ctx, SourceControlConfigurationClient, **kwargs)
 
 
 def k8s_configuration_fluxconfig_client(cli_ctx, *_):
-    return k8s_configuration_client(cli_ctx, api_version=consts.FLUXCONFIG_API_VERSION).flux_configurations
+    return k8s_configuration_client(
+        cli_ctx, api_version=consts.FLUXCONFIG_API_VERSION
+    ).flux_configurations
 
 
 def k8s_configuration_sourcecontrol_client(cli_ctx, *_):
     return k8s_configuration_client(
-        cli_ctx,
-        api_version=consts.SOURCE_CONTROL_API_VERSION
+        cli_ctx, api_version=consts.SOURCE_CONTROL_API_VERSION
     ).source_control_configurations
 
 
 def k8s_configuration_extension_client(cli_ctx, *_):
-    return k8s_configuration_client(cli_ctx, api_version=consts.EXTENSION_API_VERSION).extensions
+    return k8s_configuration_client(
+        cli_ctx, api_version=consts.EXTENSION_API_VERSION
+    ).extensions
 
 
 def resource_providers_client(cli_ctx):
     from azure.mgmt.resource import ResourceManagementClient
+
     return get_mgmt_service_client(cli_ctx, ResourceManagementClient).providers
 
 
 def cf_resource_groups(cli_ctx, subscription_id=None):
-    return get_mgmt_service_client(cli_ctx, ResourceType.MGMT_RESOURCE_RESOURCES,
-                                   subscription_id=subscription_id).resource_groups
+    return get_mgmt_service_client(
+        cli_ctx, ResourceType.MGMT_RESOURCE_RESOURCES, subscription_id=subscription_id
+    ).resource_groups
 
 
 def cf_resources(cli_ctx, subscription_id=None):
-    return get_mgmt_service_client(cli_ctx, ResourceType.MGMT_RESOURCE_RESOURCES,
-                                   subscription_id=subscription_id).resources
+    return get_mgmt_service_client(
+        cli_ctx, ResourceType.MGMT_RESOURCE_RESOURCES, subscription_id=subscription_id
+    ).resources
 
 
 def cf_log_analytics(cli_ctx, subscription_id=None):
-    from azure.mgmt.loganalytics import LogAnalyticsManagementClient  # pylint: disable=no-name-in-module
-    return get_mgmt_service_client(cli_ctx, LogAnalyticsManagementClient, subscription_id=subscription_id)
+    from azure.mgmt.loganalytics import (
+        LogAnalyticsManagementClient,
+    )  # pylint: disable=no-name-in-module
+
+    return get_mgmt_service_client(
+        cli_ctx, LogAnalyticsManagementClient, subscription_id=subscription_id
+    )

src/k8s-configuration/azext_k8s_configuration/_help.py

@@ -86,6 +86,14 @@
           --name myconfig --scope cluster --namespace my-namespace \\
           --kind git --url https://github.com/Azure/arc-k8s-demo \\
           --branch main --kustomization name=my-kustomization
+      - name: Create a Kubernetes v2 Flux Configuration with Bucket Source Kind
+        text: |-
+          az k8s-configuration flux create --resource-group my-resource-group \\
+          --cluster-name mycluster --cluster-type connectedClusters \\
+          --name myconfig --scope cluster --namespace my-namespace \\
+          --kind bucket --url https://bucket-provider.minio.io \\
+          --bucket-name my-bucket --kustomization name=my-kustomization \\
+          --access-key my-access-key --secret-key my-secret-key
 """
 
 helps[