feat: public preview support for microsoft.azuremonitor.containers.metrics in ARC clusters (managed prometheus) #227

bragi92 · 2023-05-17T22:38:49Z

This checklist is used to make sure that common guidelines for a pull request are followed.

Related command

az k8s-extension create --name azuremonitor-metrics -g kaveesharc -c kaveesharc -t connectedClusters --extension-type Microsoft.AzureMonitor.Containers.Metrics --configuration-settings azure-monitor-workspace-resource-id="{amw_id}" AzureMonitorMetrics.KubeStateMetrics.MetricAnnotationsAllowList="pods=[k8s-annotation-1,k8s-annotation-n]"

az k8s-extension delete --name azuremonitor-metrics -g kaveesharc -c kaveesharc -t connectedClusters

Here is a link to the Managed Cluster PR : Azure/azure-cli#26201

General Guidelines

Have you run azdev style <YOUR_EXT> locally? (pip install azdev required)
Have you run python scripts/ci/test_index.py -q locally?

For new extensions:

My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish

There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update src/index.json automatically.
You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify src/index.json.

NarayanThiru · 2023-05-18T00:39:04Z

.gitignore

@@ -88,6 +88,7 @@ venv/
 ENV/
 env/
 env27/
+azenv/


azenv

Why is this needed? Can you revert this change? #Closed

NarayanThiru · 2023-05-18T00:39:51Z

src/k8s-extension/HISTORY.rst

@@ -6,6 +6,7 @@ Release History
 1.4.1
 ++++++++++++++++++
 * microsoft.azureml.kubernetes: Fix sslSecret parameter in update operation
+* microsoft.azuremonitor.containers.metrics : public preview support for managed prometheus in ARC clusters



Create a new version 1.4.2 and add this comment #Closed

Its okay to add it in the version '1.4.1' itself as the most recent version is 1.4.0 . The history was updated in another pr but was not released.

fyi @NarayanThiru, pr link --> https://github.com/AzureArcForKubernetes/azure-cli-extensions/pull/224/files

NarayanThiru · 2023-05-18T00:44:10Z

src/k8s-extension/setup.py

@@ -33,7 +33,7 @@
 # TODO: Add any additional SDK dependencies here
 DEPENDENCIES = []

-VERSION = "1.4.0"
+VERSION = "1.4.1"


Looks like the previous update did not increment the version - fyi @bavneetsingh16 .
@bragi92, pl. change this to 1.4.2, which should have been the version for this update #Closed

1.4.1 is good, it is not yet released

NarayanThiru · 2023-05-18T00:54:40Z

...-extension/azext_k8s_extension/partner_extensions/azuremonitormetrics/azuremonitorprofile.py

+        raise CLIError("Azure China Cloud is not supported for the Azure Monitor Metrics extension")
+
+    if cloud_name.lower() == "azureusgovernment":
+        if configuration_settings is not None and 'grafana-resource-id' in configuration_settings:


'grafana-resource-id'

Is this case sensitive check? IOW, if this key was input in different case, will this still work? #Closed

NarayanThiru · 2023-05-18T00:59:18Z

src/k8s-extension/azext_k8s_extension/partner_extensions/azuremonitormetrics/helper.py

+            isAlertsManagementRpRegistered = True
+    if isInsightsRpRegistered is False:
+        headers = ['User-Agent=arc-azuremonitormetrics.register_insights_rp']
+        post_request(cmd, subscription_id, "microsoft.insights", headers)


post_request(cmd, subscription_id, "microsoft.insights", headers)

Since you are doing this without the customer knowing, it'd be good to print a message that this registration is being done.
Moreover, if the customer does not have sufficient permission to register this RP for that Sub, this will fail. At that time, the message above will help explain the context #Closed

NarayanThiru · 2023-05-18T00:59:53Z

src/k8s-extension/azext_k8s_extension/partner_extensions/azuremonitormetrics/helper.py

+        if cluster_rp.lower() == consts.HYBRIDCONTAINERSERVICE_RP:
+            resource = resources.get_by_id(cluster_resource_id, consts.HYBRIDCONTAINERSERVICE_API_VERSION)
+        else:
+            resource = resources.get_by_id(cluster_resource_id, '2020-01-01-preview')


'2020-01-01-preview'

Move this to constants.py #Closed

NarayanThiru · 2023-05-18T01:00:23Z

src/k8s-extension/azext_k8s_extension/partner_extensions/azuremonitormetrics/helper.py

+def get_cluster_region(cmd, cluster_rp, subscription_id, cluster_resource_group_name, cluster_name, cluster_type):
+    cluster_region = ''
+    resources = cf_resources(cmd.cli_ctx, subscription_id)
+    cluster_resource_id = '/subscriptions/{0}/resourceGroups/{1}/providers/{2}/{3}/{4}'.format(


/subscriptions/{0}/resourceGroups/{1}/providers/{2}/{3}/{4}

Consider moving these Uri templates to constants too #Closed

NarayanThiru · 2023-05-18T01:05:02Z

src/k8s-extension/azext_k8s_extension/partner_extensions/azuremonitormetrics/amg/link.py

+            }
+        })
+        headers = ['User-Agent=arc-azuremonitormetrics.add_role_assignment']
+        send_raw_request(cmd.cli_ctx, "PUT", roleDefinitionURI, body=association_body, headers=headers)


send_raw_request(cmd.cli_ctx, "PUT", roleDefinitionURI, body=association_body, headers=headers)

Role Assignment requires Ownership level permission in the Subscription. What if the user creating the extension does not have that level of permission? Is that a expected/valid scenario? Will they get proper error explaining what went wrong/? #Closed

Yes, this is a valid scenario as it only happens when a customer is trying to link a grafana workspace to their AMW. If it fails, I have a error message being displayed which tells them what needs to be done to correct it but the overall onboarding will succeed as the extension installation will succeed without issue and it will even send data to the AMW.

Thanks for explaining, Kaveesh!

NarayanThiru

bavneetsingh16 · 2023-05-19T16:36:02Z

can you please add test case for your extensions in this folder

bavneetsingh16

LGTM

* Kubernetes Data Protection Extension CLI (AzureArcForKubernetes#173) * First draft for Data Protection K8s backup extension (Pending internal review) * Removing tracing * Minor changes to improve azdev style * Internal PR review feedback Co-authored-by: Rishabh Raj <[email protected]> * {AKS - ARC} fix: Update DCR creation to Clusters resource group instead of workspace (AzureArcForKubernetes#175) * fix: Update DCR creation to Clusters resource group instead of workspace * . * . * casing check * Add self-signed cert to fix PR gate for azureml extension * adding the api version to the operation definition in the client factory * bump k8s-extension version to 1.3.6 * adding tests for all 4 extension types calls * adding to test config file * updating the api version for extension types to be the correct version expected by the service * add test case for flux extension (AzureArcForKubernetes#184) * bump k8s-extension version to 1.3.6 * bump k8s-extension version to 1.3.6 * adding upstream test for extension types * updating history.rst * [Dapr] Prompt user for existing Dapr installation during extension create (AzureArcForKubernetes#188) * Add more validations and user prompt for existing installation scenario Signed-off-by: Shubham Sharma <[email protected]> * Add Dapr test' Signed-off-by: Shubham Sharma <[email protected]> * Handle stateful set Signed-off-by: Shubham Sharma <[email protected]> * Update default handling Signed-off-by: Shubham Sharma <[email protected]> * Fix HA handling Signed-off-by: Shubham Sharma <[email protected]> * Add placement service todo Signed-off-by: Shubham Sharma <[email protected]> * Add non-interactive mode Signed-off-by: Shubham Sharma <[email protected]> * Fix lint Signed-off-by: Shubham Sharma <[email protected]> * Update tests Signed-off-by: Shubham Sharma <[email protected]> * Reset configuration for StatefulSet during k8s upgrade Signed-off-by: Shubham Sharma <[email protected]> * Fix lint Signed-off-by: Shubham Sharma <[email protected]> * Retrigger tests Signed-off-by: Shubham Sharma <[email protected]> * Add changes to manage ha and placement params Signed-off-by: Shubham Sharma <[email protected]> * Update message Signed-off-by: Shubham Sharma <[email protected]> * nits Signed-off-by: Shubham Sharma <[email protected]> Signed-off-by: Shubham Sharma <[email protected]> * bump k8s-extension version to 1.3.7 * [Dapr] Disable applying CRDs during a downgrade (AzureArcForKubernetes#193) * Add logging Signed-off-by: Shubham Sharma <[email protected]> * Lint Signed-off-by: Shubham Sharma <[email protected]> * Update log Signed-off-by: Shubham Sharma <[email protected]> * Revert applyCrds when not downgrading Signed-off-by: Shubham Sharma <[email protected]> * Update logic for removing hooks.applyCrds Signed-off-by: Shubham Sharma <[email protected]> * Revert logic Signed-off-by: Shubham Sharma <[email protected]> * Handle explicit hooks configuration Signed-off-by: Shubham Sharma <[email protected]> * Update comment Signed-off-by: Shubham Sharma <[email protected]> * re-trigger pipeline Signed-off-by: Shubham Sharma <[email protected]> Signed-off-by: Shubham Sharma <[email protected]> * ContainerInsights extension - Add dataCollectionSettings configuration settings (AzureArcForKubernetes#200) * data collection settings * add support for dataCollectionSettings * fix indention * avoid duplicate use of json loads * remove whitespaces * fix pr feedback * Upgrade Python version from 3.6 to 3.7 (AzureArcForKubernetes#203) * Upgrade Python version from 3.6 to 3.10 Upgrade to 3.10 for the job that runs Wheel, PyLint, Flake, etc., since 3.6 is not supported anymore by hosted-agent-software. * Upgrade to Python 3.10 from 3.6 Upgrade to 3.10 as 3.6 is not supported * Switch PyLink to 1.9.4 Switch PyLink to 1.9.4 from 1.9.5, as 1.9.5 is not supported with Python 3.10 * Use Python 3.7 for Static Analysis Use 3.7, as 3.10 does not support certain properties used by astpeephole.py that is used by Static Analysis tools * Try unpinned version of PyLint PyLint 1.9.5 doesn't work with Python 3.7. So, trying to see if it automatically pulls the latest compatible version. * Run pylint as a separate command * Update pylintrc (AzureArcForKubernetes#204) * Update pylintrc * Update k8s-custom-pipelines.yml * Disable PyLint (AzureArcForKubernetes#205) Disable PyLint for now, as the new version has breaking changes and requires lot more fixes * Disable PyLint on CI scripts * Fixes for script errors * Upgrade Static Analysis Python version Upgrade the Python version for Static Analysis to 3.10, from 3.7, now that PyLint is disabled * Try 3.9, as 3.10 has breaking changes for Flake8 * Remove version pinning for flake8 Try Python 3.10, without pinning flake8 to a version * Update k8s-custom-pipelines.yml * Use Python 3.8.1 & flake8 6.0.0 * Use Python 3.8 instead of 3.8.1 * Update k8s-custom-pipelines.yml * Update .flake8 Update to reflect breaking change in flake8 6.0 * Update source_code_static_analysis.py Scope static analysis tools to only k8s-extension module's source in our branch. * Update k8s-custom-pipelines.yml * Update k8s-custom-pipelines.yml * Update k8s-custom-pipelines.yml * Update pool name in StaticAnalysis To mirror what is in main of azure-cli-extensions * Update k8s-custom-pipelines.yml * Fix indentation * Update k8s-custom-pipelines.yml * Update k8s-custom-pipelines.yml * Revert changes * Revert changes * Revert changes to source_code_static_analysis.py * Update source_code_static_analysis.py * Revert changes * Use Ubuntu 20.4 for BuiltTestPublish stage * Switch to ubuntu-20.04 from latest Co-authored-by: Rishik Hombal <[email protected]> * [Dapr] Do not apply CRD hook when version is unchanged or auto-upgrade is being disabled (AzureArcForKubernetes#201) * Update logic Signed-off-by: Shubham Sharma <[email protected]> * re-trigger pipeline Signed-off-by: Shubham Sharma <[email protected]> * re-trigger pipeline Signed-off-by: Shubham Sharma <[email protected]> Signed-off-by: Shubham Sharma <[email protected]> Co-authored-by: NarayanThiru <[email protected]> * add dummy key for amalogs as well * bump k8s-extension version to 1.3.8 * Adding GA api version 2022-11-01 exposing isSystemExtension and support for plan info * Seperate args for plan name, product and publisher * updating cassete file * updating HISTORY.rst * Deprecate longer parameter names when accepting config settings (AzureArcForKubernetes#213) Co-authored-by: deeksha345 <[email protected]> * Release 1.3.9 * make containerinsights dcr name consistent (AzureArcForKubernetes#211) Co-authored-by: Bavneet Singh <[email protected]> * [Dapr] Update version comparison logic to use semver based comparison (AzureArcForKubernetes#219) * Update semver comparison Signed-off-by: Shubham Sharma <[email protected]> * Add log Signed-off-by: Shubham Sharma <[email protected]> --------- Signed-off-by: Shubham Sharma <[email protected]> * bump k8s-extension version to 1.4.0 (AzureArcForKubernetes#220) * Revert "bump k8s-extension version to 1.4.0 (AzureArcForKubernetes#220)" (AzureArcForKubernetes#222) This reverts commit ffb8a95. * [k8s-extension] Update extension CLI to v1.4.0 * update release history * fix openservice mesh cli testcase issue * Zetia/fix ssl secret flag (AzureArcForKubernetes#224) * fix bug: update operation doesn't respect sslSecret parameter * fix bug: update operation doesn't respect sslSecret parameter * fix typo * feat: public preview support for microsoft.azuremonitor.containers.metrics in ARC clusters (managed prometheus) (AzureArcForKubernetes#227) * remove redundant extension test (AzureArcForKubernetes#230) * ci MSI default for arc cluster (AzureArcForKubernetes#231) * bump k8s-extension version to 1.4.2 * ContainerInsights extension - Extend dataCollectionSettings config settings with streams field (AzureArcForKubernetes#232) * extend containerinsights datacollection settings with streams field * bug fix * fix lint issues * fix pr feedback * fix pr feedback * fix lint error * Generated files for 2023-05-01-preview * Support for 2023-05-01-preview * Rename get to show * Added ExtensionType api test cases * ContainerInsights extension - Extend dataCollectionSettings with containerlogv2 (AzureArcForKubernetes#237) * Fix for Liniting issues * Fixing test cases * comment failing test cases * [k8s-extension] add kind tag in DCR creation (AzureArcForKubernetes#240) * Use semver package (AzureArcForKubernetes#241) Signed-off-by: Shubham Sharma <[email protected]> * Reverting commented test cases * Add support to skip provisioning of prerequisites for Azure Monitor K8s extensions (AzureArcForKubernetes#234) * {ARC} fix: update logic to sanitize cluster name for dc* objects (AzureArcForKubernetes#242) * Fix osm-arc version check for CI tags (AzureArcForKubernetes#244) Signed-off-by: nshankar <[email protected]> Co-authored-by: nshankar <[email protected]> * New cassette file * Remove unused propeties from table format * bump k8s-extension version 1.4.3 * Add old commands back with deprecated status * Fix linting issues * Reverting changes for extensions type api * change the location for test runs and arc clusters * [k8s-extension] create new cli release - v1.4.3 (AzureArcForKubernetes#250) * Revert "[k8s-extension] create new cli release - v1.4.3 (AzureArcForKubernetes#250)" (AzureArcForKubernetes#251) This reverts commit 584815d. * [k8s-extension] Update extension CLI to v1.4.3 * Drop relay sdk (AzureArcForKubernetes#254) * update readme * remove useless snippets (AzureArcForKubernetes#256) * [k8s-extension] Update extension CLI to v1.4.4 --------- Signed-off-by: Shubham Sharma <[email protected]> Signed-off-by: nshankar <[email protected]> Co-authored-by: Rishabh Raj <[email protected]> Co-authored-by: Rishabh Raj <[email protected]> Co-authored-by: bragi92 <[email protected]> Co-authored-by: Yue Yu <[email protected]> Co-authored-by: Deeksha Sharma <[email protected]> Co-authored-by: deeksha345 <[email protected]> Co-authored-by: Shubham Sharma <[email protected]> Co-authored-by: Bavneet Singh <[email protected]> Co-authored-by: Ganga Mahesh Siddem <[email protected]> Co-authored-by: NarayanThiru <[email protected]> Co-authored-by: Rishik Hombal <[email protected]> Co-authored-by: Amol Agrawal <[email protected]> Co-authored-by: Amol Agrawal <[email protected]> Co-authored-by: Arif Lakhani <[email protected]> Co-authored-by: Arif-lakhani <[email protected]> Co-authored-by: Zeliang Tian <[email protected]> Co-authored-by: Long Wan <[email protected]> Co-authored-by: ms-hujia <[email protected]> Co-authored-by: Niranjan Shankar <[email protected]> Co-authored-by: nshankar <[email protected]> Co-authored-by: necusjz <[email protected]>

bragi92 added 3 commits May 17, 2023 15:25

feat: public preview support for azure monitor metrics in ARC clusters

3d8b230

remove azenv

16e5c42

version update

19ceb76

bragi92 requested review from NarayanThiru, bavneetsingh16, deeksha345 and anagg929 as code owners May 17, 2023 22:38

style

ee6bacb

NarayanThiru reviewed May 18, 2023

View reviewed changes

pr feedback

22d6c3e

NarayanThiru previously approved these changes May 19, 2023

View reviewed changes

test file

96bc2e8

bragi92 dismissed NarayanThiru’s stale review via 96bc2e8 May 19, 2023 17:27

bragi92 added 2 commits May 19, 2023 10:46

add file in build pipeline

4779964

missed }

de79cf6

bavneetsingh16 approved these changes May 19, 2023

View reviewed changes

bavneetsingh16 merged commit db556d2 into AzureArcForKubernetes:k8s-extension/public May 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: public preview support for microsoft.azuremonitor.containers.metrics in ARC clusters (managed prometheus) #227

feat: public preview support for microsoft.azuremonitor.containers.metrics in ARC clusters (managed prometheus) #227

bragi92 commented May 17, 2023 •

edited

Loading

NarayanThiru May 18, 2023 •

edited

Loading

NarayanThiru May 18, 2023 •

edited

Loading

bavneetsingh16 May 18, 2023

NarayanThiru May 18, 2023 •

edited

Loading

bavneetsingh16 May 18, 2023

NarayanThiru May 18, 2023 •

edited

Loading

NarayanThiru May 18, 2023 •

edited

Loading

NarayanThiru May 18, 2023 •

edited

Loading

NarayanThiru May 18, 2023 •

edited

Loading

NarayanThiru May 18, 2023 •

edited

Loading

bragi92 May 18, 2023

NarayanThiru May 19, 2023

NarayanThiru left a comment

bavneetsingh16 commented May 19, 2023

bavneetsingh16 left a comment

@@ @@ -88,6 +88,7 @@ venv/ @@
               ENV/
               env/
               env27/
+              azenv/

feat: public preview support for microsoft.azuremonitor.containers.metrics in ARC clusters (managed prometheus) #227

feat: public preview support for microsoft.azuremonitor.containers.metrics in ARC clusters (managed prometheus) #227

Conversation

bragi92 commented May 17, 2023 • edited Loading

Related command

General Guidelines

About Extension Publish

NarayanThiru May 18, 2023 • edited Loading

Choose a reason for hiding this comment

NarayanThiru May 18, 2023 • edited Loading

Choose a reason for hiding this comment

bavneetsingh16 May 18, 2023

Choose a reason for hiding this comment

NarayanThiru May 18, 2023 • edited Loading

Choose a reason for hiding this comment

bavneetsingh16 May 18, 2023

Choose a reason for hiding this comment

NarayanThiru May 18, 2023 • edited Loading

Choose a reason for hiding this comment

NarayanThiru May 18, 2023 • edited Loading

Choose a reason for hiding this comment

NarayanThiru May 18, 2023 • edited Loading

Choose a reason for hiding this comment

NarayanThiru May 18, 2023 • edited Loading

Choose a reason for hiding this comment

NarayanThiru May 18, 2023 • edited Loading

Choose a reason for hiding this comment

bragi92 May 18, 2023

Choose a reason for hiding this comment

NarayanThiru May 19, 2023

Choose a reason for hiding this comment

NarayanThiru left a comment

Choose a reason for hiding this comment

bavneetsingh16 commented May 19, 2023

bavneetsingh16 left a comment

Choose a reason for hiding this comment

bragi92 commented May 17, 2023 •

edited

Loading

NarayanThiru May 18, 2023 •

edited

Loading

NarayanThiru May 18, 2023 •

edited

Loading

NarayanThiru May 18, 2023 •

edited

Loading

NarayanThiru May 18, 2023 •

edited

Loading

NarayanThiru May 18, 2023 •

edited

Loading

NarayanThiru May 18, 2023 •

edited

Loading

NarayanThiru May 18, 2023 •

edited

Loading

NarayanThiru May 18, 2023 •

edited

Loading